INFO: Started server process [1] INFO: Waiting for application startup. INFO: Application startup complete. INFO: Uvicorn running on http://0.0.0.0:8000 (Press CTRL+C to quit) INFO: 172.20.0.1:52828 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52832 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38608 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51004 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34800 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57026 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40526 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40542 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54528 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54530 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32860 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43384 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37906 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60908 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35822 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35826 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45474 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45478 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37494 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36598 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54298 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36430 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54272 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54284 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34304 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34308 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35624 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41152 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36090 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49960 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55816 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55822 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50538 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50542 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57386 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32958 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58058 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55762 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58552 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58554 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58236 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58238 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39408 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34260 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57084 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43262 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50880 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50886 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57902 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57904 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41938 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57494 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54098 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50400 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53912 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53920 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46090 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46104 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33162 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46680 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39872 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49530 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58752 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58756 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46742 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46750 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47908 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45380 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51428 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43110 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47866 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47876 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59534 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59540 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58436 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39004 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47420 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50742 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51392 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51398 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55452 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55460 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47892 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52400 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33396 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43962 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37866 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37880 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38150 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38152 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45390 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37000 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47306 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38750 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37884 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37886 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60456 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60464 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42584 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46098 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41920 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38802 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43564 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43576 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55182 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55196 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50502 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59498 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42082 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40058 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54058 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54068 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35376 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35386 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38692 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33754 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60556 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60776 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32776 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32780 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43350 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43364 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43950 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34582 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52298 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60964 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40726 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40740 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50854 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50870 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49488 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60658 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38324 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37482 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50292 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50298 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37112 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37114 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57168 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40430 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50870 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47166 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43750 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43756 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40290 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40296 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32776 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40972 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49786 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37196 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45400 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45412 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35736 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35752 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57664 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56516 - "GET / HTTP/1.0" 200 OK 2026-03-13 00:39:43,363 - lava_callback - INFO - Received callback for node 69b35c55974c91e64560e4e6 2026-03-13 00:39:43,369 - lava_callback - INFO - Starting async job submit for node 69b35c55974c91e64560e4e6 INFO: 172.20.0.1:32778 - "POST /node/69b35c55974c91e64560e4e6 HTTP/1.0" 202 Accepted 2026-03-13 00:39:43,393 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:39:43,413 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35c55974c91e64560e4e6 HTTP/1.1" 200 None 2026-03-13 00:39:43,414 - lava_callback - DEBUG - Processing node 69b35c55974c91e64560e4e6: baseline-arm 2026-03-13 00:39:43,615 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 00:39:43,882 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 00:39:43,884 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69b35c55974c91e64560e4e6/log.txt.gz 2026-03-13 00:39:43,999 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 00:39:44,223 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 00:39:44,225 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69b35c55974c91e64560e4e6/lava-logs.yaml 2026-03-13 00:39:44,230 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 00:39:44,386 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 00:39:44,388 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69b35c55974c91e64560e4e6/lava_callback.json.gz 2026-03-13 00:39:44,411 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:39:44,426 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35c55974c91e64560e4e6 HTTP/1.1" 200 None 2026-03-13 00:39:44,427 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:39:44,441 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3593e974c91e64560e4a3 HTTP/1.1" 200 None 2026-03-13 00:39:44,442 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:39:44,467 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b35c55974c91e64560e4e6 HTTP/1.1" 200 None 2026-03-13 00:39:44,468 - lava_callback - INFO - Completed processing callback for node 69b35c55974c91e64560e4e6 INFO: 172.20.0.1:32782 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50650 - "GET / HTTP/1.0" 200 OK 2026-03-13 00:40:05,734 - lava_callback - INFO - Received callback for node 69b0b9dd9d191758182523ff INFO: 172.20.0.1:47002 - "POST /node/69b0b9dd9d191758182523ff HTTP/1.0" 202 Accepted 2026-03-13 00:40:05,740 - lava_callback - INFO - Starting async job submit for node 69b0b9dd9d191758182523ff 2026-03-13 00:40:05,741 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:40:05,761 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b9dd9d191758182523ff HTTP/1.1" 200 None 2026-03-13 00:40:05,762 - lava_callback - DEBUG - Processing node 69b0b9dd9d191758182523ff: baseline-arm-baylibre 2026-03-13 00:40:05,762 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0b9dd9d191758182523ff 2026-03-13 00:40:05,763 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:40:05,796 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b9dd9d191758182523ff HTTP/1.1" 200 None 2026-03-13 00:40:05,797 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:40:05,812 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b4fc9d19175818251fd6 HTTP/1.1" 200 None 2026-03-13 00:40:05,813 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:40:05,865 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0b9dd9d191758182523ff HTTP/1.1" 200 None 2026-03-13 00:40:05,867 - lava_callback - ERROR - Error processing callback for node 69b0b9dd9d191758182523ff: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:47010 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47016 - "GET / HTTP/1.0" 200 OK 2026-03-13 00:40:14,469 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:40:14,487 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:41632 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41636 - "GET / HTTP/1.0" 200 OK 2026-03-13 00:40:23,744 - lava_callback - INFO - Received callback for node 69b0b9e99d19175818252422 INFO: 172.20.0.1:37512 - "POST /node/69b0b9e99d19175818252422 HTTP/1.0" 202 Accepted 2026-03-13 00:40:23,750 - lava_callback - INFO - Starting async job submit for node 69b0b9e99d19175818252422 2026-03-13 00:40:23,750 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:40:23,765 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b9e99d19175818252422 HTTP/1.1" 200 None 2026-03-13 00:40:23,766 - lava_callback - DEBUG - Processing node 69b0b9e99d19175818252422: baseline-arm-clabbe 2026-03-13 00:40:23,766 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0b9e99d19175818252422 2026-03-13 00:40:23,766 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:40:23,785 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b9e99d19175818252422 HTTP/1.1" 200 None 2026-03-13 00:40:23,786 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:40:23,800 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b4fc9d19175818251fd6 HTTP/1.1" 200 None 2026-03-13 00:40:23,801 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:40:23,823 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0b9e99d19175818252422 HTTP/1.1" 200 None 2026-03-13 00:40:23,825 - lava_callback - ERROR - Error processing callback for node 69b0b9e99d19175818252422: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-13 00:40:24,398 - lava_callback - INFO - Received callback for node 69b0b9ed9d1917581825242f INFO: 172.20.0.1:37524 - "POST /node/69b0b9ed9d1917581825242f HTTP/1.0" 202 Accepted 2026-03-13 00:40:24,403 - lava_callback - INFO - Starting async job submit for node 69b0b9ed9d1917581825242f 2026-03-13 00:40:24,404 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:40:24,419 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b9ed9d1917581825242f HTTP/1.1" 200 None 2026-03-13 00:40:24,420 - lava_callback - DEBUG - Processing node 69b0b9ed9d1917581825242f: baseline-arm-clabbe 2026-03-13 00:40:24,420 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0b9ed9d1917581825242f 2026-03-13 00:40:24,421 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:40:24,434 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b9ed9d1917581825242f HTTP/1.1" 200 None 2026-03-13 00:40:24,435 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:40:24,453 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b4fc9d19175818251fd6 HTTP/1.1" 200 None 2026-03-13 00:40:24,454 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:40:24,474 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0b9ed9d1917581825242f HTTP/1.1" 200 None 2026-03-13 00:40:24,475 - lava_callback - ERROR - Error processing callback for node 69b0b9ed9d1917581825242f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-13 00:40:25,035 - lava_callback - INFO - Received callback for node 69b0b9ee9d19175818252432 INFO: 172.20.0.1:37538 - "POST /node/69b0b9ee9d19175818252432 HTTP/1.0" 202 Accepted 2026-03-13 00:40:25,041 - lava_callback - INFO - Starting async job submit for node 69b0b9ee9d19175818252432 2026-03-13 00:40:25,042 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:40:25,057 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b9ee9d19175818252432 HTTP/1.1" 200 None 2026-03-13 00:40:25,057 - lava_callback - DEBUG - Processing node 69b0b9ee9d19175818252432: baseline-arm-clabbe 2026-03-13 00:40:25,058 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0b9ee9d19175818252432 2026-03-13 00:40:25,058 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:40:25,072 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b9ee9d19175818252432 HTTP/1.1" 200 None 2026-03-13 00:40:25,073 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:40:25,086 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b4fc9d19175818251fd6 HTTP/1.1" 200 None 2026-03-13 00:40:25,088 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:40:25,112 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0b9ee9d19175818252432 HTTP/1.1" 200 None 2026-03-13 00:40:25,113 - lava_callback - ERROR - Error processing callback for node 69b0b9ee9d19175818252432: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-13 00:40:25,692 - lava_callback - INFO - Received callback for node 69b0b9ef9d19175818252435 INFO: 172.20.0.1:37540 - "POST /node/69b0b9ef9d19175818252435 HTTP/1.0" 202 Accepted 2026-03-13 00:40:25,697 - lava_callback - INFO - Starting async job submit for node 69b0b9ef9d19175818252435 2026-03-13 00:40:25,698 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:40:25,713 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b9ef9d19175818252435 HTTP/1.1" 200 None 2026-03-13 00:40:25,713 - lava_callback - DEBUG - Processing node 69b0b9ef9d19175818252435: baseline-arm-clabbe 2026-03-13 00:40:25,714 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0b9ef9d19175818252435 2026-03-13 00:40:25,714 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:40:25,737 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b9ef9d19175818252435 HTTP/1.1" 200 None 2026-03-13 00:40:25,739 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:40:25,752 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b4fc9d19175818251fd6 HTTP/1.1" 200 None 2026-03-13 00:40:25,754 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:40:25,775 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0b9ef9d19175818252435 HTTP/1.1" 200 None 2026-03-13 00:40:25,777 - lava_callback - ERROR - Error processing callback for node 69b0b9ef9d19175818252435: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-13 00:40:26,338 - lava_callback - INFO - Received callback for node 69b0b9f19d1917581825243e INFO: 172.20.0.1:37546 - "POST /node/69b0b9f19d1917581825243e HTTP/1.0" 202 Accepted 2026-03-13 00:40:26,344 - lava_callback - INFO - Starting async job submit for node 69b0b9f19d1917581825243e 2026-03-13 00:40:26,345 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:40:26,359 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b9f19d1917581825243e HTTP/1.1" 200 None 2026-03-13 00:40:26,360 - lava_callback - DEBUG - Processing node 69b0b9f19d1917581825243e: baseline-arm-clabbe 2026-03-13 00:40:26,360 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0b9f19d1917581825243e 2026-03-13 00:40:26,361 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:40:26,381 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b9f19d1917581825243e HTTP/1.1" 200 None 2026-03-13 00:40:26,382 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:40:26,396 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b4fc9d19175818251fd6 HTTP/1.1" 200 None 2026-03-13 00:40:26,397 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:40:26,417 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0b9f19d1917581825243e HTTP/1.1" 200 None 2026-03-13 00:40:26,422 - lava_callback - ERROR - Error processing callback for node 69b0b9f19d1917581825243e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:37562 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47556 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37348 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59580 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38776 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38792 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46024 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46030 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54360 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45194 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37596 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41648 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50892 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50896 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49714 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49724 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47848 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51538 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52480 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48364 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37106 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37110 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59710 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59722 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54676 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36176 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47524 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35460 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40424 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40432 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58674 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58688 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39920 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44304 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49556 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55264 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56482 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56488 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54008 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54024 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55450 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35468 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46594 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59464 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52010 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52020 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44072 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44088 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56660 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36526 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43270 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38852 - "GET / HTTP/1.0" 200 OK 2026-03-13 00:47:05,988 - lava_callback - INFO - Received callback for node 69b0bb809d19175818252619 INFO: 172.20.0.1:60296 - "POST /node/69b0bb809d19175818252619 HTTP/1.0" 202 Accepted 2026-03-13 00:47:05,993 - lava_callback - INFO - Starting async job submit for node 69b0bb809d19175818252619 2026-03-13 00:47:05,994 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:47:06,017 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0bb809d19175818252619 HTTP/1.1" 200 None 2026-03-13 00:47:06,018 - lava_callback - DEBUG - Processing node 69b0bb809d19175818252619: baseline-arm-baylibre 2026-03-13 00:47:06,018 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0bb809d19175818252619 2026-03-13 00:47:06,019 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:47:06,034 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0bb809d19175818252619 HTTP/1.1" 200 None 2026-03-13 00:47:06,036 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:47:06,057 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11668ef4424e487b2ea2 HTTP/1.1" 200 None 2026-03-13 00:47:06,058 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:47:06,081 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0bb809d19175818252619 HTTP/1.1" 200 None 2026-03-13 00:47:06,086 - lava_callback - ERROR - Error processing callback for node 69b0bb809d19175818252619: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:60312 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60320 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36208 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36216 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43106 - "GET / HTTP/1.0" 200 OK 2026-03-13 00:47:28,213 - lava_callback - INFO - Received callback for node 69b35e8e974c91e64560e902 INFO: 172.20.0.1:43120 - "POST /node/69b35e8e974c91e64560e902 HTTP/1.0" 202 Accepted 2026-03-13 00:47:28,241 - lava_callback - INFO - Starting async job submit for node 69b35e8e974c91e64560e902 2026-03-13 00:47:28,251 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:47:28,320 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35e8e974c91e64560e902 HTTP/1.1" 200 None 2026-03-13 00:47:28,321 - lava_callback - DEBUG - Processing node 69b35e8e974c91e64560e902: baseline-arm 2026-03-13 00:47:28,336 - lava_callback - WARNING - Failed to upload log 2026-03-13 00:47:28,341 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 00:47:28,719 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 00:47:28,721 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69b35e8e974c91e64560e902/lava-logs.yaml 2026-03-13 00:47:28,722 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 00:47:29,047 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 00:47:29,050 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69b35e8e974c91e64560e902/lava_callback.json.gz 2026-03-13 00:47:29,055 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:47:29,081 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35e8e974c91e64560e902 HTTP/1.1" 200 None 2026-03-13 00:47:29,083 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:47:29,112 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35934974c91e64560e49d HTTP/1.1" 200 None 2026-03-13 00:47:29,115 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:47:29,154 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b35e8e974c91e64560e902 HTTP/1.1" 200 None 2026-03-13 00:47:29,159 - lava_callback - INFO - Completed processing callback for node 69b35e8e974c91e64560e902 INFO: 172.20.0.1:37462 - "GET / HTTP/1.0" 200 OK 2026-03-13 00:47:44,491 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:47:44,517 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-13 00:47:46,006 - lava_callback - INFO - Received callback for node 69b0bba69d19175818252627 INFO: 172.20.0.1:55920 - "POST /node/69b0bba69d19175818252627 HTTP/1.0" 202 Accepted 2026-03-13 00:47:46,017 - lava_callback - INFO - Starting async job submit for node 69b0bba69d19175818252627 2026-03-13 00:47:46,018 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:47:46,051 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0bba69d19175818252627 HTTP/1.1" 200 None 2026-03-13 00:47:46,052 - lava_callback - DEBUG - Processing node 69b0bba69d19175818252627: baseline-arm-baylibre 2026-03-13 00:47:46,054 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0bba69d19175818252627 2026-03-13 00:47:46,055 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:47:46,083 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0bba69d19175818252627 HTTP/1.1" 200 None 2026-03-13 00:47:46,085 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:47:46,122 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11668ef4424e487b2ea2 HTTP/1.1" 200 None 2026-03-13 00:47:46,125 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:47:46,170 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0bba69d19175818252627 HTTP/1.1" 200 None 2026-03-13 00:47:46,171 - lava_callback - ERROR - Error processing callback for node 69b0bba69d19175818252627: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:55922 - "GET / HTTP/1.0" 200 OK 2026-03-13 00:47:56,152 - lava_callback - INFO - Received callback for node 69b35e8f974c91e64560e906 INFO: 172.20.0.1:48214 - "POST /node/69b35e8f974c91e64560e906 HTTP/1.0" 202 Accepted 2026-03-13 00:47:56,210 - lava_callback - INFO - Starting async job submit for node 69b35e8f974c91e64560e906 2026-03-13 00:47:56,246 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:47:56,273 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35e8f974c91e64560e906 HTTP/1.1" 200 None 2026-03-13 00:47:56,274 - lava_callback - DEBUG - Processing node 69b35e8f974c91e64560e906: baseline-arm INFO: 172.20.0.1:48230 - "GET / HTTP/1.0" 200 OK 2026-03-13 00:47:56,522 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 00:47:56,864 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 00:47:56,866 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69b35e8f974c91e64560e906/log.txt.gz 2026-03-13 00:47:57,021 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 00:47:57,587 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 00:47:57,589 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69b35e8f974c91e64560e906/lava-logs.yaml 2026-03-13 00:47:57,594 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 00:47:58,186 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 00:47:58,188 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69b35e8f974c91e64560e906/lava_callback.json.gz 2026-03-13 00:47:58,225 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:47:58,254 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35e8f974c91e64560e906 HTTP/1.1" 200 None 2026-03-13 00:47:58,256 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:47:58,289 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35934974c91e64560e49d HTTP/1.1" 200 None 2026-03-13 00:47:58,292 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:47:58,326 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b35e8f974c91e64560e906 HTTP/1.1" 200 None 2026-03-13 00:47:58,328 - lava_callback - INFO - Completed processing callback for node 69b35e8f974c91e64560e906 2026-03-13 00:48:03,922 - lava_callback - INFO - Received callback for node 69b35e92974c91e64560e910 INFO: 172.20.0.1:54806 - "POST /node/69b35e92974c91e64560e910 HTTP/1.0" 202 Accepted 2026-03-13 00:48:03,990 - lava_callback - INFO - Starting async job submit for node 69b35e92974c91e64560e910 2026-03-13 00:48:04,026 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:48:04,058 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35e92974c91e64560e910 HTTP/1.1" 200 None 2026-03-13 00:48:04,059 - lava_callback - DEBUG - Processing node 69b35e92974c91e64560e910: baseline-arm 2026-03-13 00:48:04,278 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 00:48:04,548 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 00:48:04,550 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69b35e92974c91e64560e910/log.txt.gz 2026-03-13 00:48:04,673 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 00:48:05,135 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 00:48:05,138 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69b35e92974c91e64560e910/lava-logs.yaml 2026-03-13 00:48:05,143 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 00:48:05,381 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 00:48:05,384 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69b35e92974c91e64560e910/lava_callback.json.gz 2026-03-13 00:48:05,426 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:48:05,450 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35e92974c91e64560e910 HTTP/1.1" 200 None 2026-03-13 00:48:05,453 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:48:05,486 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35934974c91e64560e49d HTTP/1.1" 200 None 2026-03-13 00:48:05,489 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:48:05,531 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b35e92974c91e64560e910 HTTP/1.1" 200 None 2026-03-13 00:48:05,533 - lava_callback - INFO - Completed processing callback for node 69b35e92974c91e64560e910 INFO: 172.20.0.1:54818 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54820 - "GET / HTTP/1.0" 200 OK 2026-03-13 00:48:14,521 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:48:14,585 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:53824 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53830 - "GET / HTTP/1.0" 200 OK 2026-03-13 00:48:26,253 - lava_callback - INFO - Received callback for node 69b0bbcc9d19175818252632 INFO: 172.20.0.1:36866 - "POST /node/69b0bbcc9d19175818252632 HTTP/1.0" 202 Accepted 2026-03-13 00:48:26,262 - lava_callback - INFO - Starting async job submit for node 69b0bbcc9d19175818252632 2026-03-13 00:48:26,263 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:48:26,316 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0bbcc9d19175818252632 HTTP/1.1" 200 None 2026-03-13 00:48:26,317 - lava_callback - DEBUG - Processing node 69b0bbcc9d19175818252632: baseline-arm-baylibre 2026-03-13 00:48:26,317 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0bbcc9d19175818252632 2026-03-13 00:48:26,318 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:48:26,344 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0bbcc9d19175818252632 HTTP/1.1" 200 None 2026-03-13 00:48:26,347 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:48:26,361 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11668ef4424e487b2ea2 HTTP/1.1" 200 None 2026-03-13 00:48:26,363 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:48:26,391 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0bbcc9d19175818252632 HTTP/1.1" 200 None 2026-03-13 00:48:26,392 - lava_callback - ERROR - Error processing callback for node 69b0bbcc9d19175818252632: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:36870 - "GET / HTTP/1.0" 200 OK 2026-03-13 00:48:34,943 - lava_callback - INFO - Received callback for node 69b35e89974c91e64560e8fb INFO: 172.20.0.1:48964 - "POST /node/69b35e89974c91e64560e8fb HTTP/1.0" 202 Accepted 2026-03-13 00:48:34,950 - lava_callback - INFO - Starting async job submit for node 69b35e89974c91e64560e8fb 2026-03-13 00:48:34,976 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:48:34,990 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35e89974c91e64560e8fb HTTP/1.1" 200 None 2026-03-13 00:48:34,991 - lava_callback - DEBUG - Processing node 69b35e89974c91e64560e8fb: baseline-arm 2026-03-13 00:48:35,173 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 00:48:35,314 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 00:48:35,316 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69b35e89974c91e64560e8fb/log.txt.gz 2026-03-13 00:48:35,488 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 00:48:35,675 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 00:48:35,677 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69b35e89974c91e64560e8fb/lava-logs.yaml 2026-03-13 00:48:35,684 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 00:48:35,869 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 00:48:35,871 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69b35e89974c91e64560e8fb/lava_callback.json.gz 2026-03-13 00:48:35,895 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:48:35,909 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35e89974c91e64560e8fb HTTP/1.1" 200 None 2026-03-13 00:48:35,911 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:48:35,926 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35934974c91e64560e49d HTTP/1.1" 200 None 2026-03-13 00:48:35,927 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:48:35,954 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b35e89974c91e64560e8fb HTTP/1.1" 200 None 2026-03-13 00:48:35,956 - lava_callback - INFO - Completed processing callback for node 69b35e89974c91e64560e8fb INFO: 172.20.0.1:48972 - "GET / HTTP/1.0" 200 OK 2026-03-13 00:48:44,588 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:48:44,604 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:38390 - "GET / HTTP/1.0" 200 OK 2026-03-13 00:48:50,786 - lava_callback - INFO - Received callback for node 69b35eb2974c91e64560e990 INFO: 172.20.0.1:52154 - "POST /node/69b35eb2974c91e64560e990 HTTP/1.0" 202 Accepted 2026-03-13 00:48:50,792 - lava_callback - INFO - Starting async job submit for node 69b35eb2974c91e64560e990 2026-03-13 00:48:50,810 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:48:50,827 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35eb2974c91e64560e990 HTTP/1.1" 200 None 2026-03-13 00:48:50,828 - lava_callback - DEBUG - Processing node 69b35eb2974c91e64560e990: baseline-arm-pengutronix 2026-03-13 00:48:50,968 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 00:48:51,206 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 00:48:51,208 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69b35eb2974c91e64560e990/log.txt.gz 2026-03-13 00:48:51,291 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 00:48:51,546 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 00:48:51,548 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69b35eb2974c91e64560e990/lava-logs.yaml 2026-03-13 00:48:51,552 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 00:48:51,744 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 00:48:51,746 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69b35eb2974c91e64560e990/lava_callback.json.gz 2026-03-13 00:48:51,765 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:48:51,780 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35eb2974c91e64560e990 HTTP/1.1" 200 None 2026-03-13 00:48:51,781 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:48:51,795 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35934974c91e64560e49d HTTP/1.1" 200 None 2026-03-13 00:48:51,797 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:48:51,822 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b35eb2974c91e64560e990 HTTP/1.1" 200 None 2026-03-13 00:48:51,829 - lava_callback - INFO - Completed processing callback for node 69b35eb2974c91e64560e990 INFO: 172.20.0.1:52162 - "GET / HTTP/1.0" 200 OK 2026-03-13 00:49:03,075 - lava_callback - INFO - Received callback for node 69b35eb7974c91e64560e99c INFO: 172.20.0.1:49618 - "POST /node/69b35eb7974c91e64560e99c HTTP/1.0" 202 Accepted 2026-03-13 00:49:03,121 - lava_callback - INFO - Starting async job submit for node 69b35eb7974c91e64560e99c 2026-03-13 00:49:03,141 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:49:03,156 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35eb7974c91e64560e99c HTTP/1.1" 200 None 2026-03-13 00:49:03,157 - lava_callback - DEBUG - Processing node 69b35eb7974c91e64560e99c: baseline-arm-pengutronix 2026-03-13 00:49:03,265 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 00:49:03,376 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 00:49:03,378 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69b35eb7974c91e64560e99c/log.txt.gz 2026-03-13 00:49:03,441 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 00:49:03,618 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 00:49:03,620 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69b35eb7974c91e64560e99c/lava-logs.yaml 2026-03-13 00:49:03,624 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 00:49:03,741 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 00:49:03,742 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69b35eb7974c91e64560e99c/lava_callback.json.gz 2026-03-13 00:49:03,763 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:49:03,785 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35eb7974c91e64560e99c HTTP/1.1" 200 None 2026-03-13 00:49:03,787 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:49:03,802 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35934974c91e64560e49d HTTP/1.1" 200 None 2026-03-13 00:49:03,803 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:49:03,828 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b35eb7974c91e64560e99c HTTP/1.1" 200 None 2026-03-13 00:49:03,829 - lava_callback - INFO - Completed processing callback for node 69b35eb7974c91e64560e99c 2026-03-13 00:49:06,096 - lava_callback - INFO - Received callback for node 69b0bbf19d19175818252641 INFO: 172.20.0.1:49632 - "POST /node/69b0bbf19d19175818252641 HTTP/1.0" 202 Accepted 2026-03-13 00:49:06,101 - lava_callback - INFO - Starting async job submit for node 69b0bbf19d19175818252641 2026-03-13 00:49:06,102 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:49:06,118 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0bbf19d19175818252641 HTTP/1.1" 200 None 2026-03-13 00:49:06,119 - lava_callback - DEBUG - Processing node 69b0bbf19d19175818252641: baseline-arm-baylibre 2026-03-13 00:49:06,119 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0bbf19d19175818252641 2026-03-13 00:49:06,120 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:49:06,152 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0bbf19d19175818252641 HTTP/1.1" 200 None 2026-03-13 00:49:06,154 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:49:06,168 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11668ef4424e487b2ea2 HTTP/1.1" 200 None 2026-03-13 00:49:06,169 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:49:06,191 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0bbf19d19175818252641 HTTP/1.1" 200 None 2026-03-13 00:49:06,197 - lava_callback - ERROR - Error processing callback for node 69b0bbf19d19175818252641: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:49634 - "GET / HTTP/1.0" 200 OK 2026-03-13 00:49:07,345 - lava_callback - INFO - Received callback for node 69b35e9c974c91e64560e932 INFO: 172.20.0.1:49644 - "POST /node/69b35e9c974c91e64560e932 HTTP/1.0" 202 Accepted 2026-03-13 00:49:07,351 - lava_callback - INFO - Starting async job submit for node 69b35e9c974c91e64560e932 2026-03-13 00:49:07,373 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:49:07,390 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35e9c974c91e64560e932 HTTP/1.1" 200 None 2026-03-13 00:49:07,391 - lava_callback - DEBUG - Processing node 69b35e9c974c91e64560e932: baseline-arm-broonie 2026-03-13 00:49:07,591 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 00:49:07,732 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 00:49:07,734 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69b35e9c974c91e64560e932/log.txt.gz 2026-03-13 00:49:07,853 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 00:49:08,028 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 00:49:08,030 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69b35e9c974c91e64560e932/lava-logs.yaml 2026-03-13 00:49:08,036 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 00:49:08,040 - lava_callback - INFO - Received callback for node 69b35efd974c91e64560ea21 2026-03-13 00:49:08,046 - lava_callback - INFO - Starting async job submit for node 69b35efd974c91e64560ea21 INFO: 172.20.0.1:49658 - "POST /node/69b35efd974c91e64560ea21 HTTP/1.0" 202 Accepted 2026-03-13 00:49:08,049 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:49:08,071 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35efd974c91e64560ea21 HTTP/1.1" 200 None 2026-03-13 00:49:08,072 - lava_callback - DEBUG - Processing node 69b35efd974c91e64560ea21: baseline-arm 2026-03-13 00:49:08,077 - lava_callback - WARNING - Failed to upload log 2026-03-13 00:49:08,081 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 00:49:08,189 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 00:49:08,191 - lava_callback - INFO - Received callback for node 69b35eb4974c91e64560e993 2026-03-13 00:49:08,197 - lava_callback - INFO - Starting async job submit for node 69b35eb4974c91e64560e993 INFO: 172.20.0.1:49670 - "POST /node/69b35eb4974c91e64560e993 HTTP/1.0" 202 Accepted 2026-03-13 00:49:08,213 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69b35e9c974c91e64560e932/lava_callback.json.gz 2026-03-13 00:49:08,235 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 00:49:08,242 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:49:08,246 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:49:08,247 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69b35efd974c91e64560ea21/lava-logs.yaml 2026-03-13 00:49:08,248 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 00:49:08,267 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35e9c974c91e64560e932 HTTP/1.1" 200 None 2026-03-13 00:49:08,269 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:49:08,270 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35eb4974c91e64560e993 HTTP/1.1" 200 None 2026-03-13 00:49:08,271 - lava_callback - DEBUG - Processing node 69b35eb4974c91e64560e993: baseline-arm-pengutronix INFO: 172.20.0.1:49676 - "GET / HTTP/1.0" 200 OK 2026-03-13 00:49:08,338 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35934974c91e64560e49d HTTP/1.1" 200 None 2026-03-13 00:49:08,361 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:49:08,414 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 00:49:08,467 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69b35efd974c91e64560ea21/lava_callback.json.gz 2026-03-13 00:49:08,475 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:49:08,485 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 00:49:08,499 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b35e9c974c91e64560e932 HTTP/1.1" 200 None 2026-03-13 00:49:08,509 - lava_callback - INFO - Completed processing callback for node 69b35e9c974c91e64560e932 2026-03-13 00:49:08,511 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35efd974c91e64560ea21 HTTP/1.1" 200 None 2026-03-13 00:49:08,535 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:49:08,563 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35934974c91e64560e49d HTTP/1.1" 200 None 2026-03-13 00:49:08,567 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:49:08,621 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b35efd974c91e64560ea21 HTTP/1.1" 200 None 2026-03-13 00:49:08,627 - lava_callback - INFO - Completed processing callback for node 69b35efd974c91e64560ea21 2026-03-13 00:49:08,728 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 00:49:08,730 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69b35eb4974c91e64560e993/log.txt.gz 2026-03-13 00:49:08,857 - lava_callback - INFO - Received callback for node 69b35e9e974c91e64560e935 INFO: 172.20.0.1:49690 - "POST /node/69b35e9e974c91e64560e935 HTTP/1.0" 202 Accepted 2026-03-13 00:49:08,869 - lava_callback - INFO - Starting async job submit for node 69b35e9e974c91e64560e935 2026-03-13 00:49:08,915 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:49:08,928 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 00:49:08,949 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35e9e974c91e64560e935 HTTP/1.1" 200 None 2026-03-13 00:49:08,951 - lava_callback - DEBUG - Processing node 69b35e9e974c91e64560e935: baseline-arm-broonie 2026-03-13 00:49:09,132 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 00:49:09,180 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69b35eb4974c91e64560e993/lava-logs.yaml 2026-03-13 00:49:09,210 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 00:49:09,211 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 00:49:09,349 - lava_callback - INFO - Received callback for node 69b35ea0974c91e64560e938 INFO: 172.20.0.1:49704 - "POST /node/69b35ea0974c91e64560e938 HTTP/1.0" 202 Accepted 2026-03-13 00:49:09,357 - lava_callback - INFO - Starting async job submit for node 69b35ea0974c91e64560e938 2026-03-13 00:49:09,375 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 00:49:09,386 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:49:09,387 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69b35e9e974c91e64560e935/log.txt.gz 2026-03-13 00:49:09,407 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 00:49:09,467 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69b35eb4974c91e64560e993/lava_callback.json.gz 2026-03-13 00:49:09,509 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:49:09,525 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35ea0974c91e64560e938 HTTP/1.1" 200 None 2026-03-13 00:49:09,531 - lava_callback - DEBUG - Processing node 69b35ea0974c91e64560e938: baseline-arm-broonie 2026-03-13 00:49:09,580 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 00:49:09,602 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35eb4974c91e64560e993 HTTP/1.1" 200 None 2026-03-13 00:49:09,630 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:49:09,723 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35934974c91e64560e49d HTTP/1.1" 200 None 2026-03-13 00:49:09,735 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:49:09,837 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b35eb4974c91e64560e993 HTTP/1.1" 200 None 2026-03-13 00:49:09,839 - lava_callback - INFO - Completed processing callback for node 69b35eb4974c91e64560e993 2026-03-13 00:49:09,845 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 00:49:09,983 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 00:49:09,986 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69b35e9e974c91e64560e935/lava-logs.yaml 2026-03-13 00:49:09,993 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 00:49:10,011 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 00:49:10,014 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69b35ea0974c91e64560e938/log.txt.gz 2026-03-13 00:49:10,178 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 00:49:10,199 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 00:49:10,199 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69b35e9e974c91e64560e935/lava_callback.json.gz 2026-03-13 00:49:10,233 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:49:10,254 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35e9e974c91e64560e935 HTTP/1.1" 200 None 2026-03-13 00:49:10,256 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:49:10,272 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35934974c91e64560e49d HTTP/1.1" 200 None 2026-03-13 00:49:10,275 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:49:10,304 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b35e9e974c91e64560e935 HTTP/1.1" 200 None 2026-03-13 00:49:10,315 - lava_callback - INFO - Completed processing callback for node 69b35e9e974c91e64560e935 2026-03-13 00:49:10,482 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 00:49:10,485 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69b35ea0974c91e64560e938/lava-logs.yaml 2026-03-13 00:49:10,493 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 00:49:10,669 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 00:49:10,671 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69b35ea0974c91e64560e938/lava_callback.json.gz 2026-03-13 00:49:10,704 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:49:10,730 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35ea0974c91e64560e938 HTTP/1.1" 200 None 2026-03-13 00:49:10,731 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:49:10,749 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35934974c91e64560e49d HTTP/1.1" 200 None 2026-03-13 00:49:10,752 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:49:10,776 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b35ea0974c91e64560e938 HTTP/1.1" 200 None 2026-03-13 00:49:10,778 - lava_callback - INFO - Completed processing callback for node 69b35ea0974c91e64560e938 2026-03-13 00:49:13,836 - lava_callback - INFO - Received callback for node 69b35ea5974c91e64560e949 INFO: 172.20.0.1:57872 - "POST /node/69b35ea5974c91e64560e949 HTTP/1.0" 202 Accepted 2026-03-13 00:49:13,842 - lava_callback - INFO - Starting async job submit for node 69b35ea5974c91e64560e949 2026-03-13 00:49:13,866 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:49:13,880 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35ea5974c91e64560e949 HTTP/1.1" 200 None 2026-03-13 00:49:13,881 - lava_callback - DEBUG - Processing node 69b35ea5974c91e64560e949: baseline-arm-clabbe 2026-03-13 00:49:14,114 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 00:49:14,240 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 00:49:14,242 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69b35ea5974c91e64560e949/log.txt.gz 2026-03-13 00:49:14,372 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 00:49:14,554 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 00:49:14,556 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69b35ea5974c91e64560e949/lava-logs.yaml 2026-03-13 00:49:14,562 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 00:49:14,606 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:49:14,629 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 2026-03-13 00:49:14,716 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 00:49:14,718 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69b35ea5974c91e64560e949/lava_callback.json.gz 2026-03-13 00:49:14,740 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:49:14,758 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35ea5974c91e64560e949 HTTP/1.1" 200 None 2026-03-13 00:49:14,760 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:49:14,774 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35934974c91e64560e49d HTTP/1.1" 200 None 2026-03-13 00:49:14,776 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:49:14,801 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b35ea5974c91e64560e949 HTTP/1.1" 200 None 2026-03-13 00:49:14,802 - lava_callback - INFO - Completed processing callback for node 69b35ea5974c91e64560e949 INFO: 172.20.0.1:57874 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57890 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33834 - "GET / HTTP/1.0" 200 OK 2026-03-13 00:49:30,299 - lava_callback - INFO - Received callback for node 69b35e9a974c91e64560e92f INFO: 172.20.0.1:60710 - "POST /node/69b35e9a974c91e64560e92f HTTP/1.0" 202 Accepted 2026-03-13 00:49:30,305 - lava_callback - INFO - Starting async job submit for node 69b35e9a974c91e64560e92f 2026-03-13 00:49:30,328 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:49:30,342 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35e9a974c91e64560e92f HTTP/1.1" 200 None 2026-03-13 00:49:30,343 - lava_callback - DEBUG - Processing node 69b35e9a974c91e64560e92f: baseline-arm-broonie 2026-03-13 00:49:30,546 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 00:49:30,700 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 00:49:30,702 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69b35e9a974c91e64560e92f/log.txt.gz 2026-03-13 00:49:30,825 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 00:49:31,016 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 00:49:31,018 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69b35e9a974c91e64560e92f/lava-logs.yaml 2026-03-13 00:49:31,024 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 00:49:31,179 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 00:49:31,180 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69b35e9a974c91e64560e92f/lava_callback.json.gz 2026-03-13 00:49:31,203 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:49:31,240 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35e9a974c91e64560e92f HTTP/1.1" 200 None 2026-03-13 00:49:31,242 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:49:31,259 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35934974c91e64560e49d HTTP/1.1" 200 None 2026-03-13 00:49:31,261 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:49:31,285 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b35e9a974c91e64560e92f HTTP/1.1" 200 None 2026-03-13 00:49:31,291 - lava_callback - INFO - Completed processing callback for node 69b35e9a974c91e64560e92f 2026-03-13 00:49:33,222 - lava_callback - INFO - Received callback for node 69b35eaf974c91e64560e963 INFO: 172.20.0.1:60714 - "POST /node/69b35eaf974c91e64560e963 HTTP/1.0" 202 Accepted 2026-03-13 00:49:33,228 - lava_callback - INFO - Starting async job submit for node 69b35eaf974c91e64560e963 2026-03-13 00:49:33,250 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:49:33,265 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35eaf974c91e64560e963 HTTP/1.1" 200 None 2026-03-13 00:49:33,267 - lava_callback - DEBUG - Processing node 69b35eaf974c91e64560e963: baseline-arm-kontron 2026-03-13 00:49:33,446 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 00:49:33,581 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 00:49:33,582 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-kontron-69b35eaf974c91e64560e963/log.txt.gz 2026-03-13 00:49:33,684 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 00:49:33,868 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 00:49:33,870 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-kontron-69b35eaf974c91e64560e963/lava-logs.yaml 2026-03-13 00:49:33,874 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 00:49:34,001 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 00:49:34,003 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-kontron-69b35eaf974c91e64560e963/lava_callback.json.gz 2026-03-13 00:49:34,026 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:49:34,040 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35eaf974c91e64560e963 HTTP/1.1" 200 None 2026-03-13 00:49:34,042 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:49:34,056 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35934974c91e64560e49d HTTP/1.1" 200 None 2026-03-13 00:49:34,058 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:49:34,083 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b35eaf974c91e64560e963 HTTP/1.1" 200 None 2026-03-13 00:49:34,088 - lava_callback - INFO - Completed processing callback for node 69b35eaf974c91e64560e963 INFO: 172.20.0.1:60728 - "GET / HTTP/1.0" 200 OK 2026-03-13 00:49:44,631 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:49:44,658 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-13 00:49:46,156 - lava_callback - INFO - Received callback for node 69b0bc179d19175818252669 INFO: 172.20.0.1:45342 - "POST /node/69b0bc179d19175818252669 HTTP/1.0" 202 Accepted 2026-03-13 00:49:46,166 - lava_callback - INFO - Starting async job submit for node 69b0bc179d19175818252669 2026-03-13 00:49:46,166 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:49:46,181 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0bc179d19175818252669 HTTP/1.1" 200 None 2026-03-13 00:49:46,182 - lava_callback - DEBUG - Processing node 69b0bc179d19175818252669: baseline-arm-baylibre 2026-03-13 00:49:46,182 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0bc179d19175818252669 2026-03-13 00:49:46,183 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:49:46,196 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0bc179d19175818252669 HTTP/1.1" 200 None 2026-03-13 00:49:46,198 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:49:46,216 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11668ef4424e487b2ea2 HTTP/1.1" 200 None 2026-03-13 00:49:46,218 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:49:46,241 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0bc179d19175818252669 HTTP/1.1" 200 None 2026-03-13 00:49:46,250 - lava_callback - ERROR - Error processing callback for node 69b0bc179d19175818252669: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:45348 - "GET / HTTP/1.0" 200 OK 2026-03-13 00:49:46,611 - lava_callback - INFO - Received callback for node 69b35e91974c91e64560e90d INFO: 172.20.0.1:45362 - "POST /node/69b35e91974c91e64560e90d HTTP/1.0" 202 Accepted 2026-03-13 00:49:46,624 - lava_callback - INFO - Starting async job submit for node 69b35e91974c91e64560e90d 2026-03-13 00:49:46,680 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:49:46,697 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35e91974c91e64560e90d HTTP/1.1" 200 None 2026-03-13 00:49:46,698 - lava_callback - DEBUG - Processing node 69b35e91974c91e64560e90d: baseline-arm 2026-03-13 00:49:46,912 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 00:49:47,055 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 00:49:47,057 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69b35e91974c91e64560e90d/log.txt.gz 2026-03-13 00:49:47,187 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 00:49:47,359 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 00:49:47,361 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69b35e91974c91e64560e90d/lava-logs.yaml 2026-03-13 00:49:47,366 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 00:49:47,509 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 00:49:47,511 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69b35e91974c91e64560e90d/lava_callback.json.gz 2026-03-13 00:49:47,533 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:49:47,548 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35e91974c91e64560e90d HTTP/1.1" 200 None 2026-03-13 00:49:47,549 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:49:47,564 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35934974c91e64560e49d HTTP/1.1" 200 None 2026-03-13 00:49:47,565 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:49:47,597 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b35e91974c91e64560e90d HTTP/1.1" 200 None 2026-03-13 00:49:47,603 - lava_callback - INFO - Completed processing callback for node 69b35e91974c91e64560e90d INFO: 172.20.0.1:60868 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34474 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34488 - "GET / HTTP/1.0" 200 OK 2026-03-13 00:50:14,660 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:50:14,680 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:48872 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48880 - "GET / HTTP/1.0" 200 OK 2026-03-13 00:50:26,169 - lava_callback - INFO - Received callback for node 69b0bc3d9d19175818252670 INFO: 172.20.0.1:39728 - "POST /node/69b0bc3d9d19175818252670 HTTP/1.0" 202 Accepted 2026-03-13 00:50:26,175 - lava_callback - INFO - Starting async job submit for node 69b0bc3d9d19175818252670 2026-03-13 00:50:26,175 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:50:26,190 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0bc3d9d19175818252670 HTTP/1.1" 200 None 2026-03-13 00:50:26,191 - lava_callback - DEBUG - Processing node 69b0bc3d9d19175818252670: baseline-arm-baylibre 2026-03-13 00:50:26,192 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0bc3d9d19175818252670 2026-03-13 00:50:26,192 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:50:26,205 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0bc3d9d19175818252670 HTTP/1.1" 200 None 2026-03-13 00:50:26,207 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:50:26,226 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11668ef4424e487b2ea2 HTTP/1.1" 200 None 2026-03-13 00:50:26,228 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:50:26,259 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0bc3d9d19175818252670 HTTP/1.1" 200 None 2026-03-13 00:50:26,261 - lava_callback - ERROR - Error processing callback for node 69b0bc3d9d19175818252670: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:39742 - "GET / HTTP/1.0" 200 OK 2026-03-13 00:50:28,011 - lava_callback - INFO - Received callback for node 69b35ed0974c91e64560e9d4 INFO: 172.20.0.1:39748 - "POST /node/69b35ed0974c91e64560e9d4 HTTP/1.0" 202 Accepted 2026-03-13 00:50:28,017 - lava_callback - INFO - Starting async job submit for node 69b35ed0974c91e64560e9d4 2026-03-13 00:50:28,040 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:50:28,056 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35ed0974c91e64560e9d4 HTTP/1.1" 200 None 2026-03-13 00:50:28,057 - lava_callback - DEBUG - Processing node 69b35ed0974c91e64560e9d4: ltp-dio 2026-03-13 00:50:28,423 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 00:50:28,579 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 00:50:28,581 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-dio-69b35ed0974c91e64560e9d4/log.txt.gz 2026-03-13 00:50:28,803 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 00:50:28,993 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 00:50:28,996 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-dio-69b35ed0974c91e64560e9d4/lava-logs.yaml 2026-03-13 00:50:29,005 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 00:50:29,184 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 00:50:29,186 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-dio-69b35ed0974c91e64560e9d4/lava_callback.json.gz 2026-03-13 00:50:29,230 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:50:29,244 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35ed0974c91e64560e9d4 HTTP/1.1" 200 None 2026-03-13 00:50:29,246 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:50:29,268 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35934974c91e64560e49d HTTP/1.1" 200 None 2026-03-13 00:50:29,269 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:50:29,293 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b35ed0974c91e64560e9d4 HTTP/1.1" 200 None 2026-03-13 00:50:29,295 - lava_callback - INFO - Completed processing callback for node 69b35ed0974c91e64560e9d4 INFO: 172.20.0.1:50702 - "GET / HTTP/1.0" 200 OK 2026-03-13 00:50:44,682 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:50:44,699 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:43380 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41970 - "GET / HTTP/1.0" 200 OK 2026-03-13 00:51:00,946 - lava_callback - INFO - Received callback for node 69b35ea6974c91e64560e94d INFO: 172.20.0.1:47798 - "POST /node/69b35ea6974c91e64560e94d HTTP/1.0" 202 Accepted 2026-03-13 00:51:00,952 - lava_callback - INFO - Starting async job submit for node 69b35ea6974c91e64560e94d 2026-03-13 00:51:00,972 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:51:00,987 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35ea6974c91e64560e94d HTTP/1.1" 200 None 2026-03-13 00:51:00,988 - lava_callback - DEBUG - Processing node 69b35ea6974c91e64560e94d: baseline-arm-clabbe 2026-03-13 00:51:01,218 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 00:51:01,384 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 00:51:01,386 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69b35ea6974c91e64560e94d/log.txt.gz 2026-03-13 00:51:01,512 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 00:51:01,707 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 00:51:01,709 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69b35ea6974c91e64560e94d/lava-logs.yaml 2026-03-13 00:51:01,714 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 00:51:01,885 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 00:51:01,886 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69b35ea6974c91e64560e94d/lava_callback.json.gz 2026-03-13 00:51:01,909 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:51:01,924 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35ea6974c91e64560e94d HTTP/1.1" 200 None 2026-03-13 00:51:01,926 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:51:01,940 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35934974c91e64560e49d HTTP/1.1" 200 None 2026-03-13 00:51:01,942 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:51:01,978 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b35ea6974c91e64560e94d HTTP/1.1" 200 None 2026-03-13 00:51:01,980 - lava_callback - INFO - Completed processing callback for node 69b35ea6974c91e64560e94d INFO: 172.20.0.1:47808 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47822 - "GET / HTTP/1.0" 200 OK 2026-03-13 00:51:14,701 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:51:14,718 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:58402 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58406 - "GET / HTTP/1.0" 200 OK 2026-03-13 00:51:24,236 - lava_callback - INFO - Received callback for node 69b0bc899d1917581825268d INFO: 172.20.0.1:42814 - "POST /node/69b0bc899d1917581825268d HTTP/1.0" 202 Accepted 2026-03-13 00:51:24,242 - lava_callback - INFO - Starting async job submit for node 69b0bc899d1917581825268d 2026-03-13 00:51:24,243 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:51:24,271 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0bc899d1917581825268d HTTP/1.1" 200 None 2026-03-13 00:51:24,272 - lava_callback - DEBUG - Processing node 69b0bc899d1917581825268d: baseline-arm-clabbe 2026-03-13 00:51:24,272 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0bc899d1917581825268d 2026-03-13 00:51:24,273 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:51:24,326 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0bc899d1917581825268d HTTP/1.1" 200 None 2026-03-13 00:51:24,328 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:51:24,340 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11668ef4424e487b2ea2 HTTP/1.1" 200 None 2026-03-13 00:51:24,342 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:51:24,365 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0bc899d1917581825268d HTTP/1.1" 200 None 2026-03-13 00:51:24,367 - lava_callback - ERROR - Error processing callback for node 69b0bc899d1917581825268d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:42824 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59698 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57630 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43424 - "GET / HTTP/1.0" 200 OK 2026-03-13 00:52:04,283 - lava_callback - INFO - Received callback for node 69b0bcaf9d191758182526a4 INFO: 172.20.0.1:40926 - "POST /node/69b0bcaf9d191758182526a4 HTTP/1.0" 202 Accepted 2026-03-13 00:52:04,288 - lava_callback - INFO - Starting async job submit for node 69b0bcaf9d191758182526a4 2026-03-13 00:52:04,289 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:52:04,303 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0bcaf9d191758182526a4 HTTP/1.1" 200 None 2026-03-13 00:52:04,304 - lava_callback - DEBUG - Processing node 69b0bcaf9d191758182526a4: baseline-arm-clabbe 2026-03-13 00:52:04,305 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0bcaf9d191758182526a4 2026-03-13 00:52:04,305 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:52:04,318 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0bcaf9d191758182526a4 HTTP/1.1" 200 None 2026-03-13 00:52:04,320 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:52:04,333 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11668ef4424e487b2ea2 HTTP/1.1" 200 None 2026-03-13 00:52:04,334 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:52:04,355 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0bcaf9d191758182526a4 HTTP/1.1" 200 None 2026-03-13 00:52:04,361 - lava_callback - ERROR - Error processing callback for node 69b0bcaf9d191758182526a4: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-13 00:52:05,634 - lava_callback - INFO - Received callback for node 69b35ec8974c91e64560e9ac INFO: 172.20.0.1:40930 - "POST /node/69b35ec8974c91e64560e9ac HTTP/1.0" 202 Accepted 2026-03-13 00:52:05,641 - lava_callback - INFO - Starting async job submit for node 69b35ec8974c91e64560e9ac 2026-03-13 00:52:05,665 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:52:05,682 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35ec8974c91e64560e9ac HTTP/1.1" 200 None 2026-03-13 00:52:05,683 - lava_callback - DEBUG - Processing node 69b35ec8974c91e64560e9ac: ltp-cap_bounds 2026-03-13 00:52:06,028 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 00:52:06,326 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 00:52:06,328 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-cap_bounds-69b35ec8974c91e64560e9ac/log.txt.gz INFO: 172.20.0.1:40940 - "GET / HTTP/1.0" 200 OK 2026-03-13 00:52:06,534 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 00:52:06,793 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 00:52:06,795 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-cap_bounds-69b35ec8974c91e64560e9ac/lava-logs.yaml 2026-03-13 00:52:06,803 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 00:52:07,003 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 00:52:07,005 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-cap_bounds-69b35ec8974c91e64560e9ac/lava_callback.json.gz 2026-03-13 00:52:07,031 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:52:07,045 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35ec8974c91e64560e9ac HTTP/1.1" 200 None 2026-03-13 00:52:07,047 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:52:07,060 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35934974c91e64560e49d HTTP/1.1" 200 None 2026-03-13 00:52:07,062 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:52:07,089 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b35ec8974c91e64560e9ac HTTP/1.1" 200 None 2026-03-13 00:52:07,095 - lava_callback - INFO - Completed processing callback for node 69b35ec8974c91e64560e9ac INFO: 172.20.0.1:40954 - "GET / HTTP/1.0" 200 OK 2026-03-13 00:52:10,691 - lava_callback - INFO - Received callback for node 69b35ec9974c91e64560e9af INFO: 172.20.0.1:49464 - "POST /node/69b35ec9974c91e64560e9af HTTP/1.0" 202 Accepted 2026-03-13 00:52:10,697 - lava_callback - INFO - Starting async job submit for node 69b35ec9974c91e64560e9af 2026-03-13 00:52:10,721 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:52:10,735 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35ec9974c91e64560e9af HTTP/1.1" 200 None 2026-03-13 00:52:10,736 - lava_callback - DEBUG - Processing node 69b35ec9974c91e64560e9af: ltp-containers 2026-03-13 00:52:11,142 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 00:52:11,305 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 00:52:11,307 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-containers-69b35ec9974c91e64560e9af/log.txt.gz 2026-03-13 00:52:11,607 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 00:52:11,913 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 00:52:11,916 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-containers-69b35ec9974c91e64560e9af/lava-logs.yaml 2026-03-13 00:52:11,933 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 00:52:12,149 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 00:52:12,150 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-containers-69b35ec9974c91e64560e9af/lava_callback.json.gz 2026-03-13 00:52:12,177 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:52:12,191 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35ec9974c91e64560e9af HTTP/1.1" 200 None 2026-03-13 00:52:12,192 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:52:12,214 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35934974c91e64560e49d HTTP/1.1" 200 None 2026-03-13 00:52:12,216 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:52:12,245 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b35ec9974c91e64560e9af HTTP/1.1" 200 None 2026-03-13 00:52:12,247 - lava_callback - INFO - Completed processing callback for node 69b35ec9974c91e64560e9af 2026-03-13 00:52:14,720 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:52:14,736 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:49476 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49490 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47594 - "GET / HTTP/1.0" 200 OK 2026-03-13 00:52:29,809 - lava_callback - INFO - Received callback for node 69b35edd974c91e64560e9f4 INFO: 172.20.0.1:45070 - "POST /node/69b35edd974c91e64560e9f4 HTTP/1.0" 202 Accepted 2026-03-13 00:52:29,816 - lava_callback - INFO - Starting async job submit for node 69b35edd974c91e64560e9f4 2026-03-13 00:52:29,841 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:52:29,858 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35edd974c91e64560e9f4 HTTP/1.1" 200 None 2026-03-13 00:52:29,859 - lava_callback - DEBUG - Processing node 69b35edd974c91e64560e9f4: ltp-syscalls 2026-03-13 00:52:30,565 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 00:52:30,725 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 00:52:30,727 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-syscalls-69b35edd974c91e64560e9f4/log.txt.gz 2026-03-13 00:52:31,128 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 00:52:31,341 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 00:52:31,344 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-syscalls-69b35edd974c91e64560e9f4/lava-logs.yaml 2026-03-13 00:52:31,361 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 00:52:31,504 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 00:52:31,506 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-syscalls-69b35edd974c91e64560e9f4/lava_callback.json.gz 2026-03-13 00:52:31,531 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:52:31,545 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35edd974c91e64560e9f4 HTTP/1.1" 200 None 2026-03-13 00:52:31,547 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:52:31,560 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35934974c91e64560e49d HTTP/1.1" 200 None 2026-03-13 00:52:31,561 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:52:31,587 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b35edd974c91e64560e9f4 HTTP/1.1" 200 None 2026-03-13 00:52:31,589 - lava_callback - INFO - Completed processing callback for node 69b35edd974c91e64560e9f4 2026-03-13 00:52:36,002 - lava_callback - INFO - Received callback for node 69b35eab974c91e64560e959 INFO: 172.20.0.1:45080 - "POST /node/69b35eab974c91e64560e959 HTTP/1.0" 202 Accepted 2026-03-13 00:52:36,008 - lava_callback - INFO - Starting async job submit for node 69b35eab974c91e64560e959 2026-03-13 00:52:36,028 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:52:36,047 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35eab974c91e64560e959 HTTP/1.1" 200 None 2026-03-13 00:52:36,048 - lava_callback - DEBUG - Processing node 69b35eab974c91e64560e959: baseline-arm-clabbe 2026-03-13 00:52:36,270 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 00:52:36,383 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 00:52:36,385 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69b35eab974c91e64560e959/log.txt.gz 2026-03-13 00:52:36,509 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:45086 - "GET / HTTP/1.0" 200 OK 2026-03-13 00:52:36,682 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 00:52:36,684 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69b35eab974c91e64560e959/lava-logs.yaml 2026-03-13 00:52:36,690 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 00:52:36,816 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 00:52:36,818 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69b35eab974c91e64560e959/lava_callback.json.gz 2026-03-13 00:52:36,840 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:52:36,857 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35eab974c91e64560e959 HTTP/1.1" 200 None 2026-03-13 00:52:36,859 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:52:36,872 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35934974c91e64560e49d HTTP/1.1" 200 None 2026-03-13 00:52:36,874 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:52:36,897 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b35eab974c91e64560e959 HTTP/1.1" 200 None 2026-03-13 00:52:36,900 - lava_callback - INFO - Completed processing callback for node 69b35eab974c91e64560e959 2026-03-13 00:52:44,389 - lava_callback - INFO - Received callback for node 69b0bcd79d191758182526ab INFO: 172.20.0.1:38216 - "POST /node/69b0bcd79d191758182526ab HTTP/1.0" 202 Accepted 2026-03-13 00:52:44,395 - lava_callback - INFO - Starting async job submit for node 69b0bcd79d191758182526ab 2026-03-13 00:52:44,396 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:52:44,410 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0bcd79d191758182526ab HTTP/1.1" 200 None 2026-03-13 00:52:44,410 - lava_callback - DEBUG - Processing node 69b0bcd79d191758182526ab: baseline-arm-clabbe 2026-03-13 00:52:44,411 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0bcd79d191758182526ab 2026-03-13 00:52:44,411 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:52:44,424 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0bcd79d191758182526ab HTTP/1.1" 200 None 2026-03-13 00:52:44,426 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:52:44,439 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11668ef4424e487b2ea2 HTTP/1.1" 200 None 2026-03-13 00:52:44,440 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:52:44,467 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0bcd79d191758182526ab HTTP/1.1" 200 None 2026-03-13 00:52:44,473 - lava_callback - ERROR - Error processing callback for node 69b0bcd79d191758182526ab: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-13 00:52:44,738 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:52:44,754 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:38230 - "GET / HTTP/1.0" 200 OK 2026-03-13 00:52:50,593 - lava_callback - INFO - Received callback for node 69b35ecb974c91e64560e9b3 INFO: 172.20.0.1:48808 - "POST /node/69b35ecb974c91e64560e9b3 HTTP/1.0" 202 Accepted 2026-03-13 00:52:50,599 - lava_callback - INFO - Starting async job submit for node 69b35ecb974c91e64560e9b3 2026-03-13 00:52:50,624 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:52:50,638 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35ecb974c91e64560e9b3 HTTP/1.1" 200 None 2026-03-13 00:52:50,639 - lava_callback - DEBUG - Processing node 69b35ecb974c91e64560e9b3: ltp-controllers 2026-03-13 00:52:51,095 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 00:52:51,244 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 00:52:51,246 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-controllers-69b35ecb974c91e64560e9b3/log.txt.gz 2026-03-13 00:52:51,606 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 00:52:51,788 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 00:52:51,790 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-controllers-69b35ecb974c91e64560e9b3/lava-logs.yaml 2026-03-13 00:52:51,801 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 00:52:51,945 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 00:52:51,947 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-controllers-69b35ecb974c91e64560e9b3/lava_callback.json.gz 2026-03-13 00:52:51,984 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:52:52,000 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35ecb974c91e64560e9b3 HTTP/1.1" 200 None 2026-03-13 00:52:52,002 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:52:52,017 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35934974c91e64560e49d HTTP/1.1" 200 None 2026-03-13 00:52:52,019 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:52:52,042 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b35ecb974c91e64560e9b3 HTTP/1.1" 200 None 2026-03-13 00:52:52,044 - lava_callback - INFO - Completed processing callback for node 69b35ecb974c91e64560e9b3 INFO: 172.20.0.1:48822 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38018 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38034 - "GET / HTTP/1.0" 200 OK 2026-03-13 00:53:14,756 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:53:14,797 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:59284 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59288 - "GET / HTTP/1.0" 200 OK 2026-03-13 00:53:24,358 - lava_callback - INFO - Received callback for node 69b0bcfe9d191758182526c1 INFO: 172.20.0.1:35348 - "POST /node/69b0bcfe9d191758182526c1 HTTP/1.0" 202 Accepted 2026-03-13 00:53:24,363 - lava_callback - INFO - Starting async job submit for node 69b0bcfe9d191758182526c1 2026-03-13 00:53:24,364 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:53:24,378 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0bcfe9d191758182526c1 HTTP/1.1" 200 None 2026-03-13 00:53:24,378 - lava_callback - DEBUG - Processing node 69b0bcfe9d191758182526c1: baseline-arm-clabbe 2026-03-13 00:53:24,379 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0bcfe9d191758182526c1 2026-03-13 00:53:24,379 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:53:24,393 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0bcfe9d191758182526c1 HTTP/1.1" 200 None 2026-03-13 00:53:24,394 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:53:24,407 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11668ef4424e487b2ea2 HTTP/1.1" 200 None 2026-03-13 00:53:24,408 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:53:24,432 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0bcfe9d191758182526c1 HTTP/1.1" 200 None 2026-03-13 00:53:24,433 - lava_callback - ERROR - Error processing callback for node 69b0bcfe9d191758182526c1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:35354 - "GET / HTTP/1.0" 200 OK 2026-03-13 00:53:29,429 - lava_callback - INFO - Received callback for node 69b35ece974c91e64560e9b6 INFO: 172.20.0.1:35366 - "POST /node/69b35ece974c91e64560e9b6 HTTP/1.0" 202 Accepted 2026-03-13 00:53:29,435 - lava_callback - INFO - Starting async job submit for node 69b35ece974c91e64560e9b6 2026-03-13 00:53:29,460 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:53:29,474 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35ece974c91e64560e9b6 HTTP/1.1" 200 None 2026-03-13 00:53:29,475 - lava_callback - DEBUG - Processing node 69b35ece974c91e64560e9b6: ltp-cve 2026-03-13 00:53:29,834 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 00:53:29,997 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 00:53:29,999 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-cve-69b35ece974c91e64560e9b6/log.txt.gz 2026-03-13 00:53:30,217 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 00:53:30,445 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 00:53:30,447 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-cve-69b35ece974c91e64560e9b6/lava-logs.yaml 2026-03-13 00:53:30,456 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 00:53:30,599 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 00:53:30,601 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-cve-69b35ece974c91e64560e9b6/lava_callback.json.gz 2026-03-13 00:53:30,628 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:53:30,641 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35ece974c91e64560e9b6 HTTP/1.1" 200 None 2026-03-13 00:53:30,643 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:53:30,656 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35934974c91e64560e49d HTTP/1.1" 200 None 2026-03-13 00:53:30,657 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:53:30,682 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b35ece974c91e64560e9b6 HTTP/1.1" 200 None 2026-03-13 00:53:30,687 - lava_callback - INFO - Completed processing callback for node 69b35ece974c91e64560e9b6 INFO: 172.20.0.1:45870 - "GET / HTTP/1.0" 200 OK 2026-03-13 00:53:44,798 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:53:44,814 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:48862 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32992 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38550 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38560 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49236 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49240 - "GET / HTTP/1.0" 200 OK 2026-03-13 00:54:19,038 - lava_callback - INFO - Received callback for node 69b35eac974c91e64560e95c INFO: 172.20.0.1:49244 - "POST /node/69b35eac974c91e64560e95c HTTP/1.0" 202 Accepted 2026-03-13 00:54:19,044 - lava_callback - INFO - Starting async job submit for node 69b35eac974c91e64560e95c 2026-03-13 00:54:19,066 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:54:19,080 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35eac974c91e64560e95c HTTP/1.1" 200 None 2026-03-13 00:54:19,081 - lava_callback - DEBUG - Processing node 69b35eac974c91e64560e95c: baseline-arm-clabbe 2026-03-13 00:54:19,299 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 00:54:19,418 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 00:54:19,420 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69b35eac974c91e64560e95c/log.txt.gz 2026-03-13 00:54:19,566 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 00:54:19,747 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 00:54:19,749 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69b35eac974c91e64560e95c/lava-logs.yaml 2026-03-13 00:54:19,755 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 00:54:19,875 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 00:54:19,877 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69b35eac974c91e64560e95c/lava_callback.json.gz 2026-03-13 00:54:19,900 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:54:19,915 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35eac974c91e64560e95c HTTP/1.1" 200 None 2026-03-13 00:54:19,916 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:54:19,933 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35934974c91e64560e49d HTTP/1.1" 200 None 2026-03-13 00:54:19,935 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:54:19,958 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b35eac974c91e64560e95c HTTP/1.1" 200 None 2026-03-13 00:54:19,968 - lava_callback - INFO - Completed processing callback for node 69b35eac974c91e64560e95c INFO: 172.20.0.1:57140 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46856 - "GET / HTTP/1.0" 200 OK 2026-03-13 00:54:44,432 - lava_callback - INFO - Received callback for node 69b0bd4a9d191758182526eb INFO: 172.20.0.1:56136 - "POST /node/69b0bd4a9d191758182526eb HTTP/1.0" 202 Accepted 2026-03-13 00:54:44,438 - lava_callback - INFO - Starting async job submit for node 69b0bd4a9d191758182526eb 2026-03-13 00:54:44,439 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:54:44,453 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0bd4a9d191758182526eb HTTP/1.1" 200 None 2026-03-13 00:54:44,454 - lava_callback - DEBUG - Processing node 69b0bd4a9d191758182526eb: baseline-arm-clabbe 2026-03-13 00:54:44,454 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0bd4a9d191758182526eb 2026-03-13 00:54:44,455 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:54:44,489 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0bd4a9d191758182526eb HTTP/1.1" 200 None 2026-03-13 00:54:44,490 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:54:44,505 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11668ef4424e487b2ea2 HTTP/1.1" 200 None 2026-03-13 00:54:44,506 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:54:44,527 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0bd4a9d191758182526eb HTTP/1.1" 200 None 2026-03-13 00:54:44,528 - lava_callback - ERROR - Error processing callback for node 69b0bd4a9d191758182526eb: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-13 00:54:44,816 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:54:44,836 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:56146 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53970 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40210 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40218 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40942 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40952 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42762 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50932 - "GET / HTTP/1.0" 200 OK 2026-03-13 00:55:38,842 - lava_callback - INFO - Received callback for node 69b3606c974c91e64560eb26 INFO: 172.20.0.1:50940 - "POST /node/69b3606c974c91e64560eb26 HTTP/1.0" 202 Accepted 2026-03-13 00:55:38,848 - lava_callback - INFO - Starting async job submit for node 69b3606c974c91e64560eb26 2026-03-13 00:55:38,851 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:55:38,865 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3606c974c91e64560eb26 HTTP/1.1" 200 None 2026-03-13 00:55:38,866 - lava_callback - DEBUG - Processing node 69b3606c974c91e64560eb26: baseline-arm 2026-03-13 00:55:38,871 - lava_callback - WARNING - Failed to upload log 2026-03-13 00:55:38,875 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 00:55:39,038 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 00:55:39,040 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69b3606c974c91e64560eb26/lava-logs.yaml 2026-03-13 00:55:39,042 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 00:55:39,183 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 00:55:39,185 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69b3606c974c91e64560eb26/lava_callback.json.gz 2026-03-13 00:55:39,188 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:55:39,202 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3606c974c91e64560eb26 HTTP/1.1" 200 None 2026-03-13 00:55:39,204 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:55:39,217 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35934974c91e64560e49d HTTP/1.1" 200 None 2026-03-13 00:55:39,219 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:55:39,243 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b3606c974c91e64560eb26 HTTP/1.1" 200 None 2026-03-13 00:55:39,257 - lava_callback - INFO - Completed processing callback for node 69b3606c974c91e64560eb26 2026-03-13 00:55:44,838 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:55:44,863 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:38396 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56126 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33520 - "GET / HTTP/1.0" 200 OK 2026-03-13 00:56:08,211 - lava_callback - INFO - Received callback for node 69b35ed4974c91e64560e9da INFO: 172.20.0.1:33526 - "POST /node/69b35ed4974c91e64560e9da HTTP/1.0" 202 Accepted 2026-03-13 00:56:08,221 - lava_callback - INFO - Starting async job submit for node 69b35ed4974c91e64560e9da 2026-03-13 00:56:08,271 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:56:08,297 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35ed4974c91e64560e9da HTTP/1.1" 200 None 2026-03-13 00:56:08,298 - lava_callback - DEBUG - Processing node 69b35ed4974c91e64560e9da: ltp-fsx INFO: 172.20.0.1:33534 - "GET / HTTP/1.0" 200 OK 2026-03-13 00:56:08,912 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 00:56:09,239 - lava_callback - INFO - Received callback for node 69b35ed2974c91e64560e9d7 INFO: 172.20.0.1:33550 - "POST /node/69b35ed2974c91e64560e9d7 HTTP/1.0" 202 Accepted 2026-03-13 00:56:09,248 - lava_callback - INFO - Starting async job submit for node 69b35ed2974c91e64560e9d7 2026-03-13 00:56:09,287 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 00:56:09,290 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-fsx-69b35ed4974c91e64560e9da/log.txt.gz 2026-03-13 00:56:09,325 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:56:09,384 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35ed2974c91e64560e9d7 HTTP/1.1" 200 None 2026-03-13 00:56:09,385 - lava_callback - DEBUG - Processing node 69b35ed2974c91e64560e9d7: ltp-fs_bind 2026-03-13 00:56:09,998 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 00:56:10,391 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 00:56:10,757 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 00:56:10,760 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-fsx-69b35ed4974c91e64560e9da/lava-logs.yaml 2026-03-13 00:56:10,770 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 00:56:10,896 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 00:56:10,898 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-fs_bind-69b35ed2974c91e64560e9d7/log.txt.gz 2026-03-13 00:56:11,281 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 00:56:11,295 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 00:56:11,297 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-fsx-69b35ed4974c91e64560e9da/lava_callback.json.gz 2026-03-13 00:56:11,350 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:56:11,382 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35ed4974c91e64560e9da HTTP/1.1" 200 None 2026-03-13 00:56:11,385 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:56:11,407 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35934974c91e64560e49d HTTP/1.1" 200 None 2026-03-13 00:56:11,410 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:56:11,444 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b35ed4974c91e64560e9da HTTP/1.1" 200 None 2026-03-13 00:56:11,445 - lava_callback - INFO - Completed processing callback for node 69b35ed4974c91e64560e9da 2026-03-13 00:56:12,080 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 00:56:12,082 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-fs_bind-69b35ed2974c91e64560e9d7/lava-logs.yaml 2026-03-13 00:56:12,096 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 00:56:12,530 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 00:56:12,532 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-fs_bind-69b35ed2974c91e64560e9d7/lava_callback.json.gz 2026-03-13 00:56:12,575 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:56:12,602 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35ed2974c91e64560e9d7 HTTP/1.1" 200 None 2026-03-13 00:56:12,605 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:56:12,634 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35934974c91e64560e49d HTTP/1.1" 200 None 2026-03-13 00:56:12,637 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:56:12,683 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b35ed2974c91e64560e9d7 HTTP/1.1" 200 None 2026-03-13 00:56:12,684 - lava_callback - INFO - Completed processing callback for node 69b35ed2974c91e64560e9d7 2026-03-13 00:56:14,865 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:56:14,890 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:56536 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56542 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58248 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43852 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41066 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49930 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34128 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34138 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50800 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50802 - "GET / HTTP/1.0" 200 OK 2026-03-13 00:57:24,686 - lava_callback - INFO - Received callback for node 69b35c56974c91e64560e4e9 INFO: 172.20.0.1:53854 - "POST /node/69b35c56974c91e64560e4e9 HTTP/1.0" 202 Accepted 2026-03-13 00:57:24,693 - lava_callback - INFO - Starting async job submit for node 69b35c56974c91e64560e4e9 2026-03-13 00:57:24,749 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:57:24,766 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35c56974c91e64560e4e9 HTTP/1.1" 200 None 2026-03-13 00:57:24,767 - lava_callback - DEBUG - Processing node 69b35c56974c91e64560e4e9: ltp-smoke 2026-03-13 00:57:25,232 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 00:57:25,372 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 00:57:25,374 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoke-69b35c56974c91e64560e4e9/log.txt.gz 2026-03-13 00:57:25,678 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 00:57:25,881 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 00:57:25,883 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-smoke-69b35c56974c91e64560e4e9/lava-logs.yaml 2026-03-13 00:57:25,895 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 00:57:26,073 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 00:57:26,075 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoke-69b35c56974c91e64560e4e9/lava_callback.json.gz 2026-03-13 00:57:26,195 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:57:26,210 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35c56974c91e64560e4e9 HTTP/1.1" 200 None 2026-03-13 00:57:26,211 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:57:26,225 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3593e974c91e64560e4a3 HTTP/1.1" 200 None 2026-03-13 00:57:26,226 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:57:26,263 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b35c56974c91e64560e4e9 HTTP/1.1" 200 None 2026-03-13 00:57:26,393 - lava_callback - INFO - Completed processing callback for node 69b35c56974c91e64560e4e9 INFO: 172.20.0.1:53868 - "GET / HTTP/1.0" 200 OK 2026-03-13 00:57:29,317 - lava_callback - INFO - Received callback for node 69b35ed5974c91e64560e9df INFO: 172.20.0.1:53880 - "POST /node/69b35ed5974c91e64560e9df HTTP/1.0" 202 Accepted 2026-03-13 00:57:29,324 - lava_callback - INFO - Starting async job submit for node 69b35ed5974c91e64560e9df 2026-03-13 00:57:29,352 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:57:29,367 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35ed5974c91e64560e9df HTTP/1.1" 200 None 2026-03-13 00:57:29,367 - lava_callback - DEBUG - Processing node 69b35ed5974c91e64560e9df: ltp-input 2026-03-13 00:57:29,733 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 00:57:29,875 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 00:57:29,877 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-input-69b35ed5974c91e64560e9df/log.txt.gz 2026-03-13 00:57:30,114 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 00:57:30,306 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 00:57:30,308 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-input-69b35ed5974c91e64560e9df/lava-logs.yaml 2026-03-13 00:57:30,317 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 00:57:30,503 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 00:57:30,505 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-input-69b35ed5974c91e64560e9df/lava_callback.json.gz 2026-03-13 00:57:30,531 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:57:30,546 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35ed5974c91e64560e9df HTTP/1.1" 200 None 2026-03-13 00:57:30,547 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:57:30,561 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35934974c91e64560e49d HTTP/1.1" 200 None 2026-03-13 00:57:30,562 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:57:30,596 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b35ed5974c91e64560e9df HTTP/1.1" 200 None 2026-03-13 00:57:30,597 - lava_callback - INFO - Completed processing callback for node 69b35ed5974c91e64560e9df 2026-03-13 00:57:30,838 - lava_callback - INFO - Received callback for node 69b35ed7974c91e64560e9e2 INFO: 172.20.0.1:47636 - "POST /node/69b35ed7974c91e64560e9e2 HTTP/1.0" 202 Accepted 2026-03-13 00:57:30,845 - lava_callback - INFO - Starting async job submit for node 69b35ed7974c91e64560e9e2 2026-03-13 00:57:30,873 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:57:30,887 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35ed7974c91e64560e9e2 HTTP/1.1" 200 None 2026-03-13 00:57:30,888 - lava_callback - DEBUG - Processing node 69b35ed7974c91e64560e9e2: ltp-ipc 2026-03-13 00:57:31,242 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 00:57:31,354 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 00:57:31,356 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-ipc-69b35ed7974c91e64560e9e2/log.txt.gz 2026-03-13 00:57:31,561 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 00:57:31,756 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 00:57:31,759 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-ipc-69b35ed7974c91e64560e9e2/lava-logs.yaml 2026-03-13 00:57:31,767 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 00:57:31,912 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 00:57:31,914 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-ipc-69b35ed7974c91e64560e9e2/lava_callback.json.gz 2026-03-13 00:57:31,940 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:57:31,957 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35ed7974c91e64560e9e2 HTTP/1.1" 200 None 2026-03-13 00:57:31,958 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:57:31,972 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35934974c91e64560e49d HTTP/1.1" 200 None 2026-03-13 00:57:31,973 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:57:32,006 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b35ed7974c91e64560e9e2 HTTP/1.1" 200 None 2026-03-13 00:57:32,007 - lava_callback - INFO - Completed processing callback for node 69b35ed7974c91e64560e9e2 INFO: 172.20.0.1:47644 - "GET / HTTP/1.0" 200 OK 2026-03-13 00:57:44,893 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:57:44,932 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:60020 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36390 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38768 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38776 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49258 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49274 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40670 - "GET / HTTP/1.0" 200 OK 2026-03-13 00:58:26,560 - lava_callback - INFO - Received callback for node 69b0be2a9d191758182527e0 INFO: 172.20.0.1:40676 - "POST /node/69b0be2a9d191758182527e0 HTTP/1.0" 202 Accepted 2026-03-13 00:58:26,584 - lava_callback - INFO - Starting async job submit for node 69b0be2a9d191758182527e0 2026-03-13 00:58:26,586 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:58:26,619 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0be2a9d191758182527e0 HTTP/1.1" 200 None 2026-03-13 00:58:26,622 - lava_callback - DEBUG - Processing node 69b0be2a9d191758182527e0: baseline-arm64-baylibre 2026-03-13 00:58:26,622 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0be2a9d191758182527e0 2026-03-13 00:58:26,623 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:58:26,654 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0be2a9d191758182527e0 HTTP/1.1" 200 None 2026-03-13 00:58:26,657 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:58:26,682 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b5149d19175818251feb HTTP/1.1" 200 None 2026-03-13 00:58:26,684 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:58:26,723 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0be2a9d191758182527e0 HTTP/1.1" 200 None 2026-03-13 00:58:26,725 - lava_callback - ERROR - Error processing callback for node 69b0be2a9d191758182527e0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-13 00:58:27,232 - lava_callback - INFO - Received callback for node 69b0be2b9d191758182527e3 INFO: 172.20.0.1:40678 - "POST /node/69b0be2b9d191758182527e3 HTTP/1.0" 202 Accepted 2026-03-13 00:58:27,241 - lava_callback - INFO - Starting async job submit for node 69b0be2b9d191758182527e3 2026-03-13 00:58:27,243 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:58:27,269 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0be2b9d191758182527e3 HTTP/1.1" 200 None 2026-03-13 00:58:27,270 - lava_callback - DEBUG - Processing node 69b0be2b9d191758182527e3: baseline-arm64-baylibre 2026-03-13 00:58:27,270 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0be2b9d191758182527e3 2026-03-13 00:58:27,271 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:58:27,297 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0be2b9d191758182527e3 HTTP/1.1" 200 None 2026-03-13 00:58:27,300 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:58:27,328 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b5149d19175818251feb HTTP/1.1" 200 None 2026-03-13 00:58:27,331 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:58:27,361 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0be2b9d191758182527e3 HTTP/1.1" 200 None 2026-03-13 00:58:27,363 - lava_callback - ERROR - Error processing callback for node 69b0be2b9d191758182527e3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-13 00:58:27,902 - lava_callback - INFO - Received callback for node 69b0be2f9d191758182527f0 INFO: 172.20.0.1:40680 - "POST /node/69b0be2f9d191758182527f0 HTTP/1.0" 202 Accepted 2026-03-13 00:58:27,910 - lava_callback - INFO - Starting async job submit for node 69b0be2f9d191758182527f0 2026-03-13 00:58:27,917 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:58:27,940 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0be2f9d191758182527f0 HTTP/1.1" 200 None 2026-03-13 00:58:27,942 - lava_callback - DEBUG - Processing node 69b0be2f9d191758182527f0: baseline-arm64-baylibre 2026-03-13 00:58:27,942 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0be2f9d191758182527f0 2026-03-13 00:58:27,943 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:58:27,980 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0be2f9d191758182527f0 HTTP/1.1" 200 None 2026-03-13 00:58:27,982 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:58:28,011 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b5149d19175818251feb HTTP/1.1" 200 None 2026-03-13 00:58:28,014 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:58:28,062 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0be2f9d191758182527f0 HTTP/1.1" 200 None 2026-03-13 00:58:28,064 - lava_callback - ERROR - Error processing callback for node 69b0be2f9d191758182527f0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-13 00:58:28,577 - lava_callback - INFO - Received callback for node 69b0be2d9d191758182527e9 INFO: 172.20.0.1:40696 - "POST /node/69b0be2d9d191758182527e9 HTTP/1.0" 202 Accepted 2026-03-13 00:58:28,588 - lava_callback - INFO - Starting async job submit for node 69b0be2d9d191758182527e9 2026-03-13 00:58:28,590 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:58:28,625 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0be2d9d191758182527e9 HTTP/1.1" 200 None 2026-03-13 00:58:28,627 - lava_callback - DEBUG - Processing node 69b0be2d9d191758182527e9: baseline-arm64-baylibre 2026-03-13 00:58:28,627 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0be2d9d191758182527e9 2026-03-13 00:58:28,628 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:58:28,668 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0be2d9d191758182527e9 HTTP/1.1" 200 None 2026-03-13 00:58:28,670 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:58:28,694 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b5149d19175818251feb HTTP/1.1" 200 None 2026-03-13 00:58:28,697 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:58:28,735 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0be2d9d191758182527e9 HTTP/1.1" 200 None 2026-03-13 00:58:28,737 - lava_callback - ERROR - Error processing callback for node 69b0be2d9d191758182527e9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:44996 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44872 - "GET / HTTP/1.0" 200 OK 2026-03-13 00:58:46,554 - lava_callback - INFO - Received callback for node 69b0be329d191758182527f4 INFO: 172.20.0.1:44882 - "POST /node/69b0be329d191758182527f4 HTTP/1.0" 202 Accepted 2026-03-13 00:58:46,562 - lava_callback - INFO - Starting async job submit for node 69b0be329d191758182527f4 2026-03-13 00:58:46,568 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:58:46,602 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0be329d191758182527f4 HTTP/1.1" 200 None 2026-03-13 00:58:46,603 - lava_callback - DEBUG - Processing node 69b0be329d191758182527f4: baseline-arm64-baylibre 2026-03-13 00:58:46,604 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0be329d191758182527f4 2026-03-13 00:58:46,604 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:58:46,626 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0be329d191758182527f4 HTTP/1.1" 200 None 2026-03-13 00:58:46,628 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:58:46,654 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b5149d19175818251feb HTTP/1.1" 200 None 2026-03-13 00:58:46,656 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:58:46,707 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0be329d191758182527f4 HTTP/1.1" 200 None 2026-03-13 00:58:46,709 - lava_callback - ERROR - Error processing callback for node 69b0be329d191758182527f4: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:50076 - "GET / HTTP/1.0" 200 OK 2026-03-13 00:59:04,608 - lava_callback - INFO - Received callback for node 69b0be439d19175818252842 INFO: 172.20.0.1:45246 - "POST /node/69b0be439d19175818252842 HTTP/1.0" 202 Accepted 2026-03-13 00:59:04,614 - lava_callback - INFO - Starting async job submit for node 69b0be439d19175818252842 2026-03-13 00:59:04,614 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:59:04,628 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0be439d19175818252842 HTTP/1.1" 200 None 2026-03-13 00:59:04,629 - lava_callback - DEBUG - Processing node 69b0be439d19175818252842: baseline-arm64-clabbe 2026-03-13 00:59:04,629 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0be439d19175818252842 2026-03-13 00:59:04,630 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:59:04,652 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0be439d19175818252842 HTTP/1.1" 200 None 2026-03-13 00:59:04,653 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:59:04,667 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b5149d19175818251feb HTTP/1.1" 200 None 2026-03-13 00:59:04,668 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:59:04,688 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0be439d19175818252842 HTTP/1.1" 200 None 2026-03-13 00:59:04,694 - lava_callback - ERROR - Error processing callback for node 69b0be439d19175818252842: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-13 00:59:05,270 - lava_callback - INFO - Received callback for node 69b0be449d19175818252845 INFO: 172.20.0.1:45252 - "POST /node/69b0be449d19175818252845 HTTP/1.0" 202 Accepted 2026-03-13 00:59:05,275 - lava_callback - INFO - Starting async job submit for node 69b0be449d19175818252845 2026-03-13 00:59:05,276 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:59:05,290 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0be449d19175818252845 HTTP/1.1" 200 None 2026-03-13 00:59:05,291 - lava_callback - DEBUG - Processing node 69b0be449d19175818252845: baseline-arm64-clabbe 2026-03-13 00:59:05,291 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0be449d19175818252845 2026-03-13 00:59:05,292 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:59:05,305 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0be449d19175818252845 HTTP/1.1" 200 None 2026-03-13 00:59:05,307 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:59:05,320 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b5149d19175818251feb HTTP/1.1" 200 None 2026-03-13 00:59:05,321 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:59:05,347 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0be449d19175818252845 HTTP/1.1" 200 None 2026-03-13 00:59:05,349 - lava_callback - ERROR - Error processing callback for node 69b0be449d19175818252845: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-13 00:59:05,923 - lava_callback - INFO - Received callback for node 69b0be459d19175818252848 INFO: 172.20.0.1:45262 - "POST /node/69b0be459d19175818252848 HTTP/1.0" 202 Accepted 2026-03-13 00:59:05,928 - lava_callback - INFO - Starting async job submit for node 69b0be459d19175818252848 2026-03-13 00:59:05,929 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:59:05,946 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0be459d19175818252848 HTTP/1.1" 200 None 2026-03-13 00:59:05,947 - lava_callback - DEBUG - Processing node 69b0be459d19175818252848: baseline-arm64-clabbe 2026-03-13 00:59:05,948 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0be459d19175818252848 2026-03-13 00:59:05,948 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:59:05,969 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0be459d19175818252848 HTTP/1.1" 200 None 2026-03-13 00:59:05,971 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:59:05,985 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b5149d19175818251feb HTTP/1.1" 200 None 2026-03-13 00:59:05,986 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:59:06,006 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0be459d19175818252848 HTTP/1.1" 200 None 2026-03-13 00:59:06,012 - lava_callback - ERROR - Error processing callback for node 69b0be459d19175818252848: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:45266 - "GET / HTTP/1.0" 200 OK 2026-03-13 00:59:06,594 - lava_callback - INFO - Received callback for node 69b0be469d1917581825284b INFO: 172.20.0.1:45272 - "POST /node/69b0be469d1917581825284b HTTP/1.0" 202 Accepted 2026-03-13 00:59:06,600 - lava_callback - INFO - Starting async job submit for node 69b0be469d1917581825284b 2026-03-13 00:59:06,601 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:59:06,620 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0be469d1917581825284b HTTP/1.1" 200 None 2026-03-13 00:59:06,621 - lava_callback - DEBUG - Processing node 69b0be469d1917581825284b: baseline-arm64-clabbe 2026-03-13 00:59:06,622 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0be469d1917581825284b 2026-03-13 00:59:06,622 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:59:06,643 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0be469d1917581825284b HTTP/1.1" 200 None 2026-03-13 00:59:06,644 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:59:06,658 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b5149d19175818251feb HTTP/1.1" 200 None 2026-03-13 00:59:06,659 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:59:06,680 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0be469d1917581825284b HTTP/1.1" 200 None 2026-03-13 00:59:06,686 - lava_callback - ERROR - Error processing callback for node 69b0be469d1917581825284b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-13 00:59:07,258 - lava_callback - INFO - Received callback for node 69b0be499d19175818252854 INFO: 172.20.0.1:45278 - "POST /node/69b0be499d19175818252854 HTTP/1.0" 202 Accepted 2026-03-13 00:59:07,263 - lava_callback - INFO - Starting async job submit for node 69b0be499d19175818252854 2026-03-13 00:59:07,264 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:59:07,279 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0be499d19175818252854 HTTP/1.1" 200 None 2026-03-13 00:59:07,280 - lava_callback - DEBUG - Processing node 69b0be499d19175818252854: baseline-arm64-clabbe 2026-03-13 00:59:07,280 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0be499d19175818252854 2026-03-13 00:59:07,281 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:59:07,294 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0be499d19175818252854 HTTP/1.1" 200 None 2026-03-13 00:59:07,295 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:59:07,312 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b5149d19175818251feb HTTP/1.1" 200 None 2026-03-13 00:59:07,313 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:59:07,333 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0be499d19175818252854 HTTP/1.1" 200 None 2026-03-13 00:59:07,338 - lava_callback - ERROR - Error processing callback for node 69b0be499d19175818252854: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-13 00:59:07,927 - lava_callback - INFO - Received callback for node 69b0be489d19175818252851 INFO: 172.20.0.1:45280 - "POST /node/69b0be489d19175818252851 HTTP/1.0" 202 Accepted 2026-03-13 00:59:07,932 - lava_callback - INFO - Starting async job submit for node 69b0be489d19175818252851 2026-03-13 00:59:07,933 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:59:07,947 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0be489d19175818252851 HTTP/1.1" 200 None 2026-03-13 00:59:07,948 - lava_callback - DEBUG - Processing node 69b0be489d19175818252851: baseline-arm64-clabbe 2026-03-13 00:59:07,948 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0be489d19175818252851 2026-03-13 00:59:07,949 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:59:07,962 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0be489d19175818252851 HTTP/1.1" 200 None 2026-03-13 00:59:07,964 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:59:07,978 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b5149d19175818251feb HTTP/1.1" 200 None 2026-03-13 00:59:07,980 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:59:08,001 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0be489d19175818252851 HTTP/1.1" 200 None 2026-03-13 00:59:08,007 - lava_callback - ERROR - Error processing callback for node 69b0be489d19175818252851: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:45282 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55860 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55864 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51444 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49094 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46744 - "GET / HTTP/1.0" 200 OK 2026-03-13 00:59:46,663 - lava_callback - INFO - Received callback for node 69b0be6f9d191758182528b3 INFO: 172.20.0.1:46758 - "POST /node/69b0be6f9d191758182528b3 HTTP/1.0" 202 Accepted 2026-03-13 00:59:46,669 - lava_callback - INFO - Starting async job submit for node 69b0be6f9d191758182528b3 2026-03-13 00:59:46,669 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:59:46,684 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0be6f9d191758182528b3 HTTP/1.1" 200 None 2026-03-13 00:59:46,685 - lava_callback - DEBUG - Processing node 69b0be6f9d191758182528b3: baseline-arm-baylibre 2026-03-13 00:59:46,685 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0be6f9d191758182528b3 2026-03-13 00:59:46,685 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:59:46,698 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0be6f9d191758182528b3 HTTP/1.1" 200 None 2026-03-13 00:59:46,700 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:59:46,714 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e34efa461f55e59cdb7 HTTP/1.1" 200 None 2026-03-13 00:59:46,716 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:59:46,742 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0be6f9d191758182528b3 HTTP/1.1" 200 None 2026-03-13 00:59:46,748 - lava_callback - ERROR - Error processing callback for node 69b0be6f9d191758182528b3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:40984 - "GET / HTTP/1.0" 200 OK 2026-03-13 01:00:04,400 - lava_callback - INFO - Received callback for node 69b35ed8974c91e64560e9e5 INFO: 172.20.0.1:37618 - "POST /node/69b35ed8974c91e64560e9e5 HTTP/1.0" 202 Accepted 2026-03-13 01:00:04,406 - lava_callback - INFO - Starting async job submit for node 69b35ed8974c91e64560e9e5 2026-03-13 01:00:04,431 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:00:04,445 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35ed8974c91e64560e9e5 HTTP/1.1" 200 None 2026-03-13 01:00:04,446 - lava_callback - DEBUG - Processing node 69b35ed8974c91e64560e9e5: ltp-pty 2026-03-13 01:00:04,821 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:00:04,972 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:00:04,974 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-pty-69b35ed8974c91e64560e9e5/log.txt.gz 2026-03-13 01:00:05,176 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:00:05,386 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:00:05,389 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-pty-69b35ed8974c91e64560e9e5/lava-logs.yaml 2026-03-13 01:00:05,400 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:00:05,586 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:00:05,588 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-pty-69b35ed8974c91e64560e9e5/lava_callback.json.gz 2026-03-13 01:00:05,613 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:00:05,628 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35ed8974c91e64560e9e5 HTTP/1.1" 200 None 2026-03-13 01:00:05,629 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:00:05,642 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35934974c91e64560e49d HTTP/1.1" 200 None 2026-03-13 01:00:05,644 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:00:05,673 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b35ed8974c91e64560e9e5 HTTP/1.1" 200 None 2026-03-13 01:00:05,676 - lava_callback - INFO - Completed processing callback for node 69b35ed8974c91e64560e9e5 INFO: 172.20.0.1:37630 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37636 - "GET / HTTP/1.0" 200 OK 2026-03-13 01:00:09,867 - lava_callback - INFO - Received callback for node 69b35eda974c91e64560e9e8 INFO: 172.20.0.1:46288 - "POST /node/69b35eda974c91e64560e9e8 HTTP/1.0" 202 Accepted 2026-03-13 01:00:09,873 - lava_callback - INFO - Starting async job submit for node 69b35eda974c91e64560e9e8 2026-03-13 01:00:09,906 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:00:09,921 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35eda974c91e64560e9e8 HTTP/1.1" 200 None 2026-03-13 01:00:09,922 - lava_callback - DEBUG - Processing node 69b35eda974c91e64560e9e8: ltp-sched 2026-03-13 01:00:10,266 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:00:10,405 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:00:10,406 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-sched-69b35eda974c91e64560e9e8/log.txt.gz 2026-03-13 01:00:10,623 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:00:10,804 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:00:10,806 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-sched-69b35eda974c91e64560e9e8/lava-logs.yaml 2026-03-13 01:00:10,815 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:00:10,979 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:00:10,981 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-sched-69b35eda974c91e64560e9e8/lava_callback.json.gz 2026-03-13 01:00:11,007 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:00:11,025 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35eda974c91e64560e9e8 HTTP/1.1" 200 None 2026-03-13 01:00:11,026 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:00:11,040 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35934974c91e64560e49d HTTP/1.1" 200 None 2026-03-13 01:00:11,042 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:00:11,067 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b35eda974c91e64560e9e8 HTTP/1.1" 200 None 2026-03-13 01:00:11,073 - lava_callback - INFO - Completed processing callback for node 69b35eda974c91e64560e9e8 2026-03-13 01:00:14,935 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:00:14,952 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:46298 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46300 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51192 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41264 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37788 - "GET / HTTP/1.0" 200 OK 2026-03-13 01:00:46,686 - lava_callback - INFO - Received callback for node 69b0beb49d19175818252960 INFO: 172.20.0.1:37800 - "POST /node/69b0beb49d19175818252960 HTTP/1.0" 202 Accepted 2026-03-13 01:00:46,692 - lava_callback - INFO - Starting async job submit for node 69b0beb49d19175818252960 2026-03-13 01:00:46,693 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:00:46,707 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0beb49d19175818252960 HTTP/1.1" 200 None 2026-03-13 01:00:46,708 - lava_callback - DEBUG - Processing node 69b0beb49d19175818252960: baseline-arm-baylibre 2026-03-13 01:00:46,708 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0beb49d19175818252960 2026-03-13 01:00:46,708 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:00:46,722 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0beb49d19175818252960 HTTP/1.1" 200 None 2026-03-13 01:00:46,723 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:00:46,742 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e34efa461f55e59cdb7 HTTP/1.1" 200 None 2026-03-13 01:00:46,744 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:00:46,792 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0beb49d19175818252960 HTTP/1.1" 200 None 2026-03-13 01:00:46,794 - lava_callback - ERROR - Error processing callback for node 69b0beb49d19175818252960: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:58674 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48308 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48312 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50652 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50668 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39672 - "GET / HTTP/1.0" 200 OK 2026-03-13 01:01:29,614 - lava_callback - INFO - Received callback for node 69b35edc974c91e64560e9ed INFO: 172.20.0.1:39688 - "POST /node/69b35edc974c91e64560e9ed HTTP/1.0" 202 Accepted 2026-03-13 01:01:29,620 - lava_callback - INFO - Starting async job submit for node 69b35edc974c91e64560e9ed 2026-03-13 01:01:29,645 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:01:29,658 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35edc974c91e64560e9ed HTTP/1.1" 200 None 2026-03-13 01:01:29,660 - lava_callback - DEBUG - Processing node 69b35edc974c91e64560e9ed: ltp-smoke 2026-03-13 01:01:30,052 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:01:31,213 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:01:31,215 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoke-69b35edc974c91e64560e9ed/log.txt.gz 2026-03-13 01:01:31,418 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:01:31,642 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:01:31,644 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-smoke-69b35edc974c91e64560e9ed/lava-logs.yaml 2026-03-13 01:01:31,653 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:01:31,824 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:01:31,826 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoke-69b35edc974c91e64560e9ed/lava_callback.json.gz 2026-03-13 01:01:31,852 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:01:31,873 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35edc974c91e64560e9ed HTTP/1.1" 200 None 2026-03-13 01:01:31,875 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:01:31,889 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35934974c91e64560e49d HTTP/1.1" 200 None 2026-03-13 01:01:31,891 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:01:31,926 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b35edc974c91e64560e9ed HTTP/1.1" 200 None 2026-03-13 01:01:31,928 - lava_callback - INFO - Completed processing callback for node 69b35edc974c91e64560e9ed INFO: 172.20.0.1:55668 - "GET / HTTP/1.0" 200 OK 2026-03-13 01:01:44,954 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:01:44,971 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:58220 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45308 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41410 - "GET / HTTP/1.0" 200 OK 2026-03-13 01:02:06,846 - lava_callback - INFO - Received callback for node 69b0befc9d191758182529a4 INFO: 172.20.0.1:41424 - "POST /node/69b0befc9d191758182529a4 HTTP/1.0" 202 Accepted 2026-03-13 01:02:06,851 - lava_callback - INFO - Starting async job submit for node 69b0befc9d191758182529a4 2026-03-13 01:02:06,853 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:02:06,872 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0befc9d191758182529a4 HTTP/1.1" 200 None 2026-03-13 01:02:06,873 - lava_callback - DEBUG - Processing node 69b0befc9d191758182529a4: baseline-arm-baylibre 2026-03-13 01:02:06,874 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0befc9d191758182529a4 2026-03-13 01:02:06,874 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:02:06,889 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0befc9d191758182529a4 HTTP/1.1" 200 None 2026-03-13 01:02:06,890 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:02:06,906 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e34efa461f55e59cdb7 HTTP/1.1" 200 None 2026-03-13 01:02:06,909 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:02:06,939 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0befc9d191758182529a4 HTTP/1.1" 200 None 2026-03-13 01:02:06,945 - lava_callback - ERROR - Error processing callback for node 69b0befc9d191758182529a4: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:41436 - "GET / HTTP/1.0" 200 OK 2026-03-13 01:02:09,751 - lava_callback - INFO - Received callback for node 69b35edf974c91e64560e9f7 INFO: 172.20.0.1:58328 - "POST /node/69b35edf974c91e64560e9f7 HTTP/1.0" 202 Accepted 2026-03-13 01:02:09,764 - lava_callback - INFO - Starting async job submit for node 69b35edf974c91e64560e9f7 2026-03-13 01:02:09,814 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:02:09,838 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35edf974c91e64560e9f7 HTTP/1.1" 200 None 2026-03-13 01:02:09,839 - lava_callback - DEBUG - Processing node 69b35edf974c91e64560e9f7: ltp-syscalls-ipc 2026-03-13 01:02:10,557 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:02:10,907 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:02:10,909 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-syscalls-ipc-69b35edf974c91e64560e9f7/log.txt.gz 2026-03-13 01:02:11,322 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:02:11,844 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:02:11,846 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-syscalls-ipc-69b35edf974c91e64560e9f7/lava-logs.yaml 2026-03-13 01:02:11,859 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:02:12,288 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:02:12,290 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-syscalls-ipc-69b35edf974c91e64560e9f7/lava_callback.json.gz 2026-03-13 01:02:12,342 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:02:12,368 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35edf974c91e64560e9f7 HTTP/1.1" 200 None 2026-03-13 01:02:12,371 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:02:12,394 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35934974c91e64560e49d HTTP/1.1" 200 None 2026-03-13 01:02:12,396 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:02:12,449 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b35edf974c91e64560e9f7 HTTP/1.1" 200 None 2026-03-13 01:02:12,451 - lava_callback - INFO - Completed processing callback for node 69b35edf974c91e64560e9f7 2026-03-13 01:02:14,973 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:02:15,007 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:58330 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58346 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60054 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41688 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41440 - "GET / HTTP/1.0" 200 OK 2026-03-13 01:02:46,784 - lava_callback - INFO - Received callback for node 69b0bf229d191758182529b9 INFO: 172.20.0.1:41450 - "POST /node/69b0bf229d191758182529b9 HTTP/1.0" 202 Accepted 2026-03-13 01:02:46,793 - lava_callback - INFO - Starting async job submit for node 69b0bf229d191758182529b9 2026-03-13 01:02:46,795 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:02:46,827 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0bf229d191758182529b9 HTTP/1.1" 200 None 2026-03-13 01:02:46,828 - lava_callback - DEBUG - Processing node 69b0bf229d191758182529b9: baseline-arm-baylibre 2026-03-13 01:02:46,828 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0bf229d191758182529b9 2026-03-13 01:02:46,829 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:02:46,854 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0bf229d191758182529b9 HTTP/1.1" 200 None 2026-03-13 01:02:46,865 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:02:46,897 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e34efa461f55e59cdb7 HTTP/1.1" 200 None 2026-03-13 01:02:46,899 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:02:46,936 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0bf229d191758182529b9 HTTP/1.1" 200 None 2026-03-13 01:02:46,937 - lava_callback - ERROR - Error processing callback for node 69b0bf229d191758182529b9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:59462 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53830 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53842 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48604 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48618 - "GET / HTTP/1.0" 200 OK 2026-03-13 01:03:17,845 - lava_callback - INFO - Received callback for node 69b35e90974c91e64560e90a INFO: 172.20.0.1:48622 - "POST /node/69b35e90974c91e64560e90a HTTP/1.0" 202 Accepted 2026-03-13 01:03:17,855 - lava_callback - INFO - Starting async job submit for node 69b35e90974c91e64560e90a 2026-03-13 01:03:17,963 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:03:17,995 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35e90974c91e64560e90a HTTP/1.1" 200 None 2026-03-13 01:03:17,996 - lava_callback - DEBUG - Processing node 69b35e90974c91e64560e90a: baseline-arm 2026-03-13 01:03:19,010 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:03:20,604 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:03:20,607 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69b35e90974c91e64560e90a/log.txt.gz 2026-03-13 01:03:21,250 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:03:24,789 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:03:24,792 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69b35e90974c91e64560e90a/lava-logs.yaml 2026-03-13 01:03:24,813 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:48554 - "GET / HTTP/1.0" 200 OK 2026-03-13 01:03:27,247 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:03:27,249 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69b35e90974c91e64560e90a/lava_callback.json.gz DEBUG: Setup failed for 69b35e90974c91e64560e90a. Transitting job node result: incomplete -> fail 2026-03-13 01:03:27,456 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:03:27,482 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35e90974c91e64560e90a HTTP/1.1" 200 None 2026-03-13 01:03:27,484 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:03:27,505 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35934974c91e64560e49d HTTP/1.1" 200 None 2026-03-13 01:03:27,507 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:03:27,551 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b35e90974c91e64560e90a HTTP/1.1" 200 None 2026-03-13 01:03:27,731 - lava_callback - INFO - Completed processing callback for node 69b35e90974c91e64560e90a INFO: 172.20.0.1:57228 - "GET / HTTP/1.0" 200 OK 2026-03-13 01:03:45,013 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:03:45,053 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:42984 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52016 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40778 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40794 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38542 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38544 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46002 - "GET / HTTP/1.0" 200 OK 2026-03-13 01:04:29,313 - lava_callback - INFO - Received callback for node 69b36283974c91e64560ec59 INFO: 172.20.0.1:46004 - "POST /node/69b36283974c91e64560ec59 HTTP/1.0" 202 Accepted 2026-03-13 01:04:29,322 - lava_callback - INFO - Starting async job submit for node 69b36283974c91e64560ec59 2026-03-13 01:04:29,332 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:04:29,365 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b36283974c91e64560ec59 HTTP/1.1" 200 None 2026-03-13 01:04:29,366 - lava_callback - DEBUG - Processing node 69b36283974c91e64560ec59: baseline-arm 2026-03-13 01:04:29,375 - lava_callback - WARNING - Failed to upload log 2026-03-13 01:04:29,381 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:04:30,608 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:04:30,610 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69b36283974c91e64560ec59/lava-logs.yaml 2026-03-13 01:04:30,612 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:04:31,769 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:04:31,772 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69b36283974c91e64560ec59/lava_callback.json.gz 2026-03-13 01:04:31,776 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:04:31,801 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b36283974c91e64560ec59 HTTP/1.1" 200 None 2026-03-13 01:04:31,803 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:04:31,843 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35934974c91e64560e49d HTTP/1.1" 200 None 2026-03-13 01:04:31,846 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:04:31,883 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b36283974c91e64560ec59 HTTP/1.1" 200 None 2026-03-13 01:04:31,889 - lava_callback - INFO - Completed processing callback for node 69b36283974c91e64560ec59 INFO: 172.20.0.1:40940 - "GET / HTTP/1.0" 200 OK 2026-03-13 01:04:45,056 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:04:45,088 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:46008 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59336 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34572 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34584 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42976 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42980 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52368 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43980 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43840 - "GET / HTTP/1.0" 200 OK 2026-03-13 01:05:47,038 - lava_callback - INFO - Received callback for node 69b0bfe59d19175818252a0b INFO: 172.20.0.1:43842 - "POST /node/69b0bfe59d19175818252a0b HTTP/1.0" 202 Accepted 2026-03-13 01:05:47,046 - lava_callback - INFO - Starting async job submit for node 69b0bfe59d19175818252a0b 2026-03-13 01:05:47,048 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:05:47,071 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0bfe59d19175818252a0b HTTP/1.1" 200 None 2026-03-13 01:05:47,072 - lava_callback - DEBUG - Processing node 69b0bfe59d19175818252a0b: baseline-arm-baylibre 2026-03-13 01:05:47,072 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0bfe59d19175818252a0b 2026-03-13 01:05:47,073 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:05:47,096 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0bfe59d19175818252a0b HTTP/1.1" 200 None 2026-03-13 01:05:47,099 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:05:47,136 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e34efa461f55e59cdb7 HTTP/1.1" 200 None 2026-03-13 01:05:47,138 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:05:47,180 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0bfe59d19175818252a0b HTTP/1.1" 200 None 2026-03-13 01:05:47,182 - lava_callback - ERROR - Error processing callback for node 69b0bfe59d19175818252a0b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:60598 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36586 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36594 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57504 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57520 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51842 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38966 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51154 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54840 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58054 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58070 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42698 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42704 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51342 - "GET / HTTP/1.0" 200 OK 2026-03-13 01:07:29,502 - lava_callback - INFO - Received callback for node 69b35eb5974c91e64560e996 INFO: 172.20.0.1:51358 - "POST /node/69b35eb5974c91e64560e996 HTTP/1.0" 202 Accepted 2026-03-13 01:07:29,553 - lava_callback - INFO - Starting async job submit for node 69b35eb5974c91e64560e996 2026-03-13 01:07:29,594 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:07:29,620 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35eb5974c91e64560e996 HTTP/1.1" 200 None 2026-03-13 01:07:29,622 - lava_callback - DEBUG - Processing node 69b35eb5974c91e64560e996: baseline-arm-pengutronix 2026-03-13 01:07:29,827 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:07:31,194 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:07:31,196 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69b35eb5974c91e64560e996/log.txt.gz 2026-03-13 01:07:31,310 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:07:33,225 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:07:33,228 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69b35eb5974c91e64560e996/lava-logs.yaml 2026-03-13 01:07:33,231 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:07:34,681 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:07:34,683 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69b35eb5974c91e64560e996/lava_callback.json.gz DEBUG: Setup failed for 69b35eb5974c91e64560e996. Transitting job node result: incomplete -> fail 2026-03-13 01:07:34,779 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:07:34,808 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35eb5974c91e64560e996 HTTP/1.1" 200 None 2026-03-13 01:07:34,810 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:07:34,834 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35934974c91e64560e49d HTTP/1.1" 200 None 2026-03-13 01:07:34,836 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:07:34,896 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b35eb5974c91e64560e996 HTTP/1.1" 200 None 2026-03-13 01:07:35,011 - lava_callback - INFO - Completed processing callback for node 69b35eb5974c91e64560e996 INFO: 172.20.0.1:48050 - "GET / HTTP/1.0" 200 OK 2026-03-13 01:07:45,091 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:07:45,118 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:56938 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41180 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45122 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45134 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55268 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55272 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33348 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50192 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47968 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51982 - "GET / HTTP/1.0" 200 OK 2026-03-13 01:09:05,139 - lava_callback - INFO - Received callback for node 69b0c0a99d19175818252a45 INFO: 172.20.0.1:43672 - "POST /node/69b0c0a99d19175818252a45 HTTP/1.0" 202 Accepted 2026-03-13 01:09:05,145 - lava_callback - INFO - Starting async job submit for node 69b0c0a99d19175818252a45 2026-03-13 01:09:05,146 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:09:05,161 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0c0a99d19175818252a45 HTTP/1.1" 200 None 2026-03-13 01:09:05,162 - lava_callback - DEBUG - Processing node 69b0c0a99d19175818252a45: baseline-arm-clabbe 2026-03-13 01:09:05,163 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0c0a99d19175818252a45 2026-03-13 01:09:05,163 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:09:05,177 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0c0a99d19175818252a45 HTTP/1.1" 200 None 2026-03-13 01:09:05,178 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:09:05,200 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e34efa461f55e59cdb7 HTTP/1.1" 200 None 2026-03-13 01:09:05,202 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:09:05,225 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0c0a99d19175818252a45 HTTP/1.1" 200 None 2026-03-13 01:09:05,232 - lava_callback - ERROR - Error processing callback for node 69b0c0a99d19175818252a45: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:43684 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43690 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56902 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56916 - "GET / HTTP/1.0" 200 OK 2026-03-13 01:09:17,185 - lava_callback - INFO - Received callback for node 69b36399974c91e64560ecc3 INFO: 172.20.0.1:56930 - "POST /node/69b36399974c91e64560ecc3 HTTP/1.0" 202 Accepted 2026-03-13 01:09:17,191 - lava_callback - INFO - Starting async job submit for node 69b36399974c91e64560ecc3 2026-03-13 01:09:17,210 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:09:17,224 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b36399974c91e64560ecc3 HTTP/1.1" 200 None 2026-03-13 01:09:17,225 - lava_callback - DEBUG - Processing node 69b36399974c91e64560ecc3: baseline-arm64 2026-03-13 01:09:17,378 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:09:17,604 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:09:17,607 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69b36399974c91e64560ecc3/log.txt.gz 2026-03-13 01:09:17,701 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:09:17,878 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:09:17,881 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69b36399974c91e64560ecc3/lava-logs.yaml 2026-03-13 01:09:17,887 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:09:18,025 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:09:18,027 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69b36399974c91e64560ecc3/lava_callback.json.gz 2026-03-13 01:09:18,046 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:09:18,061 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b36399974c91e64560ecc3 HTTP/1.1" 200 None 2026-03-13 01:09:18,063 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:09:18,076 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3594d974c91e64560e4b2 HTTP/1.1" 200 None 2026-03-13 01:09:18,078 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:09:18,108 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b36399974c91e64560ecc3 HTTP/1.1" 200 None 2026-03-13 01:09:18,110 - lava_callback - INFO - Completed processing callback for node 69b36399974c91e64560ecc3 INFO: 172.20.0.1:59406 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48396 - "GET / HTTP/1.0" 200 OK 2026-03-13 01:09:40,489 - lava_callback - INFO - Received callback for node 69b363b6974c91e64560ed18 INFO: 172.20.0.1:43148 - "POST /node/69b363b6974c91e64560ed18 HTTP/1.0" 202 Accepted 2026-03-13 01:09:40,495 - lava_callback - INFO - Starting async job submit for node 69b363b6974c91e64560ed18 2026-03-13 01:09:40,497 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:09:40,555 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b363b6974c91e64560ed18 HTTP/1.1" 200 None 2026-03-13 01:09:40,557 - lava_callback - DEBUG - Processing node 69b363b6974c91e64560ed18: baseline-nfs-arm64 2026-03-13 01:09:40,560 - lava_callback - WARNING - Failed to upload log 2026-03-13 01:09:40,562 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:09:40,768 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:09:40,770 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b363b6974c91e64560ed18/lava-logs.yaml 2026-03-13 01:09:40,773 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:09:40,943 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:09:40,945 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b363b6974c91e64560ed18/lava_callback.json.gz 2026-03-13 01:09:40,947 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:09:40,961 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b363b6974c91e64560ed18 HTTP/1.1" 200 None 2026-03-13 01:09:40,962 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:09:40,975 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3594d974c91e64560e4b2 HTTP/1.1" 200 None 2026-03-13 01:09:40,977 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:09:41,000 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b363b6974c91e64560ed18 HTTP/1.1" 200 None 2026-03-13 01:09:41,005 - lava_callback - INFO - Completed processing callback for node 69b363b6974c91e64560ed18 2026-03-13 01:09:41,918 - lava_callback - INFO - Received callback for node 69b363a5974c91e64560ecef INFO: 172.20.0.1:43164 - "POST /node/69b363a5974c91e64560ecef HTTP/1.0" 202 Accepted 2026-03-13 01:09:41,928 - lava_callback - INFO - Starting async job submit for node 69b363a5974c91e64560ecef 2026-03-13 01:09:41,950 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:09:41,970 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b363a5974c91e64560ecef HTTP/1.1" 200 None 2026-03-13 01:09:41,971 - lava_callback - DEBUG - Processing node 69b363a5974c91e64560ecef: baseline-arm64 2026-03-13 01:09:42,125 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:09:42,241 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:09:42,243 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69b363a5974c91e64560ecef/log.txt.gz 2026-03-13 01:09:42,333 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:09:42,503 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:09:42,504 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69b363a5974c91e64560ecef/lava-logs.yaml 2026-03-13 01:09:42,509 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:09:42,656 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:09:42,658 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69b363a5974c91e64560ecef/lava_callback.json.gz 2026-03-13 01:09:42,677 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:09:42,695 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b363a5974c91e64560ecef HTTP/1.1" 200 None 2026-03-13 01:09:42,697 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:09:42,715 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3594d974c91e64560e4b2 HTTP/1.1" 200 None 2026-03-13 01:09:42,717 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:09:42,754 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b363a5974c91e64560ecef HTTP/1.1" 200 None 2026-03-13 01:09:42,756 - lava_callback - INFO - Completed processing callback for node 69b363a5974c91e64560ecef 2026-03-13 01:09:45,120 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:09:45,141 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-13 01:09:46,491 - lava_callback - INFO - Received callback for node 69b3639d974c91e64560eccf INFO: 172.20.0.1:43170 - "POST /node/69b3639d974c91e64560eccf HTTP/1.0" 202 Accepted 2026-03-13 01:09:46,517 - lava_callback - INFO - Starting async job submit for node 69b3639d974c91e64560eccf INFO: 172.20.0.1:43178 - "GET / HTTP/1.0" 200 OK 2026-03-13 01:09:46,552 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:09:46,568 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3639d974c91e64560eccf HTTP/1.1" 200 None 2026-03-13 01:09:46,569 - lava_callback - DEBUG - Processing node 69b3639d974c91e64560eccf: baseline-arm64 2026-03-13 01:09:46,806 - lava_callback - WARNING - Failed to upload log 2026-03-13 01:09:46,904 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:09:47,100 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:09:47,101 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69b3639d974c91e64560eccf/lava-logs.yaml 2026-03-13 01:09:47,105 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:09:47,222 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:09:47,224 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69b3639d974c91e64560eccf/lava_callback.json.gz 2026-03-13 01:09:47,255 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:09:47,269 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3639d974c91e64560eccf HTTP/1.1" 200 None 2026-03-13 01:09:47,271 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:09:47,284 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3594d974c91e64560e4b2 HTTP/1.1" 200 None 2026-03-13 01:09:47,286 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:09:47,306 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b3639d974c91e64560eccf HTTP/1.1" 200 None 2026-03-13 01:09:47,364 - lava_callback - INFO - Completed processing callback for node 69b3639d974c91e64560eccf 2026-03-13 01:09:47,796 - lava_callback - INFO - Received callback for node 69b3639e974c91e64560ecdd INFO: 172.20.0.1:43184 - "POST /node/69b3639e974c91e64560ecdd HTTP/1.0" 202 Accepted 2026-03-13 01:09:47,803 - lava_callback - INFO - Starting async job submit for node 69b3639e974c91e64560ecdd 2026-03-13 01:09:47,834 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:09:47,853 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3639e974c91e64560ecdd HTTP/1.1" 200 None 2026-03-13 01:09:47,853 - lava_callback - DEBUG - Processing node 69b3639e974c91e64560ecdd: baseline-arm64 2026-03-13 01:09:48,003 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:09:48,116 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:09:48,118 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69b3639e974c91e64560ecdd/log.txt.gz 2026-03-13 01:09:48,191 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:09:48,457 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:09:48,459 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69b3639e974c91e64560ecdd/lava-logs.yaml 2026-03-13 01:09:48,464 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:09:48,627 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:09:48,629 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69b3639e974c91e64560ecdd/lava_callback.json.gz 2026-03-13 01:09:48,670 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:09:48,689 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3639e974c91e64560ecdd HTTP/1.1" 200 None 2026-03-13 01:09:48,690 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:09:48,705 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3594d974c91e64560e4b2 HTTP/1.1" 200 None 2026-03-13 01:09:48,706 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:09:48,730 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b3639e974c91e64560ecdd HTTP/1.1" 200 None 2026-03-13 01:09:48,793 - lava_callback - INFO - Completed processing callback for node 69b3639e974c91e64560ecdd INFO: 172.20.0.1:33278 - "GET / HTTP/1.0" 200 OK 2026-03-13 01:09:57,095 - lava_callback - INFO - Received callback for node 69b363a7974c91e64560ecf2 INFO: 172.20.0.1:33286 - "POST /node/69b363a7974c91e64560ecf2 HTTP/1.0" 202 Accepted 2026-03-13 01:09:57,103 - lava_callback - INFO - Starting async job submit for node 69b363a7974c91e64560ecf2 2026-03-13 01:09:57,134 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:09:57,149 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b363a7974c91e64560ecf2 HTTP/1.1" 200 None 2026-03-13 01:09:57,150 - lava_callback - DEBUG - Processing node 69b363a7974c91e64560ecf2: baseline-arm64 2026-03-13 01:09:57,424 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:09:57,541 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:09:57,543 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69b363a7974c91e64560ecf2/log.txt.gz 2026-03-13 01:09:57,693 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:09:57,878 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:09:57,880 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69b363a7974c91e64560ecf2/lava-logs.yaml 2026-03-13 01:09:57,888 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:09:58,052 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:09:58,054 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69b363a7974c91e64560ecf2/lava_callback.json.gz 2026-03-13 01:09:58,077 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:09:58,094 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b363a7974c91e64560ecf2 HTTP/1.1" 200 None 2026-03-13 01:09:58,096 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:09:58,113 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3594d974c91e64560e4b2 HTTP/1.1" 200 None 2026-03-13 01:09:58,115 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:09:58,163 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b363a7974c91e64560ecf2 HTTP/1.1" 200 None 2026-03-13 01:09:58,166 - lava_callback - INFO - Completed processing callback for node 69b363a7974c91e64560ecf2 2026-03-13 01:10:05,807 - lava_callback - INFO - Received callback for node 69b3639a974c91e64560ecc9 INFO: 172.20.0.1:34308 - "POST /node/69b3639a974c91e64560ecc9 HTTP/1.0" 202 Accepted 2026-03-13 01:10:05,814 - lava_callback - INFO - Starting async job submit for node 69b3639a974c91e64560ecc9 2026-03-13 01:10:05,857 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:10:05,874 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3639a974c91e64560ecc9 HTTP/1.1" 200 None 2026-03-13 01:10:05,875 - lava_callback - DEBUG - Processing node 69b3639a974c91e64560ecc9: baseline-arm64 2026-03-13 01:10:06,167 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:10:06,289 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:10:06,291 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69b3639a974c91e64560ecc9/log.txt.gz 2026-03-13 01:10:06,489 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:34316 - "GET / HTTP/1.0" 200 OK 2026-03-13 01:10:06,743 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:10:06,745 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69b3639a974c91e64560ecc9/lava-logs.yaml 2026-03-13 01:10:06,752 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:10:06,900 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:10:06,902 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69b3639a974c91e64560ecc9/lava_callback.json.gz 2026-03-13 01:10:06,934 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:10:06,947 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3639a974c91e64560ecc9 HTTP/1.1" 200 None 2026-03-13 01:10:06,949 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:10:06,963 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3594d974c91e64560e4b2 HTTP/1.1" 200 None 2026-03-13 01:10:06,965 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:10:06,992 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b3639a974c91e64560ecc9 HTTP/1.1" 200 None 2026-03-13 01:10:06,997 - lava_callback - INFO - Completed processing callback for node 69b3639a974c91e64560ecc9 INFO: 172.20.0.1:34320 - "GET / HTTP/1.0" 200 OK 2026-03-13 01:10:11,606 - lava_callback - INFO - Received callback for node 69b363a9974c91e64560ecf8 INFO: 172.20.0.1:47834 - "POST /node/69b363a9974c91e64560ecf8 HTTP/1.0" 202 Accepted 2026-03-13 01:10:11,613 - lava_callback - INFO - Starting async job submit for node 69b363a9974c91e64560ecf8 2026-03-13 01:10:11,634 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:10:11,649 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b363a9974c91e64560ecf8 HTTP/1.1" 200 None 2026-03-13 01:10:11,650 - lava_callback - DEBUG - Processing node 69b363a9974c91e64560ecf8: baseline-arm64 2026-03-13 01:10:11,923 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:10:12,044 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:10:12,046 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69b363a9974c91e64560ecf8/log.txt.gz 2026-03-13 01:10:12,210 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:10:12,399 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:10:12,401 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69b363a9974c91e64560ecf8/lava-logs.yaml 2026-03-13 01:10:12,408 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:10:12,558 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:10:12,560 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69b363a9974c91e64560ecf8/lava_callback.json.gz 2026-03-13 01:10:12,583 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:10:12,600 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b363a9974c91e64560ecf8 HTTP/1.1" 200 None 2026-03-13 01:10:12,602 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:10:12,626 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3594d974c91e64560e4b2 HTTP/1.1" 200 None 2026-03-13 01:10:12,628 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:10:12,659 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b363a9974c91e64560ecf8 HTTP/1.1" 200 None 2026-03-13 01:10:12,661 - lava_callback - INFO - Completed processing callback for node 69b363a9974c91e64560ecf8 2026-03-13 01:10:15,143 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:10:15,160 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 INFO: 172.20.0.1:47848 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47860 - "GET / HTTP/1.0" 200 OK 2026-03-13 01:10:17,253 - lava_callback - INFO - Received callback for node 69b3639b974c91e64560eccc INFO: 172.20.0.1:47870 - "POST /node/69b3639b974c91e64560eccc HTTP/1.0" 202 Accepted 2026-03-13 01:10:17,259 - lava_callback - INFO - Starting async job submit for node 69b3639b974c91e64560eccc 2026-03-13 01:10:17,282 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:10:17,297 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3639b974c91e64560eccc HTTP/1.1" 200 None 2026-03-13 01:10:17,298 - lava_callback - DEBUG - Processing node 69b3639b974c91e64560eccc: baseline-arm64 2026-03-13 01:10:17,566 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:10:17,693 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:10:17,695 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69b3639b974c91e64560eccc/log.txt.gz 2026-03-13 01:10:17,844 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:10:18,034 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:10:18,036 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69b3639b974c91e64560eccc/lava-logs.yaml 2026-03-13 01:10:18,042 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:10:18,168 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:10:18,169 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69b3639b974c91e64560eccc/lava_callback.json.gz 2026-03-13 01:10:18,200 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:10:18,215 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3639b974c91e64560eccc HTTP/1.1" 200 None 2026-03-13 01:10:18,216 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:10:18,230 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3594d974c91e64560e4b2 HTTP/1.1" 200 None 2026-03-13 01:10:18,232 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:10:18,257 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b3639b974c91e64560eccc HTTP/1.1" 200 None 2026-03-13 01:10:18,259 - lava_callback - INFO - Completed processing callback for node 69b3639b974c91e64560eccc 2026-03-13 01:10:19,561 - lava_callback - INFO - Received callback for node 69b3639a974c91e64560ecc6 INFO: 172.20.0.1:47872 - "POST /node/69b3639a974c91e64560ecc6 HTTP/1.0" 202 Accepted 2026-03-13 01:10:19,567 - lava_callback - INFO - Starting async job submit for node 69b3639a974c91e64560ecc6 2026-03-13 01:10:19,589 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:10:19,610 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3639a974c91e64560ecc6 HTTP/1.1" 200 None 2026-03-13 01:10:19,611 - lava_callback - DEBUG - Processing node 69b3639a974c91e64560ecc6: baseline-arm64 2026-03-13 01:10:19,834 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:10:19,973 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:10:19,975 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69b3639a974c91e64560ecc6/log.txt.gz 2026-03-13 01:10:20,113 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:10:20,278 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:10:20,280 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69b3639a974c91e64560ecc6/lava-logs.yaml 2026-03-13 01:10:20,286 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:10:20,408 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:10:20,410 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69b3639a974c91e64560ecc6/lava_callback.json.gz 2026-03-13 01:10:20,432 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:10:20,446 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3639a974c91e64560ecc6 HTTP/1.1" 200 None 2026-03-13 01:10:20,448 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:10:20,462 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3594d974c91e64560e4b2 HTTP/1.1" 200 None 2026-03-13 01:10:20,464 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:10:20,491 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b3639a974c91e64560ecc6 HTTP/1.1" 200 None 2026-03-13 01:10:20,499 - lava_callback - INFO - Completed processing callback for node 69b3639a974c91e64560ecc6 2026-03-13 01:10:22,443 - lava_callback - INFO - Received callback for node 69b363aa974c91e64560ecfb INFO: 172.20.0.1:55134 - "POST /node/69b363aa974c91e64560ecfb HTTP/1.0" 202 Accepted 2026-03-13 01:10:22,449 - lava_callback - INFO - Starting async job submit for node 69b363aa974c91e64560ecfb 2026-03-13 01:10:22,470 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:10:22,485 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b363aa974c91e64560ecfb HTTP/1.1" 200 None 2026-03-13 01:10:22,486 - lava_callback - DEBUG - Processing node 69b363aa974c91e64560ecfb: baseline-arm64 2026-03-13 01:10:22,763 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:10:22,891 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:10:22,893 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69b363aa974c91e64560ecfb/log.txt.gz 2026-03-13 01:10:23,047 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:10:23,220 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:10:23,222 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69b363aa974c91e64560ecfb/lava-logs.yaml 2026-03-13 01:10:23,229 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:10:23,411 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:10:23,412 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69b363aa974c91e64560ecfb/lava_callback.json.gz 2026-03-13 01:10:23,435 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:10:23,449 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b363aa974c91e64560ecfb HTTP/1.1" 200 None 2026-03-13 01:10:23,451 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:10:23,466 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3594d974c91e64560e4b2 HTTP/1.1" 200 None 2026-03-13 01:10:23,467 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:10:23,501 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b363aa974c91e64560ecfb HTTP/1.1" 200 None 2026-03-13 01:10:23,503 - lava_callback - INFO - Completed processing callback for node 69b363aa974c91e64560ecfb 2026-03-13 01:10:25,064 - lava_callback - INFO - Received callback for node 69b0c0f19d19175818252a5c INFO: 172.20.0.1:55146 - "POST /node/69b0c0f19d19175818252a5c HTTP/1.0" 202 Accepted 2026-03-13 01:10:25,070 - lava_callback - INFO - Starting async job submit for node 69b0c0f19d19175818252a5c 2026-03-13 01:10:25,071 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:10:25,085 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0c0f19d19175818252a5c HTTP/1.1" 200 None 2026-03-13 01:10:25,086 - lava_callback - DEBUG - Processing node 69b0c0f19d19175818252a5c: baseline-arm-clabbe 2026-03-13 01:10:25,086 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0c0f19d19175818252a5c 2026-03-13 01:10:25,087 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:10:25,102 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0c0f19d19175818252a5c HTTP/1.1" 200 None 2026-03-13 01:10:25,104 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:10:25,119 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e34efa461f55e59cdb7 HTTP/1.1" 200 None 2026-03-13 01:10:25,121 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:10:25,141 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0c0f19d19175818252a5c HTTP/1.1" 200 None 2026-03-13 01:10:25,149 - lava_callback - ERROR - Error processing callback for node 69b0c0f19d19175818252a5c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-13 01:10:26,490 - lava_callback - INFO - Received callback for node 69b363a0974c91e64560ece3 INFO: 172.20.0.1:55148 - "POST /node/69b363a0974c91e64560ece3 HTTP/1.0" 202 Accepted 2026-03-13 01:10:26,504 - lava_callback - INFO - Starting async job submit for node 69b363a0974c91e64560ece3 INFO: 172.20.0.1:55160 - "GET / HTTP/1.0" 200 OK 2026-03-13 01:10:26,544 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:10:26,595 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b363a0974c91e64560ece3 HTTP/1.1" 200 None 2026-03-13 01:10:26,596 - lava_callback - DEBUG - Processing node 69b363a0974c91e64560ece3: baseline-arm64 2026-03-13 01:10:27,061 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:10:27,366 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:10:27,368 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69b363a0974c91e64560ece3/log.txt.gz 2026-03-13 01:10:27,628 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:10:28,176 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:10:28,179 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69b363a0974c91e64560ece3/lava-logs.yaml 2026-03-13 01:10:28,192 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:10:28,599 - lava_callback - INFO - Received callback for node 69b363b1974c91e64560ed0c 2026-03-13 01:10:28,606 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:10:28,611 - lava_callback - INFO - Starting async job submit for node 69b363b1974c91e64560ed0c INFO: 172.20.0.1:55174 - "POST /node/69b363b1974c91e64560ed0c HTTP/1.0" 202 Accepted 2026-03-13 01:10:28,635 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69b363a0974c91e64560ece3/lava_callback.json.gz 2026-03-13 01:10:28,696 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:10:28,719 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:10:28,729 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b363b1974c91e64560ed0c HTTP/1.1" 200 None 2026-03-13 01:10:28,730 - lava_callback - DEBUG - Processing node 69b363b1974c91e64560ed0c: baseline-nfs-arm64 2026-03-13 01:10:28,773 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b363a0974c91e64560ece3 HTTP/1.1" 200 None 2026-03-13 01:10:28,776 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:10:28,842 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3594d974c91e64560e4b2 HTTP/1.1" 200 None 2026-03-13 01:10:28,845 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:10:28,925 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b363a0974c91e64560ece3 HTTP/1.1" 200 None 2026-03-13 01:10:28,926 - lava_callback - INFO - Completed processing callback for node 69b363a0974c91e64560ece3 2026-03-13 01:10:29,690 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:10:30,090 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:10:30,093 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b363b1974c91e64560ed0c/log.txt.gz 2026-03-13 01:10:30,452 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:10:30,960 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:10:30,963 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b363b1974c91e64560ed0c/lava-logs.yaml 2026-03-13 01:10:30,977 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:10:31,267 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:10:31,269 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b363b1974c91e64560ed0c/lava_callback.json.gz 2026-03-13 01:10:31,319 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:10:31,355 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b363b1974c91e64560ed0c HTTP/1.1" 200 None 2026-03-13 01:10:31,358 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:10:31,384 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3594d974c91e64560e4b2 HTTP/1.1" 200 None 2026-03-13 01:10:31,387 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:10:31,440 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b363b1974c91e64560ed0c HTTP/1.1" 200 None 2026-03-13 01:10:31,443 - lava_callback - INFO - Completed processing callback for node 69b363b1974c91e64560ed0c INFO: 172.20.0.1:57798 - "GET / HTTP/1.0" 200 OK 2026-03-13 01:10:42,796 - lava_callback - INFO - Received callback for node 69b363ac974c91e64560ecfe INFO: 172.20.0.1:44218 - "POST /node/69b363ac974c91e64560ecfe HTTP/1.0" 202 Accepted 2026-03-13 01:10:42,809 - lava_callback - INFO - Starting async job submit for node 69b363ac974c91e64560ecfe 2026-03-13 01:10:42,867 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:10:42,894 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b363ac974c91e64560ecfe HTTP/1.1" 200 None 2026-03-13 01:10:42,895 - lava_callback - DEBUG - Processing node 69b363ac974c91e64560ecfe: baseline-arm64 2026-03-13 01:10:43,439 - lava_callback - INFO - Received callback for node 69b363bc974c91e64560ed44 INFO: 172.20.0.1:44226 - "POST /node/69b363bc974c91e64560ed44 HTTP/1.0" 202 Accepted 2026-03-13 01:10:43,467 - lava_callback - INFO - Starting async job submit for node 69b363bc974c91e64560ed44 2026-03-13 01:10:43,522 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:10:43,539 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:10:43,575 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b363bc974c91e64560ed44 HTTP/1.1" 200 None 2026-03-13 01:10:43,577 - lava_callback - DEBUG - Processing node 69b363bc974c91e64560ed44: baseline-nfs-arm64 2026-03-13 01:10:43,966 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:10:43,968 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69b363ac974c91e64560ecfe/log.txt.gz 2026-03-13 01:10:44,732 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:10:44,766 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:10:45,079 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:10:45,081 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b363bc974c91e64560ed44/log.txt.gz 2026-03-13 01:10:45,165 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:10:45,249 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 2026-03-13 01:10:45,335 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:10:45,338 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69b363ac974c91e64560ecfe/lava-logs.yaml 2026-03-13 01:10:45,348 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:10:45,585 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:10:45,814 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:10:45,816 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69b363ac974c91e64560ecfe/lava_callback.json.gz 2026-03-13 01:10:45,894 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:10:45,918 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b363ac974c91e64560ecfe HTTP/1.1" 200 None 2026-03-13 01:10:45,921 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:10:45,942 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3594d974c91e64560e4b2 HTTP/1.1" 200 None 2026-03-13 01:10:45,944 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:10:45,990 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b363ac974c91e64560ecfe HTTP/1.1" 200 None 2026-03-13 01:10:45,992 - lava_callback - INFO - Completed processing callback for node 69b363ac974c91e64560ecfe 2026-03-13 01:10:46,241 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:10:46,245 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b363bc974c91e64560ed44/lava-logs.yaml 2026-03-13 01:10:46,269 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:44238 - "GET / HTTP/1.0" 200 OK 2026-03-13 01:10:46,814 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:10:46,816 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b363bc974c91e64560ed44/lava_callback.json.gz 2026-03-13 01:10:46,862 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:10:46,889 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b363bc974c91e64560ed44 HTTP/1.1" 200 None 2026-03-13 01:10:46,891 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:10:46,921 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3594d974c91e64560e4b2 HTTP/1.1" 200 None 2026-03-13 01:10:46,923 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:10:46,968 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b363bc974c91e64560ed44 HTTP/1.1" 200 None 2026-03-13 01:10:46,970 - lava_callback - INFO - Completed processing callback for node 69b363bc974c91e64560ed44 INFO: 172.20.0.1:58466 - "GET / HTTP/1.0" 200 OK 2026-03-13 01:11:05,086 - lava_callback - INFO - Received callback for node 69b0c11a9d19175818252a68 INFO: 172.20.0.1:40600 - "POST /node/69b0c11a9d19175818252a68 HTTP/1.0" 202 Accepted 2026-03-13 01:11:05,096 - lava_callback - INFO - Starting async job submit for node 69b0c11a9d19175818252a68 2026-03-13 01:11:05,098 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:11:05,123 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0c11a9d19175818252a68 HTTP/1.1" 200 None 2026-03-13 01:11:05,124 - lava_callback - DEBUG - Processing node 69b0c11a9d19175818252a68: baseline-arm-clabbe 2026-03-13 01:11:05,124 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0c11a9d19175818252a68 2026-03-13 01:11:05,125 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:11:05,150 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0c11a9d19175818252a68 HTTP/1.1" 200 None 2026-03-13 01:11:05,152 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:11:05,174 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e34efa461f55e59cdb7 HTTP/1.1" 200 None 2026-03-13 01:11:05,177 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:11:05,215 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0c11a9d19175818252a68 HTTP/1.1" 200 None 2026-03-13 01:11:05,216 - lava_callback - ERROR - Error processing callback for node 69b0c11a9d19175818252a68: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:40608 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40622 - "GET / HTTP/1.0" 200 OK 2026-03-13 01:11:12,186 - lava_callback - INFO - Received callback for node 69b363dd974c91e64560edb9 INFO: 172.20.0.1:40792 - "POST /node/69b363dd974c91e64560edb9 HTTP/1.0" 202 Accepted 2026-03-13 01:11:12,196 - lava_callback - INFO - Starting async job submit for node 69b363dd974c91e64560edb9 2026-03-13 01:11:12,229 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:11:12,256 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b363dd974c91e64560edb9 HTTP/1.1" 200 None 2026-03-13 01:11:12,257 - lava_callback - DEBUG - Processing node 69b363dd974c91e64560edb9: baseline-arm64-pengutronix 2026-03-13 01:11:12,577 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:11:12,960 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:11:12,962 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69b363dd974c91e64560edb9/log.txt.gz 2026-03-13 01:11:13,148 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:11:13,769 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:11:13,771 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69b363dd974c91e64560edb9/lava-logs.yaml 2026-03-13 01:11:13,783 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:11:14,354 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:11:14,356 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69b363dd974c91e64560edb9/lava_callback.json.gz 2026-03-13 01:11:14,392 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:11:14,433 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b363dd974c91e64560edb9 HTTP/1.1" 200 None 2026-03-13 01:11:14,436 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:11:14,464 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3594d974c91e64560e4b2 HTTP/1.1" 200 None 2026-03-13 01:11:14,467 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:11:14,521 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b363dd974c91e64560edb9 HTTP/1.1" 200 None 2026-03-13 01:11:14,523 - lava_callback - INFO - Completed processing callback for node 69b363dd974c91e64560edb9 INFO: 172.20.0.1:40802 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-13 01:11:15,252 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:11:15,279 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:40812 - "GET / HTTP/1.0" 200 OK 2026-03-13 01:11:19,406 - lava_callback - INFO - Received callback for node 69b363df974c91e64560edbc INFO: 172.20.0.1:40818 - "POST /node/69b363df974c91e64560edbc HTTP/1.0" 202 Accepted 2026-03-13 01:11:19,415 - lava_callback - INFO - Starting async job submit for node 69b363df974c91e64560edbc 2026-03-13 01:11:19,453 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:11:19,477 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b363df974c91e64560edbc HTTP/1.1" 200 None 2026-03-13 01:11:19,478 - lava_callback - DEBUG - Processing node 69b363df974c91e64560edbc: baseline-arm64-pengutronix 2026-03-13 01:11:19,879 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:11:20,194 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:11:20,196 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69b363df974c91e64560edbc/log.txt.gz 2026-03-13 01:11:20,399 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:11:20,871 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:11:20,874 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69b363df974c91e64560edbc/lava-logs.yaml 2026-03-13 01:11:20,881 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:11:21,277 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:11:21,279 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69b363df974c91e64560edbc/lava_callback.json.gz 2026-03-13 01:11:21,315 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:11:21,339 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b363df974c91e64560edbc HTTP/1.1" 200 None 2026-03-13 01:11:21,342 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:11:21,372 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3594d974c91e64560e4b2 HTTP/1.1" 200 None 2026-03-13 01:11:21,375 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:11:21,423 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b363df974c91e64560edbc HTTP/1.1" 200 None 2026-03-13 01:11:21,425 - lava_callback - INFO - Completed processing callback for node 69b363df974c91e64560edbc INFO: 172.20.0.1:45372 - "GET / HTTP/1.0" 200 OK 2026-03-13 01:11:26,850 - lava_callback - INFO - Received callback for node 69b363ba974c91e64560ed22 INFO: 172.20.0.1:45376 - "POST /node/69b363ba974c91e64560ed22 HTTP/1.0" 202 Accepted 2026-03-13 01:11:26,860 - lava_callback - INFO - Starting async job submit for node 69b363ba974c91e64560ed22 2026-03-13 01:11:26,911 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:11:26,942 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b363ba974c91e64560ed22 HTTP/1.1" 200 None 2026-03-13 01:11:26,944 - lava_callback - DEBUG - Processing node 69b363ba974c91e64560ed22: baseline-nfs-arm64 2026-03-13 01:11:27,545 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:11:27,905 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:11:27,908 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b363ba974c91e64560ed22/log.txt.gz 2026-03-13 01:11:28,278 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:11:28,783 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:11:28,786 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b363ba974c91e64560ed22/lava-logs.yaml 2026-03-13 01:11:28,801 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:11:29,409 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:11:29,411 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b363ba974c91e64560ed22/lava_callback.json.gz 2026-03-13 01:11:29,452 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:11:29,485 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b363ba974c91e64560ed22 HTTP/1.1" 200 None 2026-03-13 01:11:29,488 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:11:29,509 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3594d974c91e64560e4b2 HTTP/1.1" 200 None 2026-03-13 01:11:29,511 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:11:29,564 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b363ba974c91e64560ed22 HTTP/1.1" 200 None 2026-03-13 01:11:29,566 - lava_callback - INFO - Completed processing callback for node 69b363ba974c91e64560ed22 2026-03-13 01:11:30,312 - lava_callback - INFO - Received callback for node 69b363e0974c91e64560edbf INFO: 172.20.0.1:53658 - "POST /node/69b363e0974c91e64560edbf HTTP/1.0" 202 Accepted 2026-03-13 01:11:30,322 - lava_callback - INFO - Starting async job submit for node 69b363e0974c91e64560edbf 2026-03-13 01:11:30,384 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:11:30,412 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b363e0974c91e64560edbf HTTP/1.1" 200 None 2026-03-13 01:11:30,413 - lava_callback - DEBUG - Processing node 69b363e0974c91e64560edbf: baseline-arm64-pengutronix 2026-03-13 01:11:30,784 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:11:31,141 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:11:31,144 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69b363e0974c91e64560edbf/log.txt.gz 2026-03-13 01:11:31,356 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:11:32,025 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:11:32,027 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69b363e0974c91e64560edbf/lava-logs.yaml 2026-03-13 01:11:32,035 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:11:32,414 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:11:32,416 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69b363e0974c91e64560edbf/lava_callback.json.gz 2026-03-13 01:11:32,484 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:11:32,510 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b363e0974c91e64560edbf HTTP/1.1" 200 None 2026-03-13 01:11:32,512 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:11:32,534 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3594d974c91e64560e4b2 HTTP/1.1" 200 None 2026-03-13 01:11:32,536 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:11:32,584 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b363e0974c91e64560edbf HTTP/1.1" 200 None 2026-03-13 01:11:32,586 - lava_callback - INFO - Completed processing callback for node 69b363e0974c91e64560edbf INFO: 172.20.0.1:53664 - "GET / HTTP/1.0" 200 OK 2026-03-13 01:11:39,435 - lava_callback - INFO - Received callback for node 69b363b7974c91e64560ed1b INFO: 172.20.0.1:53670 - "POST /node/69b363b7974c91e64560ed1b HTTP/1.0" 202 Accepted 2026-03-13 01:11:39,447 - lava_callback - INFO - Starting async job submit for node 69b363b7974c91e64560ed1b 2026-03-13 01:11:39,490 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:11:39,533 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b363b7974c91e64560ed1b HTTP/1.1" 200 None 2026-03-13 01:11:39,534 - lava_callback - DEBUG - Processing node 69b363b7974c91e64560ed1b: baseline-nfs-arm64 2026-03-13 01:11:40,105 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:11:40,366 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:11:40,369 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b363b7974c91e64560ed1b/log.txt.gz 2026-03-13 01:11:40,702 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:11:41,300 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:11:41,302 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b363b7974c91e64560ed1b/lava-logs.yaml 2026-03-13 01:11:41,311 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:11:41,680 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:11:41,682 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b363b7974c91e64560ed1b/lava_callback.json.gz 2026-03-13 01:11:41,723 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:11:41,751 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b363b7974c91e64560ed1b HTTP/1.1" 200 None 2026-03-13 01:11:41,753 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:11:41,775 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3594d974c91e64560e4b2 HTTP/1.1" 200 None 2026-03-13 01:11:41,777 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:11:41,833 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b363b7974c91e64560ed1b HTTP/1.1" 200 None 2026-03-13 01:11:41,834 - lava_callback - INFO - Completed processing callback for node 69b363b7974c91e64560ed1b 2026-03-13 01:11:44,884 - lava_callback - INFO - Received callback for node 69b363a3974c91e64560ece9 INFO: 172.20.0.1:41244 - "POST /node/69b363a3974c91e64560ece9 HTTP/1.0" 202 Accepted 2026-03-13 01:11:44,895 - lava_callback - INFO - Starting async job submit for node 69b363a3974c91e64560ece9 2026-03-13 01:11:44,941 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:11:44,974 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b363a3974c91e64560ece9 HTTP/1.1" 200 None 2026-03-13 01:11:44,979 - lava_callback - DEBUG - Processing node 69b363a3974c91e64560ece9: baseline-arm64 2026-03-13 01:11:45,287 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:11:45,327 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 2026-03-13 01:11:45,816 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:11:46,219 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:11:46,221 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69b363a3974c91e64560ece9/log.txt.gz INFO: 172.20.0.1:41246 - "GET / HTTP/1.0" 200 OK 2026-03-13 01:11:46,672 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:11:47,350 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:11:47,352 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69b363a3974c91e64560ece9/lava-logs.yaml 2026-03-13 01:11:47,364 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:11:47,549 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:11:47,551 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69b363a3974c91e64560ece9/lava_callback.json.gz 2026-03-13 01:11:47,597 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:11:47,612 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b363a3974c91e64560ece9 HTTP/1.1" 200 None 2026-03-13 01:11:47,614 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:11:47,629 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3594d974c91e64560e4b2 HTTP/1.1" 200 None 2026-03-13 01:11:47,631 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:11:47,658 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b363a3974c91e64560ece9 HTTP/1.1" 200 None 2026-03-13 01:11:47,720 - lava_callback - INFO - Completed processing callback for node 69b363a3974c91e64560ece9 2026-03-13 01:11:47,862 - lava_callback - INFO - Received callback for node 69b363e1974c91e64560edc2 INFO: 172.20.0.1:41248 - "POST /node/69b363e1974c91e64560edc2 HTTP/1.0" 202 Accepted 2026-03-13 01:11:47,873 - lava_callback - INFO - Starting async job submit for node 69b363e1974c91e64560edc2 2026-03-13 01:11:47,915 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:11:47,940 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b363e1974c91e64560edc2 HTTP/1.1" 200 None 2026-03-13 01:11:47,942 - lava_callback - DEBUG - Processing node 69b363e1974c91e64560edc2: baseline-arm64-foundriesio 2026-03-13 01:11:48,182 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:11:48,298 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:11:48,299 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69b363e1974c91e64560edc2/log.txt.gz 2026-03-13 01:11:48,419 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:11:48,593 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:11:48,595 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69b363e1974c91e64560edc2/lava-logs.yaml 2026-03-13 01:11:48,601 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:11:48,739 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:11:48,741 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69b363e1974c91e64560edc2/lava_callback.json.gz 2026-03-13 01:11:48,785 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:11:48,799 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b363e1974c91e64560edc2 HTTP/1.1" 200 None 2026-03-13 01:11:48,801 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:11:48,821 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3594d974c91e64560e4b2 HTTP/1.1" 200 None 2026-03-13 01:11:48,823 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:11:48,848 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b363e1974c91e64560edc2 HTTP/1.1" 200 None 2026-03-13 01:11:48,894 - lava_callback - INFO - Completed processing callback for node 69b363e1974c91e64560edc2 2026-03-13 01:11:49,460 - lava_callback - INFO - Received callback for node 69b363ae974c91e64560ed04 INFO: 172.20.0.1:41256 - "POST /node/69b363ae974c91e64560ed04 HTTP/1.0" 202 Accepted 2026-03-13 01:11:49,467 - lava_callback - INFO - Starting async job submit for node 69b363ae974c91e64560ed04 2026-03-13 01:11:49,491 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:11:49,505 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b363ae974c91e64560ed04 HTTP/1.1" 200 None 2026-03-13 01:11:49,506 - lava_callback - DEBUG - Processing node 69b363ae974c91e64560ed04: baseline-nfs-arm64 2026-03-13 01:11:49,845 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:11:49,956 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:11:49,958 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b363ae974c91e64560ed04/log.txt.gz 2026-03-13 01:11:50,143 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:11:50,357 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:11:50,359 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b363ae974c91e64560ed04/lava-logs.yaml 2026-03-13 01:11:50,367 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:11:50,496 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:11:50,498 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b363ae974c91e64560ed04/lava_callback.json.gz 2026-03-13 01:11:50,521 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:11:50,538 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b363ae974c91e64560ed04 HTTP/1.1" 200 None 2026-03-13 01:11:50,540 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:11:50,553 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3594d974c91e64560e4b2 HTTP/1.1" 200 None 2026-03-13 01:11:50,555 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:11:50,583 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b363ae974c91e64560ed04 HTTP/1.1" 200 None 2026-03-13 01:11:50,585 - lava_callback - INFO - Completed processing callback for node 69b363ae974c91e64560ed04 2026-03-13 01:11:55,830 - lava_callback - INFO - Received callback for node 69b363bb974c91e64560ed25 INFO: 172.20.0.1:58492 - "POST /node/69b363bb974c91e64560ed25 HTTP/1.0" 202 Accepted 2026-03-13 01:11:55,837 - lava_callback - INFO - Starting async job submit for node 69b363bb974c91e64560ed25 2026-03-13 01:11:55,859 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:11:55,873 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b363bb974c91e64560ed25 HTTP/1.1" 200 None 2026-03-13 01:11:55,874 - lava_callback - DEBUG - Processing node 69b363bb974c91e64560ed25: baseline-nfs-arm64 2026-03-13 01:11:56,200 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:11:56,374 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:11:56,376 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b363bb974c91e64560ed25/log.txt.gz INFO: 172.20.0.1:58496 - "GET / HTTP/1.0" 200 OK 2026-03-13 01:11:56,572 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:11:56,789 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:11:56,791 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b363bb974c91e64560ed25/lava-logs.yaml 2026-03-13 01:11:56,799 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:11:56,935 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:11:56,936 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b363bb974c91e64560ed25/lava_callback.json.gz 2026-03-13 01:11:56,960 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:11:56,974 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b363bb974c91e64560ed25 HTTP/1.1" 200 None 2026-03-13 01:11:56,976 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:11:56,990 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3594d974c91e64560e4b2 HTTP/1.1" 200 None 2026-03-13 01:11:56,992 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:11:57,019 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b363bb974c91e64560ed25 HTTP/1.1" 200 None 2026-03-13 01:11:57,021 - lava_callback - INFO - Completed processing callback for node 69b363bb974c91e64560ed25 INFO: 172.20.0.1:40242 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40246 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35552 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-13 01:12:15,335 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:12:15,354 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 INFO: 172.20.0.1:35558 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55992 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36586 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34812 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50276 - "GET / HTTP/1.0" 200 OK 2026-03-13 01:13:01,216 - lava_callback - INFO - Received callback for node 69b363e2974c91e64560edc5 INFO: 172.20.0.1:44062 - "POST /node/69b363e2974c91e64560edc5 HTTP/1.0" 202 Accepted 2026-03-13 01:13:01,224 - lava_callback - INFO - Starting async job submit for node 69b363e2974c91e64560edc5 2026-03-13 01:13:01,262 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:13:01,277 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b363e2974c91e64560edc5 HTTP/1.1" 200 None 2026-03-13 01:13:01,278 - lava_callback - DEBUG - Processing node 69b363e2974c91e64560edc5: baseline-arm64-foundriesio 2026-03-13 01:13:02,032 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:13:02,224 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:13:02,226 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69b363e2974c91e64560edc5/log.txt.gz 2026-03-13 01:13:02,700 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:13:02,931 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:13:02,933 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69b363e2974c91e64560edc5/lava-logs.yaml 2026-03-13 01:13:02,952 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:13:03,145 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:13:03,147 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69b363e2974c91e64560edc5/lava_callback.json.gz DEBUG: 69b363e2974c91e64560edc5 Transitting job node result: pass -> incomplete 2026-03-13 01:13:03,187 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:13:03,201 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b363e2974c91e64560edc5 HTTP/1.1" 200 None 2026-03-13 01:13:03,202 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:13:03,216 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3594d974c91e64560e4b2 HTTP/1.1" 200 None 2026-03-13 01:13:03,218 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:13:03,244 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b363e2974c91e64560edc5 HTTP/1.1" 200 None 2026-03-13 01:13:03,245 - lava_callback - INFO - Completed processing callback for node 69b363e2974c91e64560edc5 2026-03-13 01:13:04,074 - lava_callback - INFO - Received callback for node 69b363b3974c91e64560ed12 INFO: 172.20.0.1:44076 - "POST /node/69b363b3974c91e64560ed12 HTTP/1.0" 202 Accepted 2026-03-13 01:13:04,081 - lava_callback - INFO - Starting async job submit for node 69b363b3974c91e64560ed12 2026-03-13 01:13:04,104 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:13:04,119 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b363b3974c91e64560ed12 HTTP/1.1" 200 None 2026-03-13 01:13:04,120 - lava_callback - DEBUG - Processing node 69b363b3974c91e64560ed12: baseline-nfs-arm64 2026-03-13 01:13:04,611 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:13:04,773 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:13:04,775 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b363b3974c91e64560ed12/log.txt.gz 2026-03-13 01:13:05,083 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:13:05,265 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:13:05,267 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b363b3974c91e64560ed12/lava-logs.yaml 2026-03-13 01:13:05,278 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:13:05,424 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:13:05,426 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b363b3974c91e64560ed12/lava_callback.json.gz 2026-03-13 01:13:05,450 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:13:05,464 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b363b3974c91e64560ed12 HTTP/1.1" 200 None 2026-03-13 01:13:05,466 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:13:05,480 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3594d974c91e64560e4b2 HTTP/1.1" 200 None 2026-03-13 01:13:05,482 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:13:05,507 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b363b3974c91e64560ed12 HTTP/1.1" 200 None 2026-03-13 01:13:05,509 - lava_callback - INFO - Completed processing callback for node 69b363b3974c91e64560ed12 INFO: 172.20.0.1:44090 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44102 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40430 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-13 01:13:15,357 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:13:15,374 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:40444 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48064 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58234 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35336 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59134 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45610 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45616 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47860 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47868 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47138 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38896 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45028 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38958 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40260 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40276 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42518 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42522 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52784 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37584 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43974 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53904 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33952 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33962 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47166 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47178 - "GET / HTTP/1.0" 200 OK 2026-03-13 01:16:17,519 - lava_callback - INFO - Received callback for node 69b363a4974c91e64560ecec INFO: 172.20.0.1:47192 - "POST /node/69b363a4974c91e64560ecec HTTP/1.0" 202 Accepted 2026-03-13 01:16:17,525 - lava_callback - INFO - Starting async job submit for node 69b363a4974c91e64560ecec 2026-03-13 01:16:17,548 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:16:17,568 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b363a4974c91e64560ecec HTTP/1.1" 200 None 2026-03-13 01:16:17,569 - lava_callback - DEBUG - Processing node 69b363a4974c91e64560ecec: baseline-arm64 2026-03-13 01:16:18,036 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:16:18,203 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:16:18,205 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69b363a4974c91e64560ecec/log.txt.gz 2026-03-13 01:16:18,463 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:16:18,679 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:16:18,682 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69b363a4974c91e64560ecec/lava-logs.yaml 2026-03-13 01:16:18,694 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:16:18,913 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:16:18,916 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69b363a4974c91e64560ecec/lava_callback.json.gz 2026-03-13 01:16:18,939 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:16:18,953 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b363a4974c91e64560ecec HTTP/1.1" 200 None 2026-03-13 01:16:18,955 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:16:18,968 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3594d974c91e64560e4b2 HTTP/1.1" 200 None 2026-03-13 01:16:18,969 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:16:18,994 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b363a4974c91e64560ecec HTTP/1.1" 200 None 2026-03-13 01:16:18,995 - lava_callback - INFO - Completed processing callback for node 69b363a4974c91e64560ecec INFO: 172.20.0.1:38764 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46794 - "GET / HTTP/1.0" 200 OK 2026-03-13 01:16:45,380 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:16:45,397 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:35672 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44236 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53044 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53046 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50284 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50286 - "GET / HTTP/1.0" 200 OK 2026-03-13 01:17:19,734 - lava_callback - INFO - Received callback for node 69b363ad974c91e64560ed01 INFO: 172.20.0.1:52204 - "POST /node/69b363ad974c91e64560ed01 HTTP/1.0" 202 Accepted 2026-03-13 01:17:19,741 - lava_callback - INFO - Starting async job submit for node 69b363ad974c91e64560ed01 2026-03-13 01:17:19,776 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:17:19,791 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b363ad974c91e64560ed01 HTTP/1.1" 200 None 2026-03-13 01:17:19,792 - lava_callback - DEBUG - Processing node 69b363ad974c91e64560ed01: baseline-arm64 2026-03-13 01:17:20,189 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:17:20,341 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:17:20,343 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69b363ad974c91e64560ed01/log.txt.gz 2026-03-13 01:17:20,551 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:17:20,785 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:17:20,788 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69b363ad974c91e64560ed01/lava-logs.yaml 2026-03-13 01:17:20,799 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:17:20,979 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:17:20,981 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69b363ad974c91e64560ed01/lava_callback.json.gz DEBUG: Setup failed for 69b363ad974c91e64560ed01. Transitting job node result: incomplete -> fail 2026-03-13 01:17:21,054 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:17:21,070 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b363ad974c91e64560ed01 HTTP/1.1" 200 None 2026-03-13 01:17:21,072 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:17:21,089 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3594d974c91e64560e4b2 HTTP/1.1" 200 None 2026-03-13 01:17:21,091 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:17:21,115 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b363ad974c91e64560ed01 HTTP/1.1" 200 None 2026-03-13 01:17:21,211 - lava_callback - INFO - Completed processing callback for node 69b363ad974c91e64560ed01 INFO: 172.20.0.1:52212 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44408 - "GET / HTTP/1.0" 200 OK 2026-03-13 01:17:45,399 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:17:45,415 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-13 01:17:45,665 - lava_callback - INFO - Received callback for node 69b363b5974c91e64560ed15 INFO: 172.20.0.1:38198 - "POST /node/69b363b5974c91e64560ed15 HTTP/1.0" 202 Accepted 2026-03-13 01:17:45,672 - lava_callback - INFO - Starting async job submit for node 69b363b5974c91e64560ed15 2026-03-13 01:17:45,695 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:17:45,710 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b363b5974c91e64560ed15 HTTP/1.1" 200 None 2026-03-13 01:17:45,711 - lava_callback - DEBUG - Processing node 69b363b5974c91e64560ed15: baseline-nfs-arm64 2026-03-13 01:17:46,308 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:17:46,483 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:17:46,485 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b363b5974c91e64560ed15/log.txt.gz INFO: 172.20.0.1:38204 - "GET / HTTP/1.0" 200 OK 2026-03-13 01:17:46,836 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:17:47,117 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:17:47,120 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b363b5974c91e64560ed15/lava-logs.yaml 2026-03-13 01:17:47,132 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:17:47,291 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:17:47,293 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b363b5974c91e64560ed15/lava_callback.json.gz 2026-03-13 01:17:47,316 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:17:47,337 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b363b5974c91e64560ed15 HTTP/1.1" 200 None 2026-03-13 01:17:47,338 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:17:47,354 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3594d974c91e64560e4b2 HTTP/1.1" 200 None 2026-03-13 01:17:47,355 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:17:47,396 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b363b5974c91e64560ed15 HTTP/1.1" 200 None 2026-03-13 01:17:47,398 - lava_callback - INFO - Completed processing callback for node 69b363b5974c91e64560ed15 INFO: 172.20.0.1:37040 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36436 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36446 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36284 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-13 01:18:15,417 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:18:15,459 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:36292 - "GET / HTTP/1.0" 200 OK 2026-03-13 01:18:21,754 - lava_callback - INFO - Received callback for node 69b363d9974c91e64560ed8d INFO: 172.20.0.1:56258 - "POST /node/69b363d9974c91e64560ed8d HTTP/1.0" 202 Accepted 2026-03-13 01:18:21,764 - lava_callback - INFO - Starting async job submit for node 69b363d9974c91e64560ed8d 2026-03-13 01:18:21,796 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:18:21,810 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b363d9974c91e64560ed8d HTTP/1.1" 200 None 2026-03-13 01:18:21,811 - lava_callback - DEBUG - Processing node 69b363d9974c91e64560ed8d: baseline-arm64-clabbe 2026-03-13 01:18:22,201 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:18:22,339 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:18:22,341 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-clabbe-69b363d9974c91e64560ed8d/log.txt.gz 2026-03-13 01:18:22,599 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:18:22,840 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:18:22,842 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-clabbe-69b363d9974c91e64560ed8d/lava-logs.yaml 2026-03-13 01:18:22,851 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:18:23,018 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:18:23,020 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-clabbe-69b363d9974c91e64560ed8d/lava_callback.json.gz 2026-03-13 01:18:23,051 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:18:23,067 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b363d9974c91e64560ed8d HTTP/1.1" 200 None 2026-03-13 01:18:23,069 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:18:23,083 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3594d974c91e64560e4b2 HTTP/1.1" 200 None 2026-03-13 01:18:23,085 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:18:23,111 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b363d9974c91e64560ed8d HTTP/1.1" 200 None 2026-03-13 01:18:23,113 - lava_callback - INFO - Completed processing callback for node 69b363d9974c91e64560ed8d INFO: 172.20.0.1:56264 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37572 - "GET / HTTP/1.0" 200 OK 2026-03-13 01:18:45,461 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:18:45,478 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:41744 - "GET / HTTP/1.0" 200 OK 2026-03-13 01:18:55,159 - lava_callback - INFO - Received callback for node 69b363a8974c91e64560ecf5 INFO: 172.20.0.1:47900 - "POST /node/69b363a8974c91e64560ecf5 HTTP/1.0" 202 Accepted 2026-03-13 01:18:55,169 - lava_callback - INFO - Starting async job submit for node 69b363a8974c91e64560ecf5 2026-03-13 01:18:55,189 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:18:55,208 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b363a8974c91e64560ecf5 HTTP/1.1" 200 None 2026-03-13 01:18:55,209 - lava_callback - DEBUG - Processing node 69b363a8974c91e64560ecf5: baseline-arm64 INFO: 172.20.0.1:47904 - "GET / HTTP/1.0" 200 OK 2026-03-13 01:18:56,544 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:18:56,703 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:18:56,705 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69b363a8974c91e64560ecf5/log.txt.gz 2026-03-13 01:18:57,564 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:18:57,811 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:18:57,813 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69b363a8974c91e64560ecf5/lava-logs.yaml 2026-03-13 01:18:57,837 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:18:58,019 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:18:58,021 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69b363a8974c91e64560ecf5/lava_callback.json.gz 2026-03-13 01:18:58,043 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:18:58,058 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b363a8974c91e64560ecf5 HTTP/1.1" 200 None 2026-03-13 01:18:58,060 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:18:58,074 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3594d974c91e64560e4b2 HTTP/1.1" 200 None 2026-03-13 01:18:58,076 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:18:58,101 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b363a8974c91e64560ecf5 HTTP/1.1" 200 None 2026-03-13 01:18:58,102 - lava_callback - INFO - Completed processing callback for node 69b363a8974c91e64560ecf5 INFO: 172.20.0.1:40030 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40036 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36798 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-13 01:19:15,480 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:19:15,495 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:36812 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44238 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60388 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48070 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56890 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51346 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51362 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46282 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46286 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52034 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50620 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52424 - "GET / HTTP/1.0" 200 OK 2026-03-13 01:20:50,056 - lava_callback - INFO - Received callback for node 69b363b8974c91e64560ed1e INFO: 172.20.0.1:41100 - "POST /node/69b363b8974c91e64560ed1e HTTP/1.0" 202 Accepted 2026-03-13 01:20:50,063 - lava_callback - INFO - Starting async job submit for node 69b363b8974c91e64560ed1e 2026-03-13 01:20:50,084 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:20:50,101 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b363b8974c91e64560ed1e HTTP/1.1" 200 None 2026-03-13 01:20:50,102 - lava_callback - DEBUG - Processing node 69b363b8974c91e64560ed1e: baseline-nfs-arm64 2026-03-13 01:20:50,531 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:20:50,710 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:20:50,712 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b363b8974c91e64560ed1e/log.txt.gz 2026-03-13 01:20:50,969 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:20:51,258 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:20:51,260 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b363b8974c91e64560ed1e/lava-logs.yaml 2026-03-13 01:20:51,271 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:20:51,435 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:20:51,437 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b363b8974c91e64560ed1e/lava_callback.json.gz 2026-03-13 01:20:51,459 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:20:51,474 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b363b8974c91e64560ed1e HTTP/1.1" 200 None 2026-03-13 01:20:51,475 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:20:51,489 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3594d974c91e64560e4b2 HTTP/1.1" 200 None 2026-03-13 01:20:51,491 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:20:51,520 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b363b8974c91e64560ed1e HTTP/1.1" 200 None 2026-03-13 01:20:51,522 - lava_callback - INFO - Completed processing callback for node 69b363b8974c91e64560ed1e INFO: 172.20.0.1:41108 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42836 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42852 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41170 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-13 01:21:15,497 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:21:15,515 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:41174 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40300 - "GET / HTTP/1.0" 200 OK 2026-03-13 01:21:27,896 - lava_callback - INFO - Received callback for node 69b0c3909d19175818252b11 INFO: 172.20.0.1:40302 - "POST /node/69b0c3909d19175818252b11 HTTP/1.0" 202 Accepted 2026-03-13 01:21:27,901 - lava_callback - INFO - Starting async job submit for node 69b0c3909d19175818252b11 2026-03-13 01:21:27,902 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:21:27,916 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0c3909d19175818252b11 HTTP/1.1" 200 None 2026-03-13 01:21:27,917 - lava_callback - DEBUG - Processing node 69b0c3909d19175818252b11: baseline-arm64-baylibre 2026-03-13 01:21:27,918 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0c3909d19175818252b11 2026-03-13 01:21:27,918 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:21:27,943 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0c3909d19175818252b11 HTTP/1.1" 200 None 2026-03-13 01:21:27,944 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:21:27,960 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a8cbcf911a9cbe7f77ccbd HTTP/1.1" 200 None 2026-03-13 01:21:27,962 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:21:27,983 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0c3909d19175818252b11 HTTP/1.1" 200 None 2026-03-13 01:21:27,993 - lava_callback - ERROR - Error processing callback for node 69b0c3909d19175818252b11: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:60558 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57110 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46232 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36938 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36944 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56488 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56500 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50048 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34646 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59406 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41168 - "GET / HTTP/1.0" 200 OK 2026-03-13 01:23:05,755 - lava_callback - INFO - Received callback for node 69b0c3e49d19175818252b1b INFO: 172.20.0.1:48006 - "POST /node/69b0c3e49d19175818252b1b HTTP/1.0" 202 Accepted 2026-03-13 01:23:05,761 - lava_callback - INFO - Starting async job submit for node 69b0c3e49d19175818252b1b 2026-03-13 01:23:05,762 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:23:05,787 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0c3e49d19175818252b1b HTTP/1.1" 200 None 2026-03-13 01:23:05,788 - lava_callback - DEBUG - Processing node 69b0c3e49d19175818252b1b: baseline-arm64-clabbe 2026-03-13 01:23:05,789 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0c3e49d19175818252b1b 2026-03-13 01:23:05,789 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:23:05,802 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0c3e49d19175818252b1b HTTP/1.1" 200 None 2026-03-13 01:23:05,804 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:23:05,821 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a8cbcf911a9cbe7f77ccbd HTTP/1.1" 200 None 2026-03-13 01:23:05,822 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:23:05,843 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0c3e49d19175818252b1b HTTP/1.1" 200 None 2026-03-13 01:23:05,844 - lava_callback - ERROR - Error processing callback for node 69b0c3e49d19175818252b1b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:48014 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48024 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36774 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36780 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49944 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35098 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56032 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32872 - "GET / HTTP/1.0" 200 OK 2026-03-13 01:24:05,809 - lava_callback - INFO - Received callback for node 69b0c4289d19175818252b20 INFO: 172.20.0.1:38540 - "POST /node/69b0c4289d19175818252b20 HTTP/1.0" 202 Accepted 2026-03-13 01:24:05,815 - lava_callback - INFO - Starting async job submit for node 69b0c4289d19175818252b20 2026-03-13 01:24:05,816 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:24:05,841 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0c4289d19175818252b20 HTTP/1.1" 200 None 2026-03-13 01:24:05,842 - lava_callback - DEBUG - Processing node 69b0c4289d19175818252b20: baseline-arm-clabbe 2026-03-13 01:24:05,842 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0c4289d19175818252b20 2026-03-13 01:24:05,843 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:24:05,857 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0c4289d19175818252b20 HTTP/1.1" 200 None 2026-03-13 01:24:05,859 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:24:05,881 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a8cbbb911a9cbe7f77ccab HTTP/1.1" 200 None 2026-03-13 01:24:05,882 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:24:05,903 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0c4289d19175818252b20 HTTP/1.1" 200 None 2026-03-13 01:24:05,904 - lava_callback - ERROR - Error processing callback for node 69b0c4289d19175818252b20: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:38548 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38554 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59164 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59178 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35852 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57892 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52610 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45396 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55772 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55786 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54722 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54734 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55004 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46768 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55236 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50732 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48048 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48052 - "GET / HTTP/1.0" 200 OK 2026-03-13 01:26:10,508 - lava_callback - INFO - Received callback for node 69b363a2974c91e64560ece6 INFO: 172.20.0.1:41526 - "POST /node/69b363a2974c91e64560ece6 HTTP/1.0" 202 Accepted 2026-03-13 01:26:10,515 - lava_callback - INFO - Starting async job submit for node 69b363a2974c91e64560ece6 2026-03-13 01:26:10,535 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:26:10,550 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b363a2974c91e64560ece6 HTTP/1.1" 200 None 2026-03-13 01:26:10,550 - lava_callback - DEBUG - Processing node 69b363a2974c91e64560ece6: baseline-arm64 2026-03-13 01:26:10,966 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:26:11,156 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:26:11,158 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69b363a2974c91e64560ece6/log.txt.gz 2026-03-13 01:26:11,363 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:26:11,589 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:26:11,592 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69b363a2974c91e64560ece6/lava-logs.yaml 2026-03-13 01:26:11,600 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:26:11,916 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:26:11,918 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69b363a2974c91e64560ece6/lava_callback.json.gz 2026-03-13 01:26:11,940 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:26:11,954 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b363a2974c91e64560ece6 HTTP/1.1" 200 None 2026-03-13 01:26:11,956 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:26:11,969 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3594d974c91e64560e4b2 HTTP/1.1" 200 None 2026-03-13 01:26:11,971 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:26:11,994 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b363a2974c91e64560ece6 HTTP/1.1" 200 None 2026-03-13 01:26:11,996 - lava_callback - INFO - Completed processing callback for node 69b363a2974c91e64560ece6 INFO: 172.20.0.1:41542 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-13 01:26:15,518 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:26:15,534 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:41554 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35188 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52466 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38340 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42394 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51128 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51140 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37936 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37952 - "GET / HTTP/1.0" 200 OK 2026-03-13 01:27:25,961 - lava_callback - INFO - Received callback for node 69b0c4ed9d19175818252b47 INFO: 172.20.0.1:37138 - "POST /node/69b0c4ed9d19175818252b47 HTTP/1.0" 202 Accepted 2026-03-13 01:27:25,966 - lava_callback - INFO - Starting async job submit for node 69b0c4ed9d19175818252b47 2026-03-13 01:27:25,967 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:27:25,982 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0c4ed9d19175818252b47 HTTP/1.1" 200 None 2026-03-13 01:27:25,982 - lava_callback - DEBUG - Processing node 69b0c4ed9d19175818252b47: baseline-arm-clabbe 2026-03-13 01:27:25,983 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0c4ed9d19175818252b47 2026-03-13 01:27:25,983 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:27:26,001 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0c4ed9d19175818252b47 HTTP/1.1" 200 None 2026-03-13 01:27:26,003 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:27:26,016 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a8cbbb911a9cbe7f77ccab HTTP/1.1" 200 None 2026-03-13 01:27:26,017 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:27:26,037 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0c4ed9d19175818252b47 HTTP/1.1" 200 None 2026-03-13 01:27:26,043 - lava_callback - ERROR - Error processing callback for node 69b0c4ed9d19175818252b47: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:37154 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50480 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37564 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44288 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57684 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57696 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45966 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45980 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57712 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37302 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39546 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56236 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44912 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44914 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45452 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45462 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58498 - "GET / HTTP/1.0" 200 OK 2026-03-13 01:29:28,248 - lava_callback - INFO - Received callback for node 69b0c5669d19175818252b71 INFO: 172.20.0.1:58504 - "POST /node/69b0c5669d19175818252b71 HTTP/1.0" 202 Accepted 2026-03-13 01:29:28,254 - lava_callback - INFO - Starting async job submit for node 69b0c5669d19175818252b71 2026-03-13 01:29:28,255 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:29:28,271 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0c5669d19175818252b71 HTTP/1.1" 200 None 2026-03-13 01:29:28,272 - lava_callback - DEBUG - Processing node 69b0c5669d19175818252b71: baseline-arm64-baylibre 2026-03-13 01:29:28,272 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0c5669d19175818252b71 2026-03-13 01:29:28,273 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:29:28,295 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0c5669d19175818252b71 HTTP/1.1" 200 None 2026-03-13 01:29:28,296 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:29:28,310 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11758ef4424e487b2eb1 HTTP/1.1" 200 None 2026-03-13 01:29:28,313 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:29:28,334 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0c5669d19175818252b71 HTTP/1.1" 200 None 2026-03-13 01:29:28,340 - lava_callback - ERROR - Error processing callback for node 69b0c5669d19175818252b71: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:46468 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52154 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42100 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33390 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33394 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48704 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48718 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43132 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46946 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47980 - "GET / HTTP/1.0" 200 OK 2026-03-13 01:30:48,369 - lava_callback - INFO - Received callback for node 69b0c5b69d19175818252b7f INFO: 172.20.0.1:47996 - "POST /node/69b0c5b69d19175818252b7f HTTP/1.0" 202 Accepted 2026-03-13 01:30:48,377 - lava_callback - INFO - Starting async job submit for node 69b0c5b69d19175818252b7f 2026-03-13 01:30:48,378 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:30:48,401 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0c5b69d19175818252b7f HTTP/1.1" 200 None 2026-03-13 01:30:48,401 - lava_callback - DEBUG - Processing node 69b0c5b69d19175818252b7f: baseline-arm64-baylibre 2026-03-13 01:30:48,402 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0c5b69d19175818252b7f 2026-03-13 01:30:48,402 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:30:48,418 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0c5b69d19175818252b7f HTTP/1.1" 200 None 2026-03-13 01:30:48,420 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:30:48,435 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11758ef4424e487b2eb1 HTTP/1.1" 200 None 2026-03-13 01:30:48,437 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:30:48,457 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0c5b69d19175818252b7f HTTP/1.1" 200 None 2026-03-13 01:30:48,458 - lava_callback - ERROR - Error processing callback for node 69b0c5b69d19175818252b7f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:60494 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38904 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38910 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52782 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52790 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46498 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52194 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58606 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42414 - "GET / HTTP/1.0" 200 OK 2026-03-13 01:32:03,855 - lava_callback - INFO - Received callback for node 69b368f9974c91e64560f129 INFO: 172.20.0.1:54928 - "POST /node/69b368f9974c91e64560f129 HTTP/1.0" 202 Accepted 2026-03-13 01:32:03,861 - lava_callback - INFO - Starting async job submit for node 69b368f9974c91e64560f129 2026-03-13 01:32:03,863 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:32:03,877 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b368f9974c91e64560f129 HTTP/1.1" 200 None 2026-03-13 01:32:03,877 - lava_callback - DEBUG - Processing node 69b368f9974c91e64560f129: baseline-nfs-arm64 2026-03-13 01:32:03,879 - lava_callback - WARNING - Failed to upload log 2026-03-13 01:32:03,880 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:32:04,030 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:32:04,032 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b368f9974c91e64560f129/lava-logs.yaml 2026-03-13 01:32:04,033 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:32:04,150 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:32:04,152 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b368f9974c91e64560f129/lava_callback.json.gz 2026-03-13 01:32:04,155 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:32:04,180 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b368f9974c91e64560f129 HTTP/1.1" 200 None 2026-03-13 01:32:04,182 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:32:04,196 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3594d974c91e64560e4b2 HTTP/1.1" 200 None 2026-03-13 01:32:04,198 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:32:04,218 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b368f9974c91e64560f129 HTTP/1.1" 200 None 2026-03-13 01:32:04,225 - lava_callback - INFO - Completed processing callback for node 69b368f9974c91e64560f129 INFO: 172.20.0.1:54930 - "GET / HTTP/1.0" 200 OK 2026-03-13 01:32:08,444 - lava_callback - INFO - Received callback for node 69b0c6099d19175818252b85 INFO: 172.20.0.1:54940 - "POST /node/69b0c6099d19175818252b85 HTTP/1.0" 202 Accepted 2026-03-13 01:32:08,450 - lava_callback - INFO - Starting async job submit for node 69b0c6099d19175818252b85 2026-03-13 01:32:08,451 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:32:08,473 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0c6099d19175818252b85 HTTP/1.1" 200 None 2026-03-13 01:32:08,474 - lava_callback - DEBUG - Processing node 69b0c6099d19175818252b85: baseline-arm64-baylibre 2026-03-13 01:32:08,474 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0c6099d19175818252b85 2026-03-13 01:32:08,475 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:32:08,488 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0c6099d19175818252b85 HTTP/1.1" 200 None 2026-03-13 01:32:08,489 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:32:08,503 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11758ef4424e487b2eb1 HTTP/1.1" 200 None 2026-03-13 01:32:08,504 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:32:08,531 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0c6099d19175818252b85 HTTP/1.1" 200 None 2026-03-13 01:32:08,534 - lava_callback - ERROR - Error processing callback for node 69b0c6099d19175818252b85: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:54950 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38408 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-13 01:32:15,536 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:32:15,553 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:38422 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59022 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35152 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35062 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60510 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40200 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40208 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48588 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48590 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54596 - "GET / HTTP/1.0" 200 OK 2026-03-13 01:33:28,457 - lava_callback - INFO - Received callback for node 69b0c6599d19175818252b8e INFO: 172.20.0.1:54604 - "POST /node/69b0c6599d19175818252b8e HTTP/1.0" 202 Accepted 2026-03-13 01:33:28,463 - lava_callback - INFO - Starting async job submit for node 69b0c6599d19175818252b8e 2026-03-13 01:33:28,464 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:33:28,477 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0c6599d19175818252b8e HTTP/1.1" 200 None 2026-03-13 01:33:28,478 - lava_callback - DEBUG - Processing node 69b0c6599d19175818252b8e: baseline-arm64-baylibre 2026-03-13 01:33:28,478 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0c6599d19175818252b8e 2026-03-13 01:33:28,479 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:33:28,492 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0c6599d19175818252b8e HTTP/1.1" 200 None 2026-03-13 01:33:28,493 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:33:28,534 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11758ef4424e487b2eb1 HTTP/1.1" 200 None 2026-03-13 01:33:28,535 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:33:28,555 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0c6599d19175818252b8e HTTP/1.1" 200 None 2026-03-13 01:33:28,557 - lava_callback - ERROR - Error processing callback for node 69b0c6599d19175818252b8e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:45388 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48544 - "GET / HTTP/1.0" 200 OK 2026-03-13 01:33:51,824 - lava_callback - INFO - Received callback for node 69b36946974c91e64560f166 INFO: 172.20.0.1:53324 - "POST /node/69b36946974c91e64560f166 HTTP/1.0" 202 Accepted 2026-03-13 01:33:51,831 - lava_callback - INFO - Starting async job submit for node 69b36946974c91e64560f166 2026-03-13 01:33:51,860 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:33:51,876 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b36946974c91e64560f166 HTTP/1.1" 200 None 2026-03-13 01:33:51,877 - lava_callback - DEBUG - Processing node 69b36946974c91e64560f166: baseline-arm64 2026-03-13 01:33:52,011 - lava_callback - WARNING - Failed to upload log 2026-03-13 01:33:52,080 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:33:52,287 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:33:52,289 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69b36946974c91e64560f166/lava-logs.yaml 2026-03-13 01:33:52,294 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:33:52,432 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:33:52,434 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69b36946974c91e64560f166/lava_callback.json.gz 2026-03-13 01:33:52,464 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:33:52,490 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b36946974c91e64560f166 HTTP/1.1" 200 None 2026-03-13 01:33:52,491 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:33:52,505 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3594d974c91e64560e4b2 HTTP/1.1" 200 None 2026-03-13 01:33:52,507 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:33:52,527 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b36946974c91e64560f166 HTTP/1.1" 200 None 2026-03-13 01:33:52,617 - lava_callback - INFO - Completed processing callback for node 69b36946974c91e64560f166 INFO: 172.20.0.1:53330 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60856 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60866 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35618 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-13 01:34:15,555 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:34:15,582 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:35630 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54876 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52636 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36268 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51006 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49670 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49672 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54542 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54556 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34396 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46642 - "GET / HTTP/1.0" 200 OK 2026-03-13 01:35:38,241 - lava_callback - INFO - Received callback for node 69b369a0974c91e64560f19c INFO: 172.20.0.1:46654 - "POST /node/69b369a0974c91e64560f19c HTTP/1.0" 202 Accepted 2026-03-13 01:35:38,248 - lava_callback - INFO - Starting async job submit for node 69b369a0974c91e64560f19c 2026-03-13 01:35:38,277 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:35:38,292 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b369a0974c91e64560f19c HTTP/1.1" 200 None 2026-03-13 01:35:38,293 - lava_callback - DEBUG - Processing node 69b369a0974c91e64560f19c: baseline-arm64 2026-03-13 01:35:38,442 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:35:38,605 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:35:38,607 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69b369a0974c91e64560f19c/log.txt.gz 2026-03-13 01:35:38,680 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:35:38,882 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:35:38,883 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69b369a0974c91e64560f19c/lava-logs.yaml 2026-03-13 01:35:38,887 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:35:39,005 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:35:39,007 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69b369a0974c91e64560f19c/lava_callback.json.gz 2026-03-13 01:35:39,037 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:35:39,054 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b369a0974c91e64560f19c HTTP/1.1" 200 None 2026-03-13 01:35:39,055 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:35:39,069 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3594d974c91e64560e4b2 HTTP/1.1" 200 None 2026-03-13 01:35:39,071 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:35:39,093 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b369a0974c91e64560f19c HTTP/1.1" 200 None 2026-03-13 01:35:39,162 - lava_callback - INFO - Completed processing callback for node 69b369a0974c91e64560f19c 2026-03-13 01:35:45,584 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:35:45,601 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:44610 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49520 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47776 - "GET / HTTP/1.0" 200 OK 2026-03-13 01:36:08,929 - lava_callback - INFO - Received callback for node 69b0c6f69d19175818252ba6 INFO: 172.20.0.1:47790 - "POST /node/69b0c6f69d19175818252ba6 HTTP/1.0" 202 Accepted 2026-03-13 01:36:08,934 - lava_callback - INFO - Starting async job submit for node 69b0c6f69d19175818252ba6 2026-03-13 01:36:08,935 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:36:08,949 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0c6f69d19175818252ba6 HTTP/1.1" 200 None 2026-03-13 01:36:08,950 - lava_callback - DEBUG - Processing node 69b0c6f69d19175818252ba6: baseline-arm64-baylibre 2026-03-13 01:36:08,950 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0c6f69d19175818252ba6 2026-03-13 01:36:08,951 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:36:08,964 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0c6f69d19175818252ba6 HTTP/1.1" 200 None 2026-03-13 01:36:08,965 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.20.0.1:47798 - "GET / HTTP/1.0" 200 OK 2026-03-13 01:36:08,979 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11758ef4424e487b2eb1 HTTP/1.1" 200 None 2026-03-13 01:36:08,980 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:36:09,001 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0c6f69d19175818252ba6 HTTP/1.1" 200 None 2026-03-13 01:36:09,002 - lava_callback - ERROR - Error processing callback for node 69b0c6f69d19175818252ba6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:56788 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56792 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53210 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39736 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46408 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54316 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52482 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52498 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60266 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60276 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49150 - "GET / HTTP/1.0" 200 OK 2026-03-13 01:37:28,722 - lava_callback - INFO - Received callback for node 69b0c7459d19175818252bb0 INFO: 172.20.0.1:49156 - "POST /node/69b0c7459d19175818252bb0 HTTP/1.0" 202 Accepted 2026-03-13 01:37:28,728 - lava_callback - INFO - Starting async job submit for node 69b0c7459d19175818252bb0 2026-03-13 01:37:28,729 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:37:28,763 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0c7459d19175818252bb0 HTTP/1.1" 200 None 2026-03-13 01:37:28,763 - lava_callback - DEBUG - Processing node 69b0c7459d19175818252bb0: baseline-arm64-baylibre 2026-03-13 01:37:28,764 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0c7459d19175818252bb0 2026-03-13 01:37:28,764 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:37:28,782 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0c7459d19175818252bb0 HTTP/1.1" 200 None 2026-03-13 01:37:28,783 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:37:28,796 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11758ef4424e487b2eb1 HTTP/1.1" 200 None 2026-03-13 01:37:28,797 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:37:28,822 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0c7459d19175818252bb0 HTTP/1.1" 200 None 2026-03-13 01:37:28,824 - lava_callback - ERROR - Error processing callback for node 69b0c7459d19175818252bb0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:40244 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49186 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58112 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44998 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45014 - "GET / HTTP/1.0" 200 OK 2026-03-13 01:38:09,152 - lava_callback - INFO - Received callback for node 69b36a53974c91e64560f1f1 INFO: 172.20.0.1:45018 - "POST /node/69b36a53974c91e64560f1f1 HTTP/1.0" 202 Accepted 2026-03-13 01:38:09,158 - lava_callback - INFO - Starting async job submit for node 69b36a53974c91e64560f1f1 2026-03-13 01:38:09,179 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:38:09,194 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b36a53974c91e64560f1f1 HTTP/1.1" 200 None 2026-03-13 01:38:09,195 - lava_callback - DEBUG - Processing node 69b36a53974c91e64560f1f1: baseline-arm64 2026-03-13 01:38:09,558 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:38:09,725 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:38:09,727 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69b36a53974c91e64560f1f1/log.txt.gz 2026-03-13 01:38:09,953 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:38:10,184 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:38:10,186 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69b36a53974c91e64560f1f1/lava-logs.yaml 2026-03-13 01:38:10,194 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:38:10,378 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:38:10,380 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69b36a53974c91e64560f1f1/lava_callback.json.gz 2026-03-13 01:38:10,402 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:38:10,416 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b36a53974c91e64560f1f1 HTTP/1.1" 200 None 2026-03-13 01:38:10,418 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:38:10,446 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3594d974c91e64560e4b2 HTTP/1.1" 200 None 2026-03-13 01:38:10,447 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:38:10,485 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b36a53974c91e64560f1f1 HTTP/1.1" 200 None 2026-03-13 01:38:10,487 - lava_callback - INFO - Completed processing callback for node 69b36a53974c91e64560f1f1 INFO: 172.20.0.1:52176 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-13 01:38:15,604 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:38:15,619 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:52190 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43568 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58984 - "GET / HTTP/1.0" 200 OK 2026-03-13 01:38:46,417 - lava_callback - INFO - Received callback for node 69b0c79b9d19175818252bb9 INFO: 172.20.0.1:43056 - "POST /node/69b0c79b9d19175818252bb9 HTTP/1.0" 202 Accepted 2026-03-13 01:38:46,423 - lava_callback - INFO - Starting async job submit for node 69b0c79b9d19175818252bb9 2026-03-13 01:38:46,424 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:38:46,438 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0c79b9d19175818252bb9 HTTP/1.1" 200 None 2026-03-13 01:38:46,439 - lava_callback - DEBUG - Processing node 69b0c79b9d19175818252bb9: baseline-arm64-clabbe 2026-03-13 01:38:46,440 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0c79b9d19175818252bb9 2026-03-13 01:38:46,440 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:38:46,461 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0c79b9d19175818252bb9 HTTP/1.1" 200 None 2026-03-13 01:38:46,462 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:38:46,476 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11758ef4424e487b2eb1 HTTP/1.1" 200 None 2026-03-13 01:38:46,477 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:38:46,498 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0c79b9d19175818252bb9 HTTP/1.1" 200 None 2026-03-13 01:38:46,505 - lava_callback - ERROR - Error processing callback for node 69b0c79b9d19175818252bb9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:43072 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52564 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50774 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50782 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39428 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39440 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32776 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58456 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36386 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57822 - "GET / HTTP/1.0" 200 OK 2026-03-13 01:40:01,500 - lava_callback - INFO - Received callback for node 69b36abf974c91e64560f232 INFO: 172.20.0.1:33344 - "POST /node/69b36abf974c91e64560f232 HTTP/1.0" 202 Accepted 2026-03-13 01:40:01,506 - lava_callback - INFO - Starting async job submit for node 69b36abf974c91e64560f232 2026-03-13 01:40:01,528 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:40:01,542 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b36abf974c91e64560f232 HTTP/1.1" 200 None 2026-03-13 01:40:01,543 - lava_callback - DEBUG - Processing node 69b36abf974c91e64560f232: baseline-arm64-foundriesio 2026-03-13 01:40:01,766 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:40:01,937 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:40:01,939 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69b36abf974c91e64560f232/log.txt.gz 2026-03-13 01:40:02,063 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:40:02,290 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:40:02,292 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69b36abf974c91e64560f232/lava-logs.yaml 2026-03-13 01:40:02,298 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:40:02,464 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:40:02,465 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69b36abf974c91e64560f232/lava_callback.json.gz 2026-03-13 01:40:02,487 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:40:02,502 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b36abf974c91e64560f232 HTTP/1.1" 200 None 2026-03-13 01:40:02,503 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:40:02,522 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3594d974c91e64560e4b2 HTTP/1.1" 200 None 2026-03-13 01:40:02,524 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:40:02,551 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b36abf974c91e64560f232 HTTP/1.1" 200 None 2026-03-13 01:40:02,566 - lava_callback - INFO - Completed processing callback for node 69b36abf974c91e64560f232 INFO: 172.20.0.1:33356 - "GET / HTTP/1.0" 200 OK 2026-03-13 01:40:06,548 - lava_callback - INFO - Received callback for node 69b0c7ec9d19175818252bbe INFO: 172.20.0.1:33360 - "POST /node/69b0c7ec9d19175818252bbe HTTP/1.0" 202 Accepted 2026-03-13 01:40:06,553 - lava_callback - INFO - Starting async job submit for node 69b0c7ec9d19175818252bbe 2026-03-13 01:40:06,554 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:40:06,568 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0c7ec9d19175818252bbe HTTP/1.1" 200 None 2026-03-13 01:40:06,569 - lava_callback - DEBUG - Processing node 69b0c7ec9d19175818252bbe: baseline-arm64-clabbe 2026-03-13 01:40:06,569 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0c7ec9d19175818252bbe 2026-03-13 01:40:06,570 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:40:06,583 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0c7ec9d19175818252bbe HTTP/1.1" 200 None 2026-03-13 01:40:06,585 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:40:06,601 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11758ef4424e487b2eb1 HTTP/1.1" 200 None 2026-03-13 01:40:06,603 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:40:06,628 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0c7ec9d19175818252bbe HTTP/1.1" 200 None 2026-03-13 01:40:06,630 - lava_callback - ERROR - Error processing callback for node 69b0c7ec9d19175818252bbe: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:33364 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37324 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-13 01:40:15,622 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:40:15,637 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:37328 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53298 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49046 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56222 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46244 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59576 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59588 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46748 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46764 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55954 - "GET / HTTP/1.0" 200 OK 2026-03-13 01:41:26,583 - lava_callback - INFO - Received callback for node 69b0c83e9d19175818252bc7 INFO: 172.20.0.1:55966 - "POST /node/69b0c83e9d19175818252bc7 HTTP/1.0" 202 Accepted 2026-03-13 01:41:26,589 - lava_callback - INFO - Starting async job submit for node 69b0c83e9d19175818252bc7 2026-03-13 01:41:26,590 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:41:26,604 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0c83e9d19175818252bc7 HTTP/1.1" 200 None 2026-03-13 01:41:26,605 - lava_callback - DEBUG - Processing node 69b0c83e9d19175818252bc7: baseline-arm64-clabbe 2026-03-13 01:41:26,605 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0c83e9d19175818252bc7 2026-03-13 01:41:26,606 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:41:26,619 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0c83e9d19175818252bc7 HTTP/1.1" 200 None 2026-03-13 01:41:26,620 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:41:26,641 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11758ef4424e487b2eb1 HTTP/1.1" 200 None 2026-03-13 01:41:26,643 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:41:26,663 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0c83e9d19175818252bc7 HTTP/1.1" 200 None 2026-03-13 01:41:26,664 - lava_callback - ERROR - Error processing callback for node 69b0c83e9d19175818252bc7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:36766 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55424 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49604 - "GET / HTTP/1.0" 200 OK 2026-03-13 01:41:56,727 - lava_callback - INFO - Received callback for node 69b36b17974c91e64560f268 INFO: 172.20.0.1:49614 - "POST /node/69b36b17974c91e64560f268 HTTP/1.0" 202 Accepted 2026-03-13 01:41:56,734 - lava_callback - INFO - Starting async job submit for node 69b36b17974c91e64560f268 2026-03-13 01:41:56,765 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:41:56,778 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b36b17974c91e64560f268 HTTP/1.1" 200 None 2026-03-13 01:41:56,779 - lava_callback - DEBUG - Processing node 69b36b17974c91e64560f268: baseline-arm64-foundriesio 2026-03-13 01:41:57,180 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:41:57,371 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:41:57,373 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69b36b17974c91e64560f268/log.txt.gz 2026-03-13 01:41:57,640 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:41:57,865 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:41:57,868 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69b36b17974c91e64560f268/lava-logs.yaml 2026-03-13 01:41:57,878 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:41:58,050 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:41:58,052 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69b36b17974c91e64560f268/lava_callback.json.gz 2026-03-13 01:41:58,082 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:41:58,095 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b36b17974c91e64560f268 HTTP/1.1" 200 None 2026-03-13 01:41:58,097 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:41:58,111 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3594d974c91e64560e4b2 HTTP/1.1" 200 None 2026-03-13 01:41:58,112 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:41:58,143 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b36b17974c91e64560f268 HTTP/1.1" 200 None 2026-03-13 01:41:58,149 - lava_callback - INFO - Completed processing callback for node 69b36b17974c91e64560f268 INFO: 172.20.0.1:40854 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40860 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57710 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-13 01:42:15,640 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:42:15,656 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:57712 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34796 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42368 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45618 - "GET / HTTP/1.0" 200 OK 2026-03-13 01:42:46,649 - lava_callback - INFO - Received callback for node 69b0c8919d19175818252bd0 INFO: 172.20.0.1:45630 - "POST /node/69b0c8919d19175818252bd0 HTTP/1.0" 202 Accepted 2026-03-13 01:42:46,655 - lava_callback - INFO - Starting async job submit for node 69b0c8919d19175818252bd0 2026-03-13 01:42:46,656 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:42:46,671 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0c8919d19175818252bd0 HTTP/1.1" 200 None 2026-03-13 01:42:46,672 - lava_callback - DEBUG - Processing node 69b0c8919d19175818252bd0: baseline-arm64-clabbe 2026-03-13 01:42:46,672 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0c8919d19175818252bd0 2026-03-13 01:42:46,673 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:42:46,686 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0c8919d19175818252bd0 HTTP/1.1" 200 None 2026-03-13 01:42:46,688 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:42:46,726 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11758ef4424e487b2eb1 HTTP/1.1" 200 None 2026-03-13 01:42:46,728 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:42:46,750 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0c8919d19175818252bd0 HTTP/1.1" 200 None 2026-03-13 01:42:46,751 - lava_callback - ERROR - Error processing callback for node 69b0c8919d19175818252bd0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:56308 - "GET /node/699aec79c1226ad11a5f4195%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:56322 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59784 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59792 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53582 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53594 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37836 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48500 - "GET / HTTP/1.0" 200 OK 2026-03-13 01:43:45,867 - lava_callback - INFO - Received callback for node 69b367e8974c91e64560f0be INFO: 172.20.0.1:50804 - "POST /node/69b367e8974c91e64560f0be HTTP/1.0" 202 Accepted 2026-03-13 01:43:45,874 - lava_callback - INFO - Starting async job submit for node 69b367e8974c91e64560f0be 2026-03-13 01:43:45,928 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:43:45,943 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b367e8974c91e64560f0be HTTP/1.1" 200 None 2026-03-13 01:43:45,944 - lava_callback - DEBUG - Processing node 69b367e8974c91e64560f0be: baseline-arm 2026-03-13 01:43:46,516 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:50808 - "GET / HTTP/1.0" 200 OK 2026-03-13 01:43:46,816 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:43:46,819 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69b367e8974c91e64560f0be/log.txt.gz 2026-03-13 01:43:47,176 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:43:47,451 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:43:47,453 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69b367e8974c91e64560f0be/lava-logs.yaml 2026-03-13 01:43:47,467 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:43:47,672 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:43:47,674 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69b367e8974c91e64560f0be/lava_callback.json.gz DEBUG: Setup failed for 69b367e8974c91e64560f0be. Transitting job node result: incomplete -> fail 2026-03-13 01:43:47,792 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:43:47,807 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b367e8974c91e64560f0be HTTP/1.1" 200 None 2026-03-13 01:43:47,809 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:43:47,823 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35934974c91e64560e49d HTTP/1.1" 200 None 2026-03-13 01:43:47,824 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:43:47,869 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b367e8974c91e64560f0be HTTP/1.1" 200 None 2026-03-13 01:43:47,987 - lava_callback - INFO - Completed processing callback for node 69b367e8974c91e64560f0be 2026-03-13 01:43:50,001 - lava_callback - INFO - Received callback for node 69b3639f974c91e64560ece0 INFO: 172.20.0.1:34020 - "POST /node/69b3639f974c91e64560ece0 HTTP/1.0" 202 Accepted 2026-03-13 01:43:50,007 - lava_callback - INFO - Starting async job submit for node 69b3639f974c91e64560ece0 2026-03-13 01:43:50,024 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:43:50,040 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3639f974c91e64560ece0 HTTP/1.1" 200 None 2026-03-13 01:43:50,041 - lava_callback - DEBUG - Processing node 69b3639f974c91e64560ece0: baseline-arm64 2026-03-13 01:43:50,355 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:43:50,515 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:43:50,517 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69b3639f974c91e64560ece0/log.txt.gz 2026-03-13 01:43:50,708 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:43:50,916 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:43:50,919 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69b3639f974c91e64560ece0/lava-logs.yaml 2026-03-13 01:43:50,928 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:43:51,057 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:43:51,058 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69b3639f974c91e64560ece0/lava_callback.json.gz 2026-03-13 01:43:51,076 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:43:51,093 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3639f974c91e64560ece0 HTTP/1.1" 200 None 2026-03-13 01:43:51,094 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:43:51,107 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3594d974c91e64560e4b2 HTTP/1.1" 200 None 2026-03-13 01:43:51,109 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:43:51,134 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b3639f974c91e64560ece0 HTTP/1.1" 200 None 2026-03-13 01:43:51,144 - lava_callback - INFO - Completed processing callback for node 69b3639f974c91e64560ece0 INFO: 172.20.0.1:34024 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54334 - "GET / HTTP/1.0" 200 OK 2026-03-13 01:44:06,679 - lava_callback - INFO - Received callback for node 69b0c8e29d19175818252bd9 INFO: 172.20.0.1:54346 - "POST /node/69b0c8e29d19175818252bd9 HTTP/1.0" 202 Accepted 2026-03-13 01:44:06,685 - lava_callback - INFO - Starting async job submit for node 69b0c8e29d19175818252bd9 2026-03-13 01:44:06,686 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:44:06,701 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0c8e29d19175818252bd9 HTTP/1.1" 200 None 2026-03-13 01:44:06,701 - lava_callback - DEBUG - Processing node 69b0c8e29d19175818252bd9: baseline-arm64-clabbe 2026-03-13 01:44:06,702 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0c8e29d19175818252bd9 2026-03-13 01:44:06,702 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:44:06,724 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0c8e29d19175818252bd9 HTTP/1.1" 200 None 2026-03-13 01:44:06,725 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:44:06,745 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11758ef4424e487b2eb1 HTTP/1.1" 200 None 2026-03-13 01:44:06,746 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:44:06,770 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0c8e29d19175818252bd9 HTTP/1.1" 200 None 2026-03-13 01:44:06,771 - lava_callback - ERROR - Error processing callback for node 69b0c8e29d19175818252bd9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:54358 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50180 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-13 01:44:15,659 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:44:15,675 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:50188 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51278 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59844 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52482 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34240 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60016 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60026 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52818 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52820 - "GET / HTTP/1.0" 200 OK 2026-03-13 01:45:23,875 - lava_callback - INFO - Received callback for node 69b363b2974c91e64560ed0f INFO: 172.20.0.1:58864 - "POST /node/69b363b2974c91e64560ed0f HTTP/1.0" 202 Accepted 2026-03-13 01:45:23,883 - lava_callback - INFO - Starting async job submit for node 69b363b2974c91e64560ed0f 2026-03-13 01:45:23,926 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:45:23,940 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b363b2974c91e64560ed0f HTTP/1.1" 200 None 2026-03-13 01:45:23,941 - lava_callback - DEBUG - Processing node 69b363b2974c91e64560ed0f: baseline-nfs-arm64 2026-03-13 01:45:25,062 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:45:25,267 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:45:25,269 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b363b2974c91e64560ed0f/log.txt.gz 2026-03-13 01:45:25,901 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:45:26,182 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:45:26,185 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b363b2974c91e64560ed0f/lava-logs.yaml 2026-03-13 01:45:26,206 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:45:26,384 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:45:26,386 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b363b2974c91e64560ed0f/lava_callback.json.gz DEBUG: Setup failed for 69b363b2974c91e64560ed0f. Transitting job node result: incomplete -> fail 2026-03-13 01:45:26,474 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:45:26,489 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b363b2974c91e64560ed0f HTTP/1.1" 200 None 2026-03-13 01:45:26,491 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:45:26,506 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3594d974c91e64560e4b2 HTTP/1.1" 200 None 2026-03-13 01:45:26,507 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:45:26,532 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b363b2974c91e64560ed0f HTTP/1.1" 200 None INFO: 172.20.0.1:58876 - "GET / HTTP/1.0" 200 OK 2026-03-13 01:45:26,640 - lava_callback - INFO - Completed processing callback for node 69b363b2974c91e64560ed0f 2026-03-13 01:45:26,746 - lava_callback - INFO - Received callback for node 69b0c9339d19175818252be4 INFO: 172.20.0.1:58882 - "POST /node/69b0c9339d19175818252be4 HTTP/1.0" 202 Accepted 2026-03-13 01:45:26,753 - lava_callback - INFO - Starting async job submit for node 69b0c9339d19175818252be4 2026-03-13 01:45:26,754 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:45:26,769 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0c9339d19175818252be4 HTTP/1.1" 200 None 2026-03-13 01:45:26,771 - lava_callback - DEBUG - Processing node 69b0c9339d19175818252be4: baseline-arm64-clabbe 2026-03-13 01:45:26,771 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0c9339d19175818252be4 2026-03-13 01:45:26,772 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:45:26,791 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0c9339d19175818252be4 HTTP/1.1" 200 None 2026-03-13 01:45:26,792 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:45:26,805 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11758ef4424e487b2eb1 HTTP/1.1" 200 None 2026-03-13 01:45:26,806 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:45:26,828 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0c9339d19175818252be4 HTTP/1.1" 200 None 2026-03-13 01:45:26,833 - lava_callback - ERROR - Error processing callback for node 69b0c9339d19175818252be4: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:51422 - "GET / HTTP/1.0" 200 OK 2026-03-13 01:45:45,677 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:45:45,694 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:33042 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47708 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42164 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42166 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36948 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36952 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52662 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46198 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42540 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38806 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50364 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50374 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39090 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39106 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38936 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58614 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52140 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57518 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44288 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44304 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46064 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46076 - "GET / HTTP/1.0" 200 OK 2026-03-13 01:48:24,456 - lava_callback - INFO - Received callback for node 69b36cc5974c91e64560f375 INFO: 172.20.0.1:38346 - "POST /node/69b36cc5974c91e64560f375 HTTP/1.0" 202 Accepted 2026-03-13 01:48:24,461 - lava_callback - INFO - Starting async job submit for node 69b36cc5974c91e64560f375 2026-03-13 01:48:24,464 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:48:24,486 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b36cc5974c91e64560f375 HTTP/1.1" 200 None 2026-03-13 01:48:24,486 - lava_callback - DEBUG - Processing node 69b36cc5974c91e64560f375: baseline-nfs-arm64 2026-03-13 01:48:24,488 - lava_callback - WARNING - Failed to upload log 2026-03-13 01:48:24,489 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:48:24,632 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:48:24,634 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b36cc5974c91e64560f375/lava-logs.yaml 2026-03-13 01:48:24,635 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:48:24,786 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:48:24,788 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b36cc5974c91e64560f375/lava_callback.json.gz 2026-03-13 01:48:24,790 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:48:24,808 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b36cc5974c91e64560f375 HTTP/1.1" 200 None 2026-03-13 01:48:24,809 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:48:24,823 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3594d974c91e64560e4b2 HTTP/1.1" 200 None 2026-03-13 01:48:24,824 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:48:24,846 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b36cc5974c91e64560f375 HTTP/1.1" 200 None 2026-03-13 01:48:24,857 - lava_callback - INFO - Completed processing callback for node 69b36cc5974c91e64560f375 INFO: 172.20.0.1:38348 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53750 - "GET / HTTP/1.0" 200 OK 2026-03-13 01:48:45,696 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:48:45,711 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:40804 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59114 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36480 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36488 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56442 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56444 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47280 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45584 - "GET / HTTP/1.0" 200 OK 2026-03-13 01:49:39,428 - lava_callback - INFO - Received callback for node 69b36b5b974c91e64560f282 INFO: 172.20.0.1:45598 - "POST /node/69b36b5b974c91e64560f282 HTTP/1.0" 202 Accepted 2026-03-13 01:49:39,435 - lava_callback - INFO - Starting async job submit for node 69b36b5b974c91e64560f282 2026-03-13 01:49:39,471 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:49:39,494 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b36b5b974c91e64560f282 HTTP/1.1" 200 None 2026-03-13 01:49:39,495 - lava_callback - DEBUG - Processing node 69b36b5b974c91e64560f282: baseline-arm64 2026-03-13 01:49:39,887 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:49:40,049 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:49:40,051 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69b36b5b974c91e64560f282/log.txt.gz 2026-03-13 01:49:40,270 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:49:40,496 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:49:40,499 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69b36b5b974c91e64560f282/lava-logs.yaml 2026-03-13 01:49:40,509 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:49:40,666 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:49:40,668 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69b36b5b974c91e64560f282/lava_callback.json.gz DEBUG: Setup failed for 69b36b5b974c91e64560f282. Transitting job node result: incomplete -> fail 2026-03-13 01:49:40,746 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:49:40,761 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b36b5b974c91e64560f282 HTTP/1.1" 200 None 2026-03-13 01:49:40,763 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:49:40,776 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3594d974c91e64560e4b2 HTTP/1.1" 200 None 2026-03-13 01:49:40,778 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:49:40,802 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b36b5b974c91e64560f282 HTTP/1.1" 200 None 2026-03-13 01:49:40,892 - lava_callback - INFO - Completed processing callback for node 69b36b5b974c91e64560f282 2026-03-13 01:49:45,713 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:49:45,737 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:54646 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47520 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58290 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58306 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41920 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41924 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56762 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42038 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60268 - "GET / HTTP/1.0" 200 OK 2026-03-13 01:50:49,638 - lava_callback - INFO - Received callback for node 69b0ca6b9d19175818252c12 INFO: 172.20.0.1:60280 - "POST /node/69b0ca6b9d19175818252c12 HTTP/1.0" 202 Accepted 2026-03-13 01:50:49,644 - lava_callback - INFO - Starting async job submit for node 69b0ca6b9d19175818252c12 2026-03-13 01:50:49,645 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:50:49,659 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0ca6b9d19175818252c12 HTTP/1.1" 200 None 2026-03-13 01:50:49,660 - lava_callback - DEBUG - Processing node 69b0ca6b9d19175818252c12: baseline-arm64-baylibre 2026-03-13 01:50:49,660 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0ca6b9d19175818252c12 2026-03-13 01:50:49,661 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:50:49,674 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0ca6b9d19175818252c12 HTTP/1.1" 200 None 2026-03-13 01:50:49,675 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:50:49,688 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e44efa461f55e59cdc8 HTTP/1.1" 200 None 2026-03-13 01:50:49,690 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:50:49,711 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0ca6b9d19175818252c12 HTTP/1.1" 200 None 2026-03-13 01:50:49,717 - lava_callback - ERROR - Error processing callback for node 69b0ca6b9d19175818252c12: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:41506 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59598 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59608 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51558 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51560 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53844 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53250 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58764 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49740 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39410 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39414 - "GET / HTTP/1.0" 200 OK 2026-03-13 01:52:09,461 - lava_callback - INFO - Received callback for node 69b0caba9d19175818252c18 INFO: 172.20.0.1:39424 - "POST /node/69b0caba9d19175818252c18 HTTP/1.0" 202 Accepted 2026-03-13 01:52:09,466 - lava_callback - INFO - Starting async job submit for node 69b0caba9d19175818252c18 2026-03-13 01:52:09,467 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:52:09,482 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0caba9d19175818252c18 HTTP/1.1" 200 None 2026-03-13 01:52:09,483 - lava_callback - DEBUG - Processing node 69b0caba9d19175818252c18: baseline-arm64-baylibre 2026-03-13 01:52:09,483 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0caba9d19175818252c18 2026-03-13 01:52:09,483 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:52:09,504 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0caba9d19175818252c18 HTTP/1.1" 200 None 2026-03-13 01:52:09,506 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:52:09,525 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e44efa461f55e59cdc8 HTTP/1.1" 200 None 2026-03-13 01:52:09,526 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:52:09,555 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0caba9d19175818252c18 HTTP/1.1" 200 None 2026-03-13 01:52:09,557 - lava_callback - ERROR - Error processing callback for node 69b0caba9d19175818252c18: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:52516 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52518 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57466 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38390 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40910 - "GET / HTTP/1.0" 200 OK 2026-03-13 01:52:52,657 - lava_callback - INFO - Received callback for node 69b36dc7974c91e64560f3fd INFO: 172.20.0.1:44460 - "POST /node/69b36dc7974c91e64560f3fd HTTP/1.0" 202 Accepted 2026-03-13 01:52:52,664 - lava_callback - INFO - Starting async job submit for node 69b36dc7974c91e64560f3fd 2026-03-13 01:52:52,693 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:52:52,709 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b36dc7974c91e64560f3fd HTTP/1.1" 200 None 2026-03-13 01:52:52,710 - lava_callback - DEBUG - Processing node 69b36dc7974c91e64560f3fd: baseline-arm64 2026-03-13 01:52:52,858 - lava_callback - WARNING - Failed to upload log 2026-03-13 01:52:52,928 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:52:53,156 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:52:53,159 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69b36dc7974c91e64560f3fd/lava-logs.yaml 2026-03-13 01:52:53,163 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:52:53,315 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:52:53,317 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69b36dc7974c91e64560f3fd/lava_callback.json.gz 2026-03-13 01:52:53,347 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:52:53,361 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b36dc7974c91e64560f3fd HTTP/1.1" 200 None 2026-03-13 01:52:53,363 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:52:53,377 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3594d974c91e64560e4b2 HTTP/1.1" 200 None 2026-03-13 01:52:53,379 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:52:53,406 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b36dc7974c91e64560f3fd HTTP/1.1" 200 None 2026-03-13 01:52:53,486 - lava_callback - INFO - Completed processing callback for node 69b36dc7974c91e64560f3fd INFO: 172.20.0.1:44474 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57152 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57160 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38838 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-13 01:53:15,739 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:53:15,754 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:38850 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40708 - "GET / HTTP/1.0" 200 OK 2026-03-13 01:53:29,592 - lava_callback - INFO - Received callback for node 69b0cb0b9d19175818252c21 INFO: 172.20.0.1:40720 - "POST /node/69b0cb0b9d19175818252c21 HTTP/1.0" 202 Accepted 2026-03-13 01:53:29,598 - lava_callback - INFO - Starting async job submit for node 69b0cb0b9d19175818252c21 2026-03-13 01:53:29,599 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:53:29,620 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0cb0b9d19175818252c21 HTTP/1.1" 200 None 2026-03-13 01:53:29,621 - lava_callback - DEBUG - Processing node 69b0cb0b9d19175818252c21: baseline-arm64-baylibre 2026-03-13 01:53:29,621 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0cb0b9d19175818252c21 2026-03-13 01:53:29,622 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:53:29,635 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0cb0b9d19175818252c21 HTTP/1.1" 200 None 2026-03-13 01:53:29,636 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:53:29,649 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e44efa461f55e59cdc8 HTTP/1.1" 200 None 2026-03-13 01:53:29,651 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:53:29,671 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0cb0b9d19175818252c21 HTTP/1.1" 200 None 2026-03-13 01:53:29,677 - lava_callback - ERROR - Error processing callback for node 69b0cb0b9d19175818252c21: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:41920 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37522 - "GET / HTTP/1.0" 200 OK 2026-03-13 01:53:49,219 - lava_callback - INFO - Received callback for node 69b363b0974c91e64560ed07 INFO: 172.20.0.1:37536 - "POST /node/69b363b0974c91e64560ed07 HTTP/1.0" 202 Accepted 2026-03-13 01:53:49,225 - lava_callback - INFO - Starting async job submit for node 69b363b0974c91e64560ed07 2026-03-13 01:53:49,247 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:53:49,287 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b363b0974c91e64560ed07 HTTP/1.1" 200 None 2026-03-13 01:53:49,288 - lava_callback - DEBUG - Processing node 69b363b0974c91e64560ed07: baseline-nfs-arm64 2026-03-13 01:53:49,746 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:53:49,911 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:53:49,913 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b363b0974c91e64560ed07/log.txt.gz 2026-03-13 01:53:50,176 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:53:50,388 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:53:50,390 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b363b0974c91e64560ed07/lava-logs.yaml 2026-03-13 01:53:50,401 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:53:50,677 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:53:50,678 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b363b0974c91e64560ed07/lava_callback.json.gz 2026-03-13 01:53:50,700 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:53:50,715 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b363b0974c91e64560ed07 HTTP/1.1" 200 None 2026-03-13 01:53:50,717 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:53:50,731 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3594d974c91e64560e4b2 HTTP/1.1" 200 None 2026-03-13 01:53:50,733 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:53:50,758 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b363b0974c91e64560ed07 HTTP/1.1" 200 None 2026-03-13 01:53:50,759 - lava_callback - INFO - Completed processing callback for node 69b363b0974c91e64560ed07 INFO: 172.20.0.1:37728 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48614 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48626 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37754 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-13 01:54:15,756 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:54:15,773 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:37762 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37748 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47270 - "GET / HTTP/1.0" 200 OK 2026-03-13 01:54:39,048 - lava_callback - INFO - Received callback for node 69b36e22974c91e64560f432 INFO: 172.20.0.1:47280 - "POST /node/69b36e22974c91e64560f432 HTTP/1.0" 202 Accepted 2026-03-13 01:54:39,055 - lava_callback - INFO - Starting async job submit for node 69b36e22974c91e64560f432 2026-03-13 01:54:39,084 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:54:39,101 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b36e22974c91e64560f432 HTTP/1.1" 200 None 2026-03-13 01:54:39,102 - lava_callback - DEBUG - Processing node 69b36e22974c91e64560f432: baseline-arm64 2026-03-13 01:54:39,243 - lava_callback - WARNING - Failed to upload log 2026-03-13 01:54:39,315 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:54:39,528 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:54:39,530 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69b36e22974c91e64560f432/lava-logs.yaml 2026-03-13 01:54:39,534 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:54:39,671 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:54:39,673 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69b36e22974c91e64560f432/lava_callback.json.gz 2026-03-13 01:54:39,704 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:54:39,718 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b36e22974c91e64560f432 HTTP/1.1" 200 None 2026-03-13 01:54:39,719 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:54:39,733 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3594d974c91e64560e4b2 HTTP/1.1" 200 None 2026-03-13 01:54:39,734 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:54:39,755 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b36e22974c91e64560f432 HTTP/1.1" 200 None 2026-03-13 01:54:39,828 - lava_callback - INFO - Completed processing callback for node 69b36e22974c91e64560f432 2026-03-13 01:54:45,775 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:54:45,792 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:39806 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35032 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45704 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45716 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41364 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41372 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38558 - "GET / HTTP/1.0" 200 OK 2026-03-13 01:55:29,672 - lava_callback - INFO - Received callback for node 69b0cb809d19175818252c35 INFO: 172.20.0.1:38562 - "POST /node/69b0cb809d19175818252c35 HTTP/1.0" 202 Accepted 2026-03-13 01:55:29,678 - lava_callback - INFO - Starting async job submit for node 69b0cb809d19175818252c35 2026-03-13 01:55:29,679 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:55:29,698 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0cb809d19175818252c35 HTTP/1.1" 200 None 2026-03-13 01:55:29,699 - lava_callback - DEBUG - Processing node 69b0cb809d19175818252c35: baseline-arm64-baylibre 2026-03-13 01:55:29,699 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0cb809d19175818252c35 2026-03-13 01:55:29,700 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:55:29,713 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0cb809d19175818252c35 HTTP/1.1" 200 None 2026-03-13 01:55:29,715 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:55:29,727 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e44efa461f55e59cdc8 HTTP/1.1" 200 None 2026-03-13 01:55:29,729 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:55:29,809 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0cb809d19175818252c35 HTTP/1.1" 200 None 2026-03-13 01:55:29,815 - lava_callback - ERROR - Error processing callback for node 69b0cb809d19175818252c35: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:33016 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39168 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37880 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60110 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60122 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57728 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57744 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60124 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50956 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33640 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59916 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40854 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40856 - "GET / HTTP/1.0" 200 OK 2026-03-13 01:57:09,753 - lava_callback - INFO - Received callback for node 69b0cbf29d19175818252c4f INFO: 172.20.0.1:50934 - "POST /node/69b0cbf29d19175818252c4f HTTP/1.0" 202 Accepted 2026-03-13 01:57:09,758 - lava_callback - INFO - Starting async job submit for node 69b0cbf29d19175818252c4f 2026-03-13 01:57:09,759 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:57:09,783 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0cbf29d19175818252c4f HTTP/1.1" 200 None 2026-03-13 01:57:09,785 - lava_callback - DEBUG - Processing node 69b0cbf29d19175818252c4f: baseline-arm64-baylibre 2026-03-13 01:57:09,785 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0cbf29d19175818252c4f 2026-03-13 01:57:09,785 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:57:09,802 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0cbf29d19175818252c4f HTTP/1.1" 200 None 2026-03-13 01:57:09,804 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:57:09,817 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e44efa461f55e59cdc8 HTTP/1.1" 200 None 2026-03-13 01:57:09,818 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:57:09,839 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0cbf29d19175818252c4f HTTP/1.1" 200 None 2026-03-13 01:57:09,840 - lava_callback - ERROR - Error processing callback for node 69b0cbf29d19175818252c4f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:50950 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50956 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49080 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58452 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58458 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47518 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41180 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41182 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35660 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35668 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48782 - "GET / HTTP/1.0" 200 OK 2026-03-13 01:58:29,876 - lava_callback - INFO - Received callback for node 69b0cc409d19175818252c56 INFO: 172.20.0.1:46368 - "POST /node/69b0cc409d19175818252c56 HTTP/1.0" 202 Accepted 2026-03-13 01:58:29,881 - lava_callback - INFO - Starting async job submit for node 69b0cc409d19175818252c56 2026-03-13 01:58:29,882 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:58:29,898 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0cc409d19175818252c56 HTTP/1.1" 200 None 2026-03-13 01:58:29,898 - lava_callback - DEBUG - Processing node 69b0cc409d19175818252c56: baseline-arm64-baylibre 2026-03-13 01:58:29,899 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0cc409d19175818252c56 2026-03-13 01:58:29,899 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:58:29,913 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0cc409d19175818252c56 HTTP/1.1" 200 None 2026-03-13 01:58:29,915 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:58:29,933 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e44efa461f55e59cdc8 HTTP/1.1" 200 None 2026-03-13 01:58:29,935 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:58:29,961 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0cc409d19175818252c56 HTTP/1.1" 200 None 2026-03-13 01:58:29,963 - lava_callback - ERROR - Error processing callback for node 69b0cc409d19175818252c56: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:46380 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58222 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41842 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36694 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36702 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41194 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41206 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60026 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54750 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58606 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41142 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38560 - "GET / HTTP/1.0" 200 OK 2026-03-13 02:00:07,693 - lava_callback - INFO - Received callback for node 69b0cc959d19175818252c60 INFO: 172.20.0.1:38562 - "POST /node/69b0cc959d19175818252c60 HTTP/1.0" 202 Accepted 2026-03-13 02:00:07,699 - lava_callback - INFO - Starting async job submit for node 69b0cc959d19175818252c60 2026-03-13 02:00:07,700 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 02:00:07,719 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0cc959d19175818252c60 HTTP/1.1" 200 None 2026-03-13 02:00:07,719 - lava_callback - DEBUG - Processing node 69b0cc959d19175818252c60: baseline-arm64-clabbe 2026-03-13 02:00:07,720 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0cc959d19175818252c60 2026-03-13 02:00:07,720 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 02:00:07,738 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0cc959d19175818252c60 HTTP/1.1" 200 None 2026-03-13 02:00:07,739 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 02:00:07,754 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e44efa461f55e59cdc8 HTTP/1.1" 200 None 2026-03-13 02:00:07,757 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 02:00:07,778 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0cc959d19175818252c60 HTTP/1.1" 200 None 2026-03-13 02:00:07,783 - lava_callback - ERROR - Error processing callback for node 69b0cc959d19175818252c60: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:38572 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53806 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53812 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49228 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50746 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44110 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39514 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45666 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45674 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51256 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51270 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50482 - "GET / HTTP/1.0" 200 OK 2026-03-13 02:01:27,444 - lava_callback - INFO - Received callback for node 69b0cce69d19175818252c6a INFO: 172.20.0.1:50488 - "POST /node/69b0cce69d19175818252c6a HTTP/1.0" 202 Accepted 2026-03-13 02:01:27,450 - lava_callback - INFO - Starting async job submit for node 69b0cce69d19175818252c6a 2026-03-13 02:01:27,451 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 02:01:27,465 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0cce69d19175818252c6a HTTP/1.1" 200 None 2026-03-13 02:01:27,465 - lava_callback - DEBUG - Processing node 69b0cce69d19175818252c6a: baseline-arm64-clabbe 2026-03-13 02:01:27,466 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0cce69d19175818252c6a 2026-03-13 02:01:27,466 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 02:01:27,483 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0cce69d19175818252c6a HTTP/1.1" 200 None 2026-03-13 02:01:27,484 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 02:01:27,499 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e44efa461f55e59cdc8 HTTP/1.1" 200 None 2026-03-13 02:01:27,500 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 02:01:27,544 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0cce69d19175818252c6a HTTP/1.1" 200 None 2026-03-13 02:01:27,545 - lava_callback - ERROR - Error processing callback for node 69b0cce69d19175818252c6a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:56302 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37520 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58258 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38710 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38726 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46506 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46508 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51614 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55508 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54952 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49660 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42294 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42304 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50288 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50302 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59676 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42240 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51096 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48322 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56124 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56130 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41336 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41350 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55486 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54294 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38356 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46124 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57812 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57816 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34860 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34864 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40210 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37014 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58440 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38860 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52640 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52652 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57930 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57946 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59656 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56490 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38650 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54864 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52868 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52870 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55120 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55132 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57532 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42182 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58094 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56638 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48128 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48130 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58798 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58806 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46944 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54770 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47062 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35216 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56768 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56778 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37984 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37992 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57894 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45612 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46436 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36494 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37776 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37792 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52514 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52522 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48916 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32850 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33542 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48692 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47806 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47816 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54370 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54378 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60554 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57982 - "GET / HTTP/1.0" 200 OK 2026-03-13 02:11:45,389 - lava_callback - INFO - Received callback for node 69b37242974c91e64560f5e2 INFO: 172.20.0.1:54316 - "POST /node/69b37242974c91e64560f5e2 HTTP/1.0" 202 Accepted 2026-03-13 02:11:45,395 - lava_callback - INFO - Starting async job submit for node 69b37242974c91e64560f5e2 2026-03-13 02:11:45,397 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 02:11:45,411 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b37242974c91e64560f5e2 HTTP/1.1" 200 None 2026-03-13 02:11:45,412 - lava_callback - DEBUG - Processing node 69b37242974c91e64560f5e2: baseline-nfs-arm64 2026-03-13 02:11:45,413 - lava_callback - WARNING - Failed to upload log 2026-03-13 02:11:45,414 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 02:11:45,581 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 02:11:45,583 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b37242974c91e64560f5e2/lava-logs.yaml 2026-03-13 02:11:45,585 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 02:11:45,732 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 02:11:45,735 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b37242974c91e64560f5e2/lava_callback.json.gz 2026-03-13 02:11:45,736 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 02:11:45,751 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b37242974c91e64560f5e2 HTTP/1.1" 200 None 2026-03-13 02:11:45,753 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 02:11:45,766 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3594d974c91e64560e4b2 HTTP/1.1" 200 None 2026-03-13 02:11:45,768 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 02:11:45,789 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b37242974c91e64560f5e2 HTTP/1.1" 200 None 2026-03-13 02:11:45,793 - lava_callback - INFO - Completed processing callback for node 69b37242974c91e64560f5e2 2026-03-13 02:11:45,797 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 02:11:45,874 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:54328 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35836 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35722 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35732 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57314 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57322 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58902 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47438 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33538 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57892 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47206 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47208 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46392 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46408 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34866 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38160 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38000 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51336 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52894 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52910 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55490 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55500 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59476 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40354 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49656 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52814 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38028 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38038 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57464 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57470 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41156 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38192 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37014 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52466 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59240 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59256 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38086 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38100 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55956 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41348 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49062 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40164 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36174 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36180 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39128 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39132 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53358 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57938 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49620 - "GET / HTTP/1.0" 200 OK 2026-03-13 02:17:52,262 - lava_callback - INFO - Received callback for node 69b3729e974c91e64560f5f9 INFO: 172.20.0.1:38982 - "POST /node/69b3729e974c91e64560f5f9 HTTP/1.0" 202 Accepted 2026-03-13 02:17:52,269 - lava_callback - INFO - Starting async job submit for node 69b3729e974c91e64560f5f9 2026-03-13 02:17:52,304 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 02:17:52,318 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3729e974c91e64560f5f9 HTTP/1.1" 200 None 2026-03-13 02:17:52,319 - lava_callback - DEBUG - Processing node 69b3729e974c91e64560f5f9: baseline-arm64 2026-03-13 02:17:52,705 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 02:17:52,856 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 02:17:52,857 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69b3729e974c91e64560f5f9/log.txt.gz 2026-03-13 02:17:53,075 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 02:17:53,312 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 02:17:53,314 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69b3729e974c91e64560f5f9/lava-logs.yaml 2026-03-13 02:17:53,323 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 02:17:53,535 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 02:17:53,537 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69b3729e974c91e64560f5f9/lava_callback.json.gz DEBUG: Setup failed for 69b3729e974c91e64560f5f9. Transitting job node result: incomplete -> fail 2026-03-13 02:17:53,611 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 02:17:53,633 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3729e974c91e64560f5f9 HTTP/1.1" 200 None 2026-03-13 02:17:53,635 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 02:17:53,648 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3594d974c91e64560e4b2 HTTP/1.1" 200 None 2026-03-13 02:17:53,650 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 02:17:53,680 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b3729e974c91e64560f5f9 HTTP/1.1" 200 None 2026-03-13 02:17:53,751 - lava_callback - INFO - Completed processing callback for node 69b3729e974c91e64560f5f9 INFO: 172.20.0.1:38994 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43012 - "GET / HTTP/1.0" 200 OK 2026-03-13 02:18:09,157 - lava_callback - INFO - Received callback for node 69b373ad974c91e64560f642 INFO: 172.20.0.1:43028 - "POST /node/69b373ad974c91e64560f642 HTTP/1.0" 202 Accepted 2026-03-13 02:18:09,163 - lava_callback - INFO - Starting async job submit for node 69b373ad974c91e64560f642 2026-03-13 02:18:09,192 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 02:18:09,206 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b373ad974c91e64560f642 HTTP/1.1" 200 None 2026-03-13 02:18:09,207 - lava_callback - DEBUG - Processing node 69b373ad974c91e64560f642: baseline-arm64 2026-03-13 02:18:09,341 - lava_callback - WARNING - Failed to upload log 2026-03-13 02:18:09,415 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:43030 - "GET / HTTP/1.0" 200 OK 2026-03-13 02:18:09,614 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 02:18:09,616 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69b373ad974c91e64560f642/lava-logs.yaml 2026-03-13 02:18:09,619 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 02:18:09,750 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 02:18:09,752 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69b373ad974c91e64560f642/lava_callback.json.gz 2026-03-13 02:18:09,782 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 02:18:09,797 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b373ad974c91e64560f642 HTTP/1.1" 200 None 2026-03-13 02:18:09,798 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 02:18:09,812 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3594d974c91e64560e4b2 HTTP/1.1" 200 None 2026-03-13 02:18:09,813 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 02:18:09,834 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b373ad974c91e64560f642 HTTP/1.1" 200 None 2026-03-13 02:18:09,942 - lava_callback - INFO - Completed processing callback for node 69b373ad974c91e64560f642 INFO: 172.20.0.1:46442 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-13 02:18:15,878 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 02:18:15,906 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:46448 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58676 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59632 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43462 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60612 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39830 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39834 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41280 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41294 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38756 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53178 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48854 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55372 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33450 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33460 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43482 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43496 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54370 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49536 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34990 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58552 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35608 - "GET / HTTP/1.0" 200 OK 2026-03-13 02:21:08,567 - lava_callback - INFO - Received callback for node 69b37462974c91e64560f678 INFO: 172.20.0.1:35610 - "POST /node/69b37462974c91e64560f678 HTTP/1.0" 202 Accepted 2026-03-13 02:21:08,574 - lava_callback - INFO - Starting async job submit for node 69b37462974c91e64560f678 2026-03-13 02:21:08,603 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 02:21:08,642 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b37462974c91e64560f678 HTTP/1.1" 200 None 2026-03-13 02:21:08,643 - lava_callback - DEBUG - Processing node 69b37462974c91e64560f678: baseline-arm64 2026-03-13 02:21:08,789 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 02:21:08,938 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 02:21:08,940 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69b37462974c91e64560f678/log.txt.gz 2026-03-13 02:21:09,011 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 02:21:09,219 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 02:21:09,221 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69b37462974c91e64560f678/lava-logs.yaml 2026-03-13 02:21:09,224 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 02:21:09,373 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 02:21:09,374 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69b37462974c91e64560f678/lava_callback.json.gz 2026-03-13 02:21:09,405 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 02:21:09,419 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b37462974c91e64560f678 HTTP/1.1" 200 None 2026-03-13 02:21:09,420 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 02:21:09,435 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3594d974c91e64560e4b2 HTTP/1.1" 200 None 2026-03-13 02:21:09,436 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 02:21:09,458 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b37462974c91e64560f678 HTTP/1.1" 200 None 2026-03-13 02:21:09,510 - lava_callback - INFO - Completed processing callback for node 69b37462974c91e64560f678 INFO: 172.20.0.1:35616 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43196 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-13 02:21:15,908 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 02:21:15,925 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:43208 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45214 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37784 - "GET / HTTP/1.0" 200 OK 2026-03-13 02:21:45,169 - lava_callback - INFO - Received callback for node 69b370a1974c91e64560f56d INFO: 172.20.0.1:39102 - "POST /node/69b370a1974c91e64560f56d HTTP/1.0" 202 Accepted 2026-03-13 02:21:45,175 - lava_callback - INFO - Starting async job submit for node 69b370a1974c91e64560f56d 2026-03-13 02:21:45,227 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 02:21:45,242 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b370a1974c91e64560f56d HTTP/1.1" 200 None 2026-03-13 02:21:45,243 - lava_callback - DEBUG - Processing node 69b370a1974c91e64560f56d: baseline-arm 2026-03-13 02:21:45,830 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 02:21:45,946 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 02:21:45,948 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69b370a1974c91e64560f56d/log.txt.gz 2026-03-13 02:21:46,281 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:39114 - "GET / HTTP/1.0" 200 OK 2026-03-13 02:21:46,565 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 02:21:46,567 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69b370a1974c91e64560f56d/lava-logs.yaml 2026-03-13 02:21:46,582 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 02:21:46,752 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 02:21:46,754 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69b370a1974c91e64560f56d/lava_callback.json.gz DEBUG: Setup failed for 69b370a1974c91e64560f56d. Transitting job node result: incomplete -> fail 2026-03-13 02:21:46,864 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 02:21:46,878 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b370a1974c91e64560f56d HTTP/1.1" 200 None 2026-03-13 02:21:46,879 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 02:21:46,893 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35934974c91e64560e49d HTTP/1.1" 200 None 2026-03-13 02:21:46,894 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 02:21:46,929 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b370a1974c91e64560f56d HTTP/1.1" 200 None 2026-03-13 02:21:47,034 - lava_callback - INFO - Completed processing callback for node 69b370a1974c91e64560f56d INFO: 172.20.0.1:51484 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35014 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35020 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42764 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-13 02:22:15,927 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 02:22:15,943 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:42770 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51238 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45498 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53128 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49046 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37630 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37634 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53990 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54000 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48386 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51210 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57692 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34986 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59720 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59732 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46726 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46728 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56802 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40578 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54612 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39826 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51048 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51058 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56876 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56884 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44310 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44398 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58014 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55878 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52654 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49200 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49214 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49216 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53738 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36234 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43834 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47844 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42834 - "GET /node/69944f545ae459b9df3dcee1%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:42836 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56948 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56960 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56968 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51812 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55652 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54720 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33170 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48638 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59798 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59806 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59808 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58710 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43368 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53424 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39184 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43742 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45384 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45388 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45394 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38532 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55980 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52824 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34860 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56260 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47746 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47754 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47764 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53734 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50190 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49864 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46880 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59836 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46928 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46942 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46956 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36754 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57252 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51184 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45214 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38190 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42728 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42734 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42748 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38378 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50498 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38232 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59508 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48414 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56058 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56070 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56078 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42088 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54640 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39390 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41500 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51532 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38188 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38204 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38218 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43864 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39504 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56950 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43974 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57914 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52044 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52058 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52070 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59764 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43754 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39602 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49022 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60508 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47862 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47872 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47876 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33470 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40908 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55854 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48898 - "GET / HTTP/1.0" 200 OK 2026-03-13 02:37:00,590 - lava_callback - INFO - Received callback for node 69b37684974c91e64560f702 INFO: 172.20.0.1:34370 - "POST /node/69b37684974c91e64560f702 HTTP/1.0" 202 Accepted 2026-03-13 02:37:00,597 - lava_callback - INFO - Starting async job submit for node 69b37684974c91e64560f702 2026-03-13 02:37:00,632 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 02:37:00,647 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b37684974c91e64560f702 HTTP/1.1" 200 None 2026-03-13 02:37:00,648 - lava_callback - DEBUG - Processing node 69b37684974c91e64560f702: baseline-arm64 2026-03-13 02:37:01,027 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 02:37:01,195 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 02:37:01,197 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69b37684974c91e64560f702/log.txt.gz 2026-03-13 02:37:01,404 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 02:37:01,638 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 02:37:01,641 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69b37684974c91e64560f702/lava-logs.yaml 2026-03-13 02:37:01,650 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 02:37:01,845 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 02:37:01,847 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69b37684974c91e64560f702/lava_callback.json.gz DEBUG: Setup failed for 69b37684974c91e64560f702. Transitting job node result: incomplete -> fail 2026-03-13 02:37:01,920 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 02:37:01,954 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b37684974c91e64560f702 HTTP/1.1" 200 None 2026-03-13 02:37:01,955 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 02:37:01,979 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3594d974c91e64560e4b2 HTTP/1.1" 200 None 2026-03-13 02:37:01,981 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 02:37:02,020 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b37684974c91e64560f702 HTTP/1.1" 200 None 2026-03-13 02:37:02,107 - lava_callback - INFO - Completed processing callback for node 69b37684974c91e64560f702 INFO: 172.20.0.1:34376 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47382 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47392 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-13 02:37:15,948 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 02:37:15,964 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:47398 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35056 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36550 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54292 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55452 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49156 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59044 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59056 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59068 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54390 - "GET / HTTP/1.0" 200 OK 2026-03-13 02:38:29,952 - lava_callback - INFO - Received callback for node 69b36301974c91e64560ec82 INFO: 172.20.0.1:60370 - "POST /node/69b36301974c91e64560ec82 HTTP/1.0" 202 Accepted 2026-03-13 02:38:29,959 - lava_callback - INFO - Starting async job submit for node 69b36301974c91e64560ec82 2026-03-13 02:38:30,017 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 02:38:30,046 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b36301974c91e64560ec82 HTTP/1.1" 200 None 2026-03-13 02:38:30,047 - lava_callback - DEBUG - Processing node 69b36301974c91e64560ec82: ltp-smoke 2026-03-13 02:38:30,516 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 02:38:30,678 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 02:38:30,679 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoke-69b36301974c91e64560ec82/log.txt.gz 2026-03-13 02:38:30,965 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 02:38:31,197 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 02:38:31,200 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-smoke-69b36301974c91e64560ec82/lava-logs.yaml 2026-03-13 02:38:31,211 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 02:38:31,404 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 02:38:31,406 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoke-69b36301974c91e64560ec82/lava_callback.json.gz 2026-03-13 02:38:31,526 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 02:38:31,541 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b36301974c91e64560ec82 HTTP/1.1" 200 None 2026-03-13 02:38:31,542 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 02:38:31,565 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3593e974c91e64560e4a3 HTTP/1.1" 200 None 2026-03-13 02:38:31,567 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 02:38:31,596 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b36301974c91e64560ec82 HTTP/1.1" 200 None 2026-03-13 02:38:31,705 - lava_callback - INFO - Completed processing callback for node 69b36301974c91e64560ec82 INFO: 172.20.0.1:60384 - "GET / HTTP/1.0" 200 OK 2026-03-13 02:38:45,966 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 02:38:45,983 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:34656 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35886 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42150 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58144 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58146 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58152 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44988 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42632 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56442 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50882 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58050 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36106 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36110 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36114 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38876 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43722 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60934 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52662 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59918 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35810 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35814 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35826 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37476 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42672 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54814 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57706 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54722 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60726 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60740 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60750 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47092 - "GET / HTTP/1.0" 200 OK 2026-03-13 02:42:30,598 - lava_callback - INFO - Received callback for node 69b363cd974c91e64560ed64 INFO: 172.20.0.1:56202 - "POST /node/69b363cd974c91e64560ed64 HTTP/1.0" 202 Accepted 2026-03-13 02:42:30,604 - lava_callback - INFO - Starting async job submit for node 69b363cd974c91e64560ed64 2026-03-13 02:42:30,627 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 02:42:30,649 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b363cd974c91e64560ed64 HTTP/1.1" 200 None 2026-03-13 02:42:30,649 - lava_callback - DEBUG - Processing node 69b363cd974c91e64560ed64: baseline-arm64-broonie 2026-03-13 02:42:30,906 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 02:42:31,058 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 02:42:31,060 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69b363cd974c91e64560ed64/log.txt.gz 2026-03-13 02:42:31,216 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 02:42:31,421 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 02:42:31,423 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69b363cd974c91e64560ed64/lava-logs.yaml 2026-03-13 02:42:31,430 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 02:42:31,606 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 02:42:31,608 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69b363cd974c91e64560ed64/lava_callback.json.gz 2026-03-13 02:42:31,631 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 02:42:31,645 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b363cd974c91e64560ed64 HTTP/1.1" 200 None 2026-03-13 02:42:31,647 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 02:42:31,660 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3594d974c91e64560e4b2 HTTP/1.1" 200 None 2026-03-13 02:42:31,662 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 02:42:31,692 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b363cd974c91e64560ed64 HTTP/1.1" 200 None 2026-03-13 02:42:31,693 - lava_callback - INFO - Completed processing callback for node 69b363cd974c91e64560ed64 INFO: 172.20.0.1:56214 - "GET / HTTP/1.0" 200 OK 2026-03-13 02:42:45,986 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 02:42:46,002 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:57414 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35458 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44666 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43856 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43858 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43870 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40684 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46786 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44032 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58562 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39756 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58928 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58940 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58948 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40014 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50472 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34168 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44380 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51198 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52084 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52098 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52106 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41832 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55072 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49226 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51594 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58106 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35766 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35780 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35792 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47762 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58978 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34076 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41436 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54440 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49386 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49390 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49394 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47930 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36942 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42566 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54498 - "GET / HTTP/1.0" 200 OK 2026-03-13 02:48:02,571 - lava_callback - INFO - Received callback for node 69b376e1974c91e64560f716 INFO: 172.20.0.1:57582 - "POST /node/69b376e1974c91e64560f716 HTTP/1.0" 202 Accepted 2026-03-13 02:48:02,578 - lava_callback - INFO - Starting async job submit for node 69b376e1974c91e64560f716 2026-03-13 02:48:02,630 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 02:48:02,645 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b376e1974c91e64560f716 HTTP/1.1" 200 None 2026-03-13 02:48:02,646 - lava_callback - DEBUG - Processing node 69b376e1974c91e64560f716: baseline-arm 2026-03-13 02:48:03,235 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 02:48:03,392 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 02:48:03,394 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69b376e1974c91e64560f716/log.txt.gz 2026-03-13 02:48:03,742 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 02:48:03,988 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 02:48:03,990 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69b376e1974c91e64560f716/lava-logs.yaml 2026-03-13 02:48:04,005 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 02:48:04,194 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 02:48:04,195 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69b376e1974c91e64560f716/lava_callback.json.gz DEBUG: Setup failed for 69b376e1974c91e64560f716. Transitting job node result: incomplete -> fail 2026-03-13 02:48:04,302 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 02:48:04,321 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b376e1974c91e64560f716 HTTP/1.1" 200 None 2026-03-13 02:48:04,322 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 02:48:04,337 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35934974c91e64560e49d HTTP/1.1" 200 None 2026-03-13 02:48:04,338 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 02:48:04,363 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b376e1974c91e64560f716 HTTP/1.1" 200 None 2026-03-13 02:48:04,497 - lava_callback - INFO - Completed processing callback for node 69b376e1974c91e64560f716 INFO: 172.20.0.1:57590 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40342 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40352 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-13 02:48:16,005 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 02:48:16,021 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:40364 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47130 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49150 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54738 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52300 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60446 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39946 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39958 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39972 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33568 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54152 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60376 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42276 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55256 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48590 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48600 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48602 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41530 - "GET / HTTP/1.0" 200 OK 2026-03-13 02:50:32,732 - lava_callback - INFO - Received callback for node 69b0d8639d19175818252cd6 INFO: 172.20.0.1:34540 - "POST /node/69b0d8639d19175818252cd6 HTTP/1.0" 202 Accepted 2026-03-13 02:50:32,738 - lava_callback - INFO - Starting async job submit for node 69b0d8639d19175818252cd6 2026-03-13 02:50:32,738 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 02:50:32,761 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0d8639d19175818252cd6 HTTP/1.1" 200 None 2026-03-13 02:50:32,762 - lava_callback - DEBUG - Processing node 69b0d8639d19175818252cd6: baseline-arm64-baylibre 2026-03-13 02:50:32,762 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0d8639d19175818252cd6 2026-03-13 02:50:32,763 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 02:50:32,776 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0d8639d19175818252cd6 HTTP/1.1" 200 None 2026-03-13 02:50:32,778 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 02:50:32,792 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aafea227cb11dcdfc758f1 HTTP/1.1" 200 None 2026-03-13 02:50:32,793 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 02:50:32,838 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0d8639d19175818252cd6 HTTP/1.1" 200 None 2026-03-13 02:50:32,845 - lava_callback - ERROR - Error processing callback for node 69b0d8639d19175818252cd6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:34542 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56600 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41000 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54360 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36372 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36382 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36396 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45748 - "GET / HTTP/1.0" 200 OK 2026-03-13 02:51:32,700 - lava_callback - INFO - Received callback for node 69b0d8b19d19175818252ce6 INFO: 172.20.0.1:54566 - "POST /node/69b0d8b19d19175818252ce6 HTTP/1.0" 202 Accepted 2026-03-13 02:51:32,706 - lava_callback - INFO - Starting async job submit for node 69b0d8b19d19175818252ce6 2026-03-13 02:51:32,707 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 02:51:32,721 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0d8b19d19175818252ce6 HTTP/1.1" 200 None 2026-03-13 02:51:32,721 - lava_callback - DEBUG - Processing node 69b0d8b19d19175818252ce6: baseline-arm64-baylibre 2026-03-13 02:51:32,722 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0d8b19d19175818252ce6 2026-03-13 02:51:32,723 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 02:51:32,744 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0d8b19d19175818252ce6 HTTP/1.1" 200 None 2026-03-13 02:51:32,745 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 02:51:32,760 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aafea227cb11dcdfc758f1 HTTP/1.1" 200 None 2026-03-13 02:51:32,762 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 02:51:32,783 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0d8b19d19175818252ce6 HTTP/1.1" 200 None 2026-03-13 02:51:32,792 - lava_callback - ERROR - Error processing callback for node 69b0d8b19d19175818252ce6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:54582 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41756 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45400 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43990 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34348 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34362 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34378 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46754 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60624 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54278 - "GET / HTTP/1.0" 200 OK 2026-03-13 02:52:52,813 - lava_callback - INFO - Received callback for node 69b0d9019d19175818252ced INFO: 172.20.0.1:43028 - "POST /node/69b0d9019d19175818252ced HTTP/1.0" 202 Accepted 2026-03-13 02:52:52,819 - lava_callback - INFO - Starting async job submit for node 69b0d9019d19175818252ced 2026-03-13 02:52:52,819 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 02:52:52,835 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0d9019d19175818252ced HTTP/1.1" 200 None 2026-03-13 02:52:52,836 - lava_callback - DEBUG - Processing node 69b0d9019d19175818252ced: baseline-arm64-baylibre 2026-03-13 02:52:52,836 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0d9019d19175818252ced 2026-03-13 02:52:52,837 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 02:52:52,856 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0d9019d19175818252ced HTTP/1.1" 200 None 2026-03-13 02:52:52,857 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 02:52:52,871 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aafea227cb11dcdfc758f1 HTTP/1.1" 200 None 2026-03-13 02:52:52,872 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 02:52:52,893 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0d9019d19175818252ced HTTP/1.1" 200 None 2026-03-13 02:52:52,899 - lava_callback - ERROR - Error processing callback for node 69b0d9019d19175818252ced: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:43044 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38766 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49692 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49706 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49710 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34440 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53930 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52640 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52774 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44126 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43490 - "GET / HTTP/1.0" 200 OK 2026-03-13 02:54:12,873 - lava_callback - INFO - Received callback for node 69b0d94f9d19175818252cfe INFO: 172.20.0.1:43496 - "POST /node/69b0d94f9d19175818252cfe HTTP/1.0" 202 Accepted 2026-03-13 02:54:12,879 - lava_callback - INFO - Starting async job submit for node 69b0d94f9d19175818252cfe 2026-03-13 02:54:12,880 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 02:54:12,894 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0d94f9d19175818252cfe HTTP/1.1" 200 None 2026-03-13 02:54:12,895 - lava_callback - DEBUG - Processing node 69b0d94f9d19175818252cfe: baseline-arm64-baylibre 2026-03-13 02:54:12,895 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0d94f9d19175818252cfe 2026-03-13 02:54:12,896 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 02:54:12,909 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0d94f9d19175818252cfe HTTP/1.1" 200 None 2026-03-13 02:54:12,911 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 02:54:12,931 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aafea227cb11dcdfc758f1 HTTP/1.1" 200 None 2026-03-13 02:54:12,932 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 02:54:12,955 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0d94f9d19175818252cfe HTTP/1.1" 200 None 2026-03-13 02:54:12,956 - lava_callback - ERROR - Error processing callback for node 69b0d94f9d19175818252cfe: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:43502 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43516 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51482 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36284 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54446 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48840 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59002 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39062 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39064 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39068 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56858 - "GET / HTTP/1.0" 200 OK 2026-03-13 02:55:32,942 - lava_callback - INFO - Received callback for node 69b0d99d9d19175818252d0c INFO: 172.20.0.1:56708 - "POST /node/69b0d99d9d19175818252d0c HTTP/1.0" 202 Accepted 2026-03-13 02:55:32,947 - lava_callback - INFO - Starting async job submit for node 69b0d99d9d19175818252d0c 2026-03-13 02:55:32,948 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 02:55:32,962 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0d99d9d19175818252d0c HTTP/1.1" 200 None 2026-03-13 02:55:32,963 - lava_callback - DEBUG - Processing node 69b0d99d9d19175818252d0c: baseline-arm64-baylibre 2026-03-13 02:55:32,963 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0d99d9d19175818252d0c 2026-03-13 02:55:32,964 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 02:55:32,998 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0d99d9d19175818252d0c HTTP/1.1" 200 None 2026-03-13 02:55:32,999 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 02:55:33,012 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aafea227cb11dcdfc758f1 HTTP/1.1" 200 None 2026-03-13 02:55:33,014 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 02:55:33,033 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0d99d9d19175818252d0c HTTP/1.1" 200 None 2026-03-13 02:55:33,040 - lava_callback - ERROR - Error processing callback for node 69b0d99d9d19175818252d0c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:56718 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38580 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53716 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33568 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34532 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34548 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34562 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32794 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53866 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56266 - "GET / HTTP/1.0" 200 OK 2026-03-13 02:56:53,060 - lava_callback - INFO - Received callback for node 69b0d9eb9d19175818252d1d INFO: 172.20.0.1:56452 - "POST /node/69b0d9eb9d19175818252d1d HTTP/1.0" 202 Accepted 2026-03-13 02:56:53,066 - lava_callback - INFO - Starting async job submit for node 69b0d9eb9d19175818252d1d 2026-03-13 02:56:53,067 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 02:56:53,097 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0d9eb9d19175818252d1d HTTP/1.1" 200 None 2026-03-13 02:56:53,098 - lava_callback - DEBUG - Processing node 69b0d9eb9d19175818252d1d: baseline-arm64-baylibre 2026-03-13 02:56:53,098 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0d9eb9d19175818252d1d 2026-03-13 02:56:53,099 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 02:56:53,133 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0d9eb9d19175818252d1d HTTP/1.1" 200 None 2026-03-13 02:56:53,134 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 02:56:53,166 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aafea227cb11dcdfc758f1 HTTP/1.1" 200 None 2026-03-13 02:56:53,168 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 02:56:53,211 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0d9eb9d19175818252d1d HTTP/1.1" 200 None 2026-03-13 02:56:53,212 - lava_callback - ERROR - Error processing callback for node 69b0d9eb9d19175818252d1d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:56458 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60564 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46774 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46790 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46798 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35496 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35198 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54496 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57284 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36310 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46644 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46648 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46656 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35132 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35136 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58558 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48076 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52672 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51646 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51658 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51674 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46870 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56368 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45648 - "GET / HTTP/1.0" 200 OK 2026-03-13 02:59:50,013 - lava_callback - INFO - Received callback for node 69b0da959d19175818252d3f INFO: 172.20.0.1:57070 - "POST /node/69b0da959d19175818252d3f HTTP/1.0" 202 Accepted 2026-03-13 02:59:50,018 - lava_callback - INFO - Starting async job submit for node 69b0da959d19175818252d3f 2026-03-13 02:59:50,019 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 02:59:50,034 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0da959d19175818252d3f HTTP/1.1" 200 None 2026-03-13 02:59:50,035 - lava_callback - DEBUG - Processing node 69b0da959d19175818252d3f: baseline-arm64-clabbe 2026-03-13 02:59:50,035 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0da959d19175818252d3f 2026-03-13 02:59:50,035 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 02:59:50,048 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0da959d19175818252d3f HTTP/1.1" 200 None 2026-03-13 02:59:50,049 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 02:59:50,071 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aafea227cb11dcdfc758f1 HTTP/1.1" 200 None 2026-03-13 02:59:50,073 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 02:59:50,139 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0da959d19175818252d3f HTTP/1.1" 200 None 2026-03-13 02:59:50,140 - lava_callback - ERROR - Error processing callback for node 69b0da959d19175818252d3f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:57082 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39900 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53438 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53446 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53454 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51806 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33630 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33740 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58710 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33382 - "GET / HTTP/1.0" 200 OK 2026-03-13 03:01:10,064 - lava_callback - INFO - Received callback for node 69b0dae39d19175818252d49 INFO: 172.20.0.1:53888 - "POST /node/69b0dae39d19175818252d49 HTTP/1.0" 202 Accepted 2026-03-13 03:01:10,070 - lava_callback - INFO - Starting async job submit for node 69b0dae39d19175818252d49 2026-03-13 03:01:10,071 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:01:10,096 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0dae39d19175818252d49 HTTP/1.1" 200 None 2026-03-13 03:01:10,097 - lava_callback - DEBUG - Processing node 69b0dae39d19175818252d49: baseline-arm64-clabbe 2026-03-13 03:01:10,097 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0dae39d19175818252d49 2026-03-13 03:01:10,098 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:01:10,111 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0dae39d19175818252d49 HTTP/1.1" 200 None 2026-03-13 03:01:10,113 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:01:10,138 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aafea227cb11dcdfc758f1 HTTP/1.1" 200 None 2026-03-13 03:01:10,139 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:01:10,159 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0dae39d19175818252d49 HTTP/1.1" 200 None 2026-03-13 03:01:10,173 - lava_callback - ERROR - Error processing callback for node 69b0dae39d19175818252d49: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:53896 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53904 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53914 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40882 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37752 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44340 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37978 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58466 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44910 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44922 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44930 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40268 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49070 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55052 - "GET / HTTP/1.0" 200 OK 2026-03-13 03:02:49,735 - lava_callback - INFO - Received callback for node 69b3718b974c91e64560f5b8 INFO: 172.20.0.1:45276 - "POST /node/69b3718b974c91e64560f5b8 HTTP/1.0" 202 Accepted 2026-03-13 03:02:49,743 - lava_callback - INFO - Starting async job submit for node 69b3718b974c91e64560f5b8 2026-03-13 03:02:49,785 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:02:49,800 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3718b974c91e64560f5b8 HTTP/1.1" 200 None 2026-03-13 03:02:49,801 - lava_callback - DEBUG - Processing node 69b3718b974c91e64560f5b8: baseline-nfs-arm64 2026-03-13 03:02:50,890 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 03:02:51,086 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 03:02:51,088 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b3718b974c91e64560f5b8/log.txt.gz 2026-03-13 03:02:51,721 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 03:02:51,962 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 03:02:51,965 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b3718b974c91e64560f5b8/lava-logs.yaml 2026-03-13 03:02:51,986 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 03:02:52,174 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 03:02:52,176 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b3718b974c91e64560f5b8/lava_callback.json.gz DEBUG: Setup failed for 69b3718b974c91e64560f5b8. Transitting job node result: incomplete -> fail 2026-03-13 03:02:52,264 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:02:52,286 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3718b974c91e64560f5b8 HTTP/1.1" 200 None 2026-03-13 03:02:52,287 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:02:52,319 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3594d974c91e64560e4b2 HTTP/1.1" 200 None 2026-03-13 03:02:52,321 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:02:52,344 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b3718b974c91e64560f5b8 HTTP/1.1" 200 None 2026-03-13 03:02:52,464 - lava_callback - INFO - Completed processing callback for node 69b3718b974c91e64560f5b8 INFO: 172.20.0.1:45292 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33978 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34366 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34374 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-13 03:03:16,026 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:03:16,046 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:34386 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41032 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47680 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46764 - "GET / HTTP/1.0" 200 OK 2026-03-13 03:03:50,125 - lava_callback - INFO - Received callback for node 69b0db839d19175818252d6a INFO: 172.20.0.1:59478 - "POST /node/69b0db839d19175818252d6a HTTP/1.0" 202 Accepted 2026-03-13 03:03:50,131 - lava_callback - INFO - Starting async job submit for node 69b0db839d19175818252d6a 2026-03-13 03:03:50,132 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:03:50,159 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0db839d19175818252d6a HTTP/1.1" 200 None 2026-03-13 03:03:50,160 - lava_callback - DEBUG - Processing node 69b0db839d19175818252d6a: baseline-arm64-clabbe 2026-03-13 03:03:50,160 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0db839d19175818252d6a 2026-03-13 03:03:50,161 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:03:50,174 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0db839d19175818252d6a HTTP/1.1" 200 None 2026-03-13 03:03:50,175 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:03:50,188 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aafea227cb11dcdfc758f1 HTTP/1.1" 200 None 2026-03-13 03:03:50,189 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:03:50,213 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0db839d19175818252d6a HTTP/1.1" 200 None 2026-03-13 03:03:50,214 - lava_callback - ERROR - Error processing callback for node 69b0db839d19175818252d6a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:59490 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45898 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50978 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50988 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51000 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35968 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56670 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53548 - "GET / HTTP/1.0" 200 OK 2026-03-13 03:04:53,482 - lava_callback - INFO - Received callback for node 69b0dbcb9d19175818252d74 INFO: 172.20.0.1:57244 - "POST /node/69b0dbcb9d19175818252d74 HTTP/1.0" 202 Accepted 2026-03-13 03:04:53,487 - lava_callback - INFO - Starting async job submit for node 69b0dbcb9d19175818252d74 2026-03-13 03:04:53,488 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:04:53,502 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0dbcb9d19175818252d74 HTTP/1.1" 200 None 2026-03-13 03:04:53,503 - lava_callback - DEBUG - Processing node 69b0dbcb9d19175818252d74: baseline-arm64-baylibre 2026-03-13 03:04:53,503 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0dbcb9d19175818252d74 2026-03-13 03:04:53,504 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:04:53,518 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0dbcb9d19175818252d74 HTTP/1.1" 200 None 2026-03-13 03:04:53,520 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:04:53,533 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aafea227cb11dcdfc758f1 HTTP/1.1" 200 None 2026-03-13 03:04:53,534 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:04:53,555 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0dbcb9d19175818252d74 HTTP/1.1" 200 None 2026-03-13 03:04:53,562 - lava_callback - ERROR - Error processing callback for node 69b0dbcb9d19175818252d74: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:57252 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40542 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47280 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47292 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47298 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45250 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46694 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42328 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42800 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36276 - "GET / HTTP/1.0" 200 OK 2026-03-13 03:06:10,240 - lava_callback - INFO - Received callback for node 69b0dc1d9d19175818252d85 INFO: 172.20.0.1:34670 - "POST /node/69b0dc1d9d19175818252d85 HTTP/1.0" 202 Accepted 2026-03-13 03:06:10,246 - lava_callback - INFO - Starting async job submit for node 69b0dc1d9d19175818252d85 2026-03-13 03:06:10,247 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:06:10,267 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0dc1d9d19175818252d85 HTTP/1.1" 200 None 2026-03-13 03:06:10,268 - lava_callback - DEBUG - Processing node 69b0dc1d9d19175818252d85: baseline-arm64-clabbe 2026-03-13 03:06:10,268 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0dc1d9d19175818252d85 2026-03-13 03:06:10,269 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:06:10,282 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0dc1d9d19175818252d85 HTTP/1.1" 200 None 2026-03-13 03:06:10,283 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.20.0.1:34682 - "GET / HTTP/1.0" 200 OK 2026-03-13 03:06:10,311 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aafea227cb11dcdfc758f1 HTTP/1.1" 200 None 2026-03-13 03:06:10,312 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:06:10,333 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0dc1d9d19175818252d85 HTTP/1.1" 200 None 2026-03-13 03:06:10,334 - lava_callback - ERROR - Error processing callback for node 69b0dc1d9d19175818252d85: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:34692 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34694 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56378 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37284 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39452 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49242 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45514 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53694 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53708 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53716 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48582 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54058 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33972 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47132 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44288 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33960 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33968 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33970 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50850 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38804 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50418 - "GET / HTTP/1.0" 200 OK 2026-03-13 03:08:50,360 - lava_callback - INFO - Received callback for node 69b0dcb49d19175818252da3 INFO: 172.20.0.1:60748 - "POST /node/69b0dcb49d19175818252da3 HTTP/1.0" 202 Accepted 2026-03-13 03:08:50,366 - lava_callback - INFO - Starting async job submit for node 69b0dcb49d19175818252da3 2026-03-13 03:08:50,367 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:08:50,381 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0dcb49d19175818252da3 HTTP/1.1" 200 None 2026-03-13 03:08:50,382 - lava_callback - DEBUG - Processing node 69b0dcb49d19175818252da3: baseline-arm64-clabbe 2026-03-13 03:08:50,382 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0dcb49d19175818252da3 2026-03-13 03:08:50,383 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:08:50,396 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0dcb49d19175818252da3 HTTP/1.1" 200 None 2026-03-13 03:08:50,397 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:08:50,411 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aafea227cb11dcdfc758f1 HTTP/1.1" 200 None 2026-03-13 03:08:50,412 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:08:50,432 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0dcb49d19175818252da3 HTTP/1.1" 200 None 2026-03-13 03:08:50,434 - lava_callback - ERROR - Error processing callback for node 69b0dcb49d19175818252da3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:60758 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36682 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50894 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50896 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50904 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40452 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53158 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52360 - "GET / HTTP/1.0" 200 OK 2026-03-13 03:09:53,756 - lava_callback - INFO - Received callback for node 69b0dcfa9d19175818252dab INFO: 172.20.0.1:44846 - "POST /node/69b0dcfa9d19175818252dab HTTP/1.0" 202 Accepted 2026-03-13 03:09:53,762 - lava_callback - INFO - Starting async job submit for node 69b0dcfa9d19175818252dab 2026-03-13 03:09:53,763 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:09:53,785 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0dcfa9d19175818252dab HTTP/1.1" 200 None 2026-03-13 03:09:53,785 - lava_callback - DEBUG - Processing node 69b0dcfa9d19175818252dab: baseline-arm64-baylibre 2026-03-13 03:09:53,786 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0dcfa9d19175818252dab 2026-03-13 03:09:53,786 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:09:53,813 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0dcfa9d19175818252dab HTTP/1.1" 200 None 2026-03-13 03:09:53,814 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:09:53,828 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afd3b0c9209b146d8e1d77 HTTP/1.1" 200 None 2026-03-13 03:09:53,829 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:09:53,849 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0dcfa9d19175818252dab HTTP/1.1" 200 None 2026-03-13 03:09:53,850 - lava_callback - ERROR - Error processing callback for node 69b0dcfa9d19175818252dab: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:44862 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51890 - "GET / HTTP/1.0" 200 OK 2026-03-13 03:10:09,516 - lava_callback - INFO - Received callback for node 69b363d4974c91e64560ed70 INFO: 172.20.0.1:51900 - "POST /node/69b363d4974c91e64560ed70 HTTP/1.0" 202 Accepted 2026-03-13 03:10:09,522 - lava_callback - INFO - Starting async job submit for node 69b363d4974c91e64560ed70 2026-03-13 03:10:09,543 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:10:09,558 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b363d4974c91e64560ed70 HTTP/1.1" 200 None 2026-03-13 03:10:09,559 - lava_callback - DEBUG - Processing node 69b363d4974c91e64560ed70: baseline-arm64-broonie 2026-03-13 03:10:09,800 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 03:10:10,003 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 03:10:10,004 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69b363d4974c91e64560ed70/log.txt.gz 2026-03-13 03:10:10,158 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:36478 - "GET / HTTP/1.0" 200 OK 2026-03-13 03:10:10,371 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 03:10:10,373 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69b363d4974c91e64560ed70/lava-logs.yaml 2026-03-13 03:10:10,380 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 03:10:10,543 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 03:10:10,545 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69b363d4974c91e64560ed70/lava_callback.json.gz 2026-03-13 03:10:10,568 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:10:10,582 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b363d4974c91e64560ed70 HTTP/1.1" 200 None 2026-03-13 03:10:10,584 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:10:10,598 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3594d974c91e64560e4b2 HTTP/1.1" 200 None 2026-03-13 03:10:10,600 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:10:10,625 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b363d4974c91e64560ed70 HTTP/1.1" 200 None 2026-03-13 03:10:10,626 - lava_callback - INFO - Completed processing callback for node 69b363d4974c91e64560ed70 INFO: 172.20.0.1:36492 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-13 03:10:16,049 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:10:16,072 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:36502 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35534 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52964 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35760 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55612 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49094 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41360 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41366 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41378 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39424 - "GET / HTTP/1.0" 200 OK 2026-03-13 03:11:30,455 - lava_callback - INFO - Received callback for node 69b0dd4f9d19175818252dbb INFO: 172.20.0.1:41258 - "POST /node/69b0dd4f9d19175818252dbb HTTP/1.0" 202 Accepted 2026-03-13 03:11:30,461 - lava_callback - INFO - Starting async job submit for node 69b0dd4f9d19175818252dbb 2026-03-13 03:11:30,462 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:11:30,478 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0dd4f9d19175818252dbb HTTP/1.1" 200 None 2026-03-13 03:11:30,478 - lava_callback - DEBUG - Processing node 69b0dd4f9d19175818252dbb: baseline-arm64-clabbe 2026-03-13 03:11:30,479 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0dd4f9d19175818252dbb 2026-03-13 03:11:30,479 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:11:30,511 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0dd4f9d19175818252dbb HTTP/1.1" 200 None 2026-03-13 03:11:30,513 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:11:30,526 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aafea227cb11dcdfc758f1 HTTP/1.1" 200 None 2026-03-13 03:11:30,528 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:11:30,548 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0dd4f9d19175818252dbb HTTP/1.1" 200 None 2026-03-13 03:11:30,549 - lava_callback - ERROR - Error processing callback for node 69b0dd4f9d19175818252dbb: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:41274 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35820 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59476 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42056 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36728 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36744 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36748 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39444 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42146 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50640 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44396 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39280 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56128 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56144 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56152 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53942 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46372 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40136 - "GET / HTTP/1.0" 200 OK 2026-03-13 03:13:53,961 - lava_callback - INFO - Received callback for node 69b0dde39d19175818252dd5 INFO: 172.20.0.1:52426 - "POST /node/69b0dde39d19175818252dd5 HTTP/1.0" 202 Accepted 2026-03-13 03:13:53,966 - lava_callback - INFO - Starting async job submit for node 69b0dde39d19175818252dd5 2026-03-13 03:13:53,967 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:13:53,981 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0dde39d19175818252dd5 HTTP/1.1" 200 None 2026-03-13 03:13:53,982 - lava_callback - DEBUG - Processing node 69b0dde39d19175818252dd5: baseline-arm64-baylibre 2026-03-13 03:13:53,983 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0dde39d19175818252dd5 2026-03-13 03:13:53,983 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:13:54,006 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0dde39d19175818252dd5 HTTP/1.1" 200 None 2026-03-13 03:13:54,007 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:13:54,021 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afd3b0c9209b146d8e1d77 HTTP/1.1" 200 None 2026-03-13 03:13:54,022 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:13:54,070 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0dde39d19175818252dd5 HTTP/1.1" 200 None 2026-03-13 03:13:54,071 - lava_callback - ERROR - Error processing callback for node 69b0dde39d19175818252dd5: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:52430 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54254 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43994 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43996 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44006 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50130 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53908 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44492 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48108 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33682 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38280 - "GET / HTTP/1.0" 200 OK 2026-03-13 03:15:14,045 - lava_callback - INFO - Received callback for node 69b0de329d19175818252de0 INFO: 172.20.0.1:38290 - "POST /node/69b0de329d19175818252de0 HTTP/1.0" 202 Accepted 2026-03-13 03:15:14,050 - lava_callback - INFO - Starting async job submit for node 69b0de329d19175818252de0 2026-03-13 03:15:14,051 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:15:14,066 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0de329d19175818252de0 HTTP/1.1" 200 None 2026-03-13 03:15:14,067 - lava_callback - DEBUG - Processing node 69b0de329d19175818252de0: baseline-arm64-baylibre 2026-03-13 03:15:14,067 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0de329d19175818252de0 2026-03-13 03:15:14,068 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:15:14,103 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0de329d19175818252de0 HTTP/1.1" 200 None 2026-03-13 03:15:14,104 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:15:14,144 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afd3b0c9209b146d8e1d77 HTTP/1.1" 200 None 2026-03-13 03:15:14,146 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:15:14,170 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0de329d19175818252de0 HTTP/1.1" 200 None 2026-03-13 03:15:14,172 - lava_callback - ERROR - Error processing callback for node 69b0de329d19175818252de0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:38294 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38310 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57904 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41334 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36142 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56174 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40102 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38102 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38108 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38120 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52136 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34780 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38048 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35196 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40960 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55434 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55438 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55448 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52290 - "GET / HTTP/1.0" 200 OK 2026-03-13 03:17:34,187 - lava_callback - INFO - Received callback for node 69b0decb9d19175818252dff INFO: 172.20.0.1:56008 - "POST /node/69b0decb9d19175818252dff HTTP/1.0" 202 Accepted 2026-03-13 03:17:34,192 - lava_callback - INFO - Starting async job submit for node 69b0decb9d19175818252dff 2026-03-13 03:17:34,193 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:17:34,208 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0decb9d19175818252dff HTTP/1.1" 200 None 2026-03-13 03:17:34,209 - lava_callback - DEBUG - Processing node 69b0decb9d19175818252dff: baseline-arm64-baylibre 2026-03-13 03:17:34,209 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0decb9d19175818252dff 2026-03-13 03:17:34,210 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:17:34,232 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0decb9d19175818252dff HTTP/1.1" 200 None 2026-03-13 03:17:34,234 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:17:34,247 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afd3b0c9209b146d8e1d77 HTTP/1.1" 200 None 2026-03-13 03:17:34,248 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:17:34,269 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0decb9d19175818252dff HTTP/1.1" 200 None 2026-03-13 03:17:34,274 - lava_callback - ERROR - Error processing callback for node 69b0decb9d19175818252dff: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:56016 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39032 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59660 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55524 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39544 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39550 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39552 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46330 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49862 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51114 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33250 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49452 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43488 - "GET / HTTP/1.0" 200 OK 2026-03-13 03:19:10,804 - lava_callback - INFO - Received callback for node 69b0df1d9d19175818252e0a INFO: 172.20.0.1:43494 - "POST /node/69b0df1d9d19175818252e0a HTTP/1.0" 202 Accepted 2026-03-13 03:19:10,810 - lava_callback - INFO - Starting async job submit for node 69b0df1d9d19175818252e0a 2026-03-13 03:19:10,811 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:19:10,826 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0df1d9d19175818252e0a HTTP/1.1" 200 None 2026-03-13 03:19:10,827 - lava_callback - DEBUG - Processing node 69b0df1d9d19175818252e0a: baseline-arm64-clabbe 2026-03-13 03:19:10,827 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0df1d9d19175818252e0a 2026-03-13 03:19:10,828 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:19:10,871 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0df1d9d19175818252e0a HTTP/1.1" 200 None 2026-03-13 03:19:10,873 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:19:10,898 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afd3b0c9209b146d8e1d77 HTTP/1.1" 200 None 2026-03-13 03:19:10,900 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:19:10,929 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0df1d9d19175818252e0a HTTP/1.1" 200 None 2026-03-13 03:19:10,931 - lava_callback - ERROR - Error processing callback for node 69b0df1d9d19175818252e0a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:43500 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43502 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57550 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32790 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56012 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52704 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40598 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47082 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47084 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47092 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58674 - "GET / HTTP/1.0" 200 OK 2026-03-13 03:20:30,820 - lava_callback - INFO - Received callback for node 69b0df699d19175818252e1c INFO: 172.20.0.1:42534 - "POST /node/69b0df699d19175818252e1c HTTP/1.0" 202 Accepted 2026-03-13 03:20:30,825 - lava_callback - INFO - Starting async job submit for node 69b0df699d19175818252e1c 2026-03-13 03:20:30,826 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:20:30,840 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0df699d19175818252e1c HTTP/1.1" 200 None 2026-03-13 03:20:30,841 - lava_callback - DEBUG - Processing node 69b0df699d19175818252e1c: baseline-arm64-clabbe 2026-03-13 03:20:30,841 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0df699d19175818252e1c 2026-03-13 03:20:30,842 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:20:30,854 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0df699d19175818252e1c HTTP/1.1" 200 None 2026-03-13 03:20:30,856 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:20:30,869 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afd3b0c9209b146d8e1d77 HTTP/1.1" 200 None 2026-03-13 03:20:30,870 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:20:30,891 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0df699d19175818252e1c HTTP/1.1" 200 None 2026-03-13 03:20:30,897 - lava_callback - ERROR - Error processing callback for node 69b0df699d19175818252e1c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:42546 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49790 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54772 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34350 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36444 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36460 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36474 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41338 - "GET / HTTP/1.0" 200 OK 2026-03-13 03:21:30,872 - lava_callback - INFO - Received callback for node 69b0dfb69d19175818252e2e INFO: 172.20.0.1:51042 - "POST /node/69b0dfb69d19175818252e2e HTTP/1.0" 202 Accepted 2026-03-13 03:21:30,877 - lava_callback - INFO - Starting async job submit for node 69b0dfb69d19175818252e2e 2026-03-13 03:21:30,879 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:21:30,893 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0dfb69d19175818252e2e HTTP/1.1" 200 None 2026-03-13 03:21:30,894 - lava_callback - DEBUG - Processing node 69b0dfb69d19175818252e2e: baseline-arm64-clabbe 2026-03-13 03:21:30,894 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0dfb69d19175818252e2e 2026-03-13 03:21:30,895 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:21:30,908 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0dfb69d19175818252e2e HTTP/1.1" 200 None 2026-03-13 03:21:30,910 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:21:30,946 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afd3b0c9209b146d8e1d77 HTTP/1.1" 200 None 2026-03-13 03:21:30,948 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:21:30,968 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0dfb69d19175818252e2e HTTP/1.1" 200 None 2026-03-13 03:21:30,970 - lava_callback - ERROR - Error processing callback for node 69b0dfb69d19175818252e2e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:51054 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58886 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54016 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35434 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38376 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38388 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38390 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51230 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53736 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59006 - "GET / HTTP/1.0" 200 OK 2026-03-13 03:22:50,925 - lava_callback - INFO - Received callback for node 69b0e0059d19175818252e3f INFO: 172.20.0.1:42418 - "POST /node/69b0e0059d19175818252e3f HTTP/1.0" 202 Accepted 2026-03-13 03:22:50,930 - lava_callback - INFO - Starting async job submit for node 69b0e0059d19175818252e3f 2026-03-13 03:22:50,931 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:22:50,965 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0e0059d19175818252e3f HTTP/1.1" 200 None 2026-03-13 03:22:50,966 - lava_callback - DEBUG - Processing node 69b0e0059d19175818252e3f: baseline-arm64-clabbe 2026-03-13 03:22:50,966 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0e0059d19175818252e3f 2026-03-13 03:22:50,967 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:22:50,980 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0e0059d19175818252e3f HTTP/1.1" 200 None 2026-03-13 03:22:50,981 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:22:51,016 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afd3b0c9209b146d8e1d77 HTTP/1.1" 200 None 2026-03-13 03:22:51,018 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:22:51,068 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0e0059d19175818252e3f HTTP/1.1" 200 None 2026-03-13 03:22:51,070 - lava_callback - ERROR - Error processing callback for node 69b0e0059d19175818252e3f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:42434 - "GET / HTTP/1.0" 200 OK 2026-03-13 03:23:06,466 - lava_callback - INFO - Received callback for node 69b37e65974c91e64560f80b INFO: 172.20.0.1:60240 - "POST /node/69b37e65974c91e64560f80b HTTP/1.0" 202 Accepted 2026-03-13 03:23:06,474 - lava_callback - INFO - Starting async job submit for node 69b37e65974c91e64560f80b INFO: 172.20.0.1:60250 - "GET / HTTP/1.0" 200 OK 2026-03-13 03:23:06,520 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:23:06,534 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b37e65974c91e64560f80b HTTP/1.1" 200 None 2026-03-13 03:23:06,535 - lava_callback - DEBUG - Processing node 69b37e65974c91e64560f80b: baseline-nfs-arm64 2026-03-13 03:23:07,611 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 03:23:07,809 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 03:23:07,811 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b37e65974c91e64560f80b/log.txt.gz 2026-03-13 03:23:08,457 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 03:23:08,696 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 03:23:08,698 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b37e65974c91e64560f80b/lava-logs.yaml 2026-03-13 03:23:08,720 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 03:23:08,927 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 03:23:08,929 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b37e65974c91e64560f80b/lava_callback.json.gz DEBUG: Setup failed for 69b37e65974c91e64560f80b. Transitting job node result: incomplete -> fail 2026-03-13 03:23:09,019 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:23:09,033 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b37e65974c91e64560f80b HTTP/1.1" 200 None 2026-03-13 03:23:09,034 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:23:09,047 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3594d974c91e64560e4b2 HTTP/1.1" 200 None 2026-03-13 03:23:09,048 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:23:09,072 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b37e65974c91e64560f80b HTTP/1.1" 200 None 2026-03-13 03:23:09,168 - lava_callback - INFO - Completed processing callback for node 69b37e65974c91e64560f80b INFO: 172.20.0.1:45646 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45660 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-13 03:23:16,077 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:23:16,100 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:45676 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49024 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58732 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56592 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47938 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41676 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52628 - "GET / HTTP/1.0" 200 OK 2026-03-13 03:24:10,968 - lava_callback - INFO - Received callback for node 69b0e0529d19175818252e56 INFO: 172.20.0.1:52632 - "POST /node/69b0e0529d19175818252e56 HTTP/1.0" 202 Accepted 2026-03-13 03:24:10,973 - lava_callback - INFO - Starting async job submit for node 69b0e0529d19175818252e56 2026-03-13 03:24:10,974 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:24:10,987 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0e0529d19175818252e56 HTTP/1.1" 200 None 2026-03-13 03:24:10,988 - lava_callback - DEBUG - Processing node 69b0e0529d19175818252e56: baseline-arm64-clabbe 2026-03-13 03:24:10,988 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0e0529d19175818252e56 2026-03-13 03:24:10,989 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:24:11,002 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0e0529d19175818252e56 HTTP/1.1" 200 None 2026-03-13 03:24:11,004 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:24:11,019 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afd3b0c9209b146d8e1d77 HTTP/1.1" 200 None 2026-03-13 03:24:11,020 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:24:11,041 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0e0529d19175818252e56 HTTP/1.1" 200 None 2026-03-13 03:24:11,049 - lava_callback - ERROR - Error processing callback for node 69b0e0529d19175818252e56: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:52638 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52654 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50864 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45678 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38960 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40304 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35006 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45818 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45824 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45838 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35948 - "GET / HTTP/1.0" 200 OK 2026-03-13 03:25:31,022 - lava_callback - INFO - Received callback for node 69b0e0a19d19175818252e5f INFO: 172.20.0.1:46718 - "POST /node/69b0e0a19d19175818252e5f HTTP/1.0" 202 Accepted 2026-03-13 03:25:31,028 - lava_callback - INFO - Starting async job submit for node 69b0e0a19d19175818252e5f 2026-03-13 03:25:31,028 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:25:31,042 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0e0a19d19175818252e5f HTTP/1.1" 200 None 2026-03-13 03:25:31,043 - lava_callback - DEBUG - Processing node 69b0e0a19d19175818252e5f: baseline-arm64-clabbe 2026-03-13 03:25:31,044 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0e0a19d19175818252e5f 2026-03-13 03:25:31,044 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:25:31,057 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0e0a19d19175818252e5f HTTP/1.1" 200 None 2026-03-13 03:25:31,058 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:25:31,071 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afd3b0c9209b146d8e1d77 HTTP/1.1" 200 None 2026-03-13 03:25:31,073 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:25:31,099 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0e0a19d19175818252e5f HTTP/1.1" 200 None 2026-03-13 03:25:31,101 - lava_callback - ERROR - Error processing callback for node 69b0e0a19d19175818252e5f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:46734 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44998 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36584 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50466 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48570 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48582 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48584 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36556 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37572 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45498 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49076 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46404 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42674 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42684 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42698 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48722 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50296 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51924 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44948 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46566 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53762 - "GET / HTTP/1.0" 200 OK 2026-03-13 03:28:14,741 - lava_callback - INFO - Received callback for node 69b0e1449d19175818252e93 INFO: 172.20.0.1:53778 - "POST /node/69b0e1449d19175818252e93 HTTP/1.0" 202 Accepted 2026-03-13 03:28:14,747 - lava_callback - INFO - Starting async job submit for node 69b0e1449d19175818252e93 2026-03-13 03:28:14,748 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:28:14,806 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0e1449d19175818252e93 HTTP/1.1" 200 None 2026-03-13 03:28:14,807 - lava_callback - DEBUG - Processing node 69b0e1449d19175818252e93: baseline-arm-baylibre 2026-03-13 03:28:14,807 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0e1449d19175818252e93 2026-03-13 03:28:14,808 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:28:14,827 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0e1449d19175818252e93 HTTP/1.1" 200 None 2026-03-13 03:28:14,829 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:28:14,842 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afd3a3c9209b146d8e1d68 HTTP/1.1" 200 None 2026-03-13 03:28:14,843 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:28:14,869 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0e1449d19175818252e93 HTTP/1.1" 200 None 2026-03-13 03:28:14,871 - lava_callback - ERROR - Error processing callback for node 69b0e1449d19175818252e93: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:53784 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53800 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55078 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34462 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34918 - "GET / HTTP/1.0" 200 OK 2026-03-13 03:28:54,721 - lava_callback - INFO - Received callback for node 69b0e1689d19175818252e9f INFO: 172.20.0.1:57642 - "POST /node/69b0e1689d19175818252e9f HTTP/1.0" 202 Accepted 2026-03-13 03:28:54,726 - lava_callback - INFO - Starting async job submit for node 69b0e1689d19175818252e9f 2026-03-13 03:28:54,727 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:28:54,764 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0e1689d19175818252e9f HTTP/1.1" 200 None 2026-03-13 03:28:54,765 - lava_callback - DEBUG - Processing node 69b0e1689d19175818252e9f: baseline-arm-baylibre 2026-03-13 03:28:54,765 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0e1689d19175818252e9f 2026-03-13 03:28:54,766 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:28:54,782 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0e1689d19175818252e9f HTTP/1.1" 200 None 2026-03-13 03:28:54,783 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:28:54,803 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afd3a3c9209b146d8e1d68 HTTP/1.1" 200 None 2026-03-13 03:28:54,804 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:28:54,825 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0e1689d19175818252e9f HTTP/1.1" 200 None 2026-03-13 03:28:54,826 - lava_callback - ERROR - Error processing callback for node 69b0e1689d19175818252e9f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:57646 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55452 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36296 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36310 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36316 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51458 - "GET / HTTP/1.0" 200 OK 2026-03-13 03:29:34,746 - lava_callback - INFO - Received callback for node 69b0e18c9d19175818252eab INFO: 172.20.0.1:49206 - "POST /node/69b0e18c9d19175818252eab HTTP/1.0" 202 Accepted 2026-03-13 03:29:34,752 - lava_callback - INFO - Starting async job submit for node 69b0e18c9d19175818252eab 2026-03-13 03:29:34,753 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:29:34,769 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0e18c9d19175818252eab HTTP/1.1" 200 None 2026-03-13 03:29:34,769 - lava_callback - DEBUG - Processing node 69b0e18c9d19175818252eab: baseline-arm-baylibre 2026-03-13 03:29:34,770 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0e18c9d19175818252eab 2026-03-13 03:29:34,770 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:29:34,783 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0e18c9d19175818252eab HTTP/1.1" 200 None 2026-03-13 03:29:34,784 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:29:34,797 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afd3a3c9209b146d8e1d68 HTTP/1.1" 200 None 2026-03-13 03:29:34,798 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:29:34,819 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0e18c9d19175818252eab HTTP/1.1" 200 None 2026-03-13 03:29:34,824 - lava_callback - ERROR - Error processing callback for node 69b0e18c9d19175818252eab: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:49220 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47530 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47234 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46988 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59374 - "GET / HTTP/1.0" 200 OK 2026-03-13 03:30:14,768 - lava_callback - INFO - Received callback for node 69b0e1b19d19175818252eb9 INFO: 172.20.0.1:59382 - "POST /node/69b0e1b19d19175818252eb9 HTTP/1.0" 202 Accepted 2026-03-13 03:30:14,773 - lava_callback - INFO - Starting async job submit for node 69b0e1b19d19175818252eb9 2026-03-13 03:30:14,774 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:30:14,803 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0e1b19d19175818252eb9 HTTP/1.1" 200 None 2026-03-13 03:30:14,804 - lava_callback - DEBUG - Processing node 69b0e1b19d19175818252eb9: baseline-arm-baylibre 2026-03-13 03:30:14,804 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0e1b19d19175818252eb9 2026-03-13 03:30:14,805 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:30:14,839 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0e1b19d19175818252eb9 HTTP/1.1" 200 None 2026-03-13 03:30:14,840 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:30:14,863 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afd3a3c9209b146d8e1d68 HTTP/1.1" 200 None 2026-03-13 03:30:14,865 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:30:14,894 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0e1b19d19175818252eb9 HTTP/1.1" 200 None 2026-03-13 03:30:14,895 - lava_callback - ERROR - Error processing callback for node 69b0e1b19d19175818252eb9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:59394 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59400 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35286 - "GET / HTTP/1.0" 200 OK 2026-03-13 03:30:34,808 - lava_callback - INFO - Received callback for node 69b0e1d59d19175818252ec8 INFO: 172.20.0.1:35628 - "POST /node/69b0e1d59d19175818252ec8 HTTP/1.0" 202 Accepted 2026-03-13 03:30:34,813 - lava_callback - INFO - Starting async job submit for node 69b0e1d59d19175818252ec8 2026-03-13 03:30:34,814 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:30:34,836 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0e1d59d19175818252ec8 HTTP/1.1" 200 None 2026-03-13 03:30:34,837 - lava_callback - DEBUG - Processing node 69b0e1d59d19175818252ec8: baseline-arm-baylibre 2026-03-13 03:30:34,837 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0e1d59d19175818252ec8 2026-03-13 03:30:34,838 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:30:34,871 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0e1d59d19175818252ec8 HTTP/1.1" 200 None 2026-03-13 03:30:34,872 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:30:34,885 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afd3a3c9209b146d8e1d68 HTTP/1.1" 200 None 2026-03-13 03:30:34,886 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:30:34,906 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0e1d59d19175818252ec8 HTTP/1.1" 200 None 2026-03-13 03:30:34,911 - lava_callback - ERROR - Error processing callback for node 69b0e1d59d19175818252ec8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:35638 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38912 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46330 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47228 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44732 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44742 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44758 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46494 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37388 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59924 - "GET / HTTP/1.0" 200 OK 2026-03-13 03:31:55,508 - lava_callback - INFO - Received callback for node 69b35ee2974c91e64560e9fd INFO: 172.20.0.1:42876 - "POST /node/69b35ee2974c91e64560e9fd HTTP/1.0" 202 Accepted 2026-03-13 03:31:55,514 - lava_callback - INFO - Starting async job submit for node 69b35ee2974c91e64560e9fd 2026-03-13 03:31:55,539 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:31:55,554 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35ee2974c91e64560e9fd HTTP/1.1" 200 None 2026-03-13 03:31:55,555 - lava_callback - DEBUG - Processing node 69b35ee2974c91e64560e9fd: ltp-watchqueue 2026-03-13 03:31:55,894 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 03:31:56,063 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 03:31:56,065 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-watchqueue-69b35ee2974c91e64560e9fd/log.txt.gz 2026-03-13 03:31:56,289 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:42884 - "GET / HTTP/1.0" 200 OK 2026-03-13 03:31:56,523 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 03:31:56,525 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-watchqueue-69b35ee2974c91e64560e9fd/lava-logs.yaml 2026-03-13 03:31:56,534 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 03:31:56,710 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 03:31:56,712 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-watchqueue-69b35ee2974c91e64560e9fd/lava_callback.json.gz 2026-03-13 03:31:56,738 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:31:56,755 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35ee2974c91e64560e9fd HTTP/1.1" 200 None 2026-03-13 03:31:56,757 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:31:56,771 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35934974c91e64560e49d HTTP/1.1" 200 None 2026-03-13 03:31:56,773 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:31:56,797 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b35ee2974c91e64560e9fd HTTP/1.1" 200 None 2026-03-13 03:31:56,798 - lava_callback - INFO - Completed processing callback for node 69b35ee2974c91e64560e9fd INFO: 172.20.0.1:46346 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57740 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57750 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-13 03:32:16,104 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:32:16,120 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:57754 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60342 - "GET / HTTP/1.0" 200 OK 2026-03-13 03:32:31,399 - lava_callback - INFO - Received callback for node 69b0e23f9d19175818252ef8 INFO: 172.20.0.1:52388 - "POST /node/69b0e23f9d19175818252ef8 HTTP/1.0" 202 Accepted 2026-03-13 03:32:31,407 - lava_callback - INFO - Starting async job submit for node 69b0e23f9d19175818252ef8 2026-03-13 03:32:31,409 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:32:31,426 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0e23f9d19175818252ef8 HTTP/1.1" 200 None 2026-03-13 03:32:31,426 - lava_callback - DEBUG - Processing node 69b0e23f9d19175818252ef8: baseline-arm-clabbe 2026-03-13 03:32:31,427 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0e23f9d19175818252ef8 2026-03-13 03:32:31,427 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:32:31,440 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0e23f9d19175818252ef8 HTTP/1.1" 200 None 2026-03-13 03:32:31,441 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:32:31,459 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afd3a3c9209b146d8e1d68 HTTP/1.1" 200 None 2026-03-13 03:32:31,460 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:32:31,481 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0e23f9d19175818252ef8 HTTP/1.1" 200 None 2026-03-13 03:32:31,482 - lava_callback - ERROR - Error processing callback for node 69b0e23f9d19175818252ef8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-13 03:32:35,019 - lava_callback - INFO - Received callback for node 69b35ee0974c91e64560e9fa INFO: 172.20.0.1:52390 - "POST /node/69b35ee0974c91e64560e9fa HTTP/1.0" 202 Accepted 2026-03-13 03:32:35,025 - lava_callback - INFO - Starting async job submit for node 69b35ee0974c91e64560e9fa 2026-03-13 03:32:35,050 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:32:35,071 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35ee0974c91e64560e9fa HTTP/1.1" 200 None 2026-03-13 03:32:35,071 - lava_callback - DEBUG - Processing node 69b35ee0974c91e64560e9fa: ltp-timers 2026-03-13 03:32:35,431 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 03:32:35,571 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 03:32:35,573 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-timers-69b35ee0974c91e64560e9fa/log.txt.gz 2026-03-13 03:32:35,790 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 03:32:35,973 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 03:32:35,975 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-timers-69b35ee0974c91e64560e9fa/lava-logs.yaml 2026-03-13 03:32:35,984 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 03:32:36,174 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 03:32:36,175 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-timers-69b35ee0974c91e64560e9fa/lava_callback.json.gz 2026-03-13 03:32:36,201 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:32:36,218 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35ee0974c91e64560e9fa HTTP/1.1" 200 None 2026-03-13 03:32:36,220 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:32:36,235 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35934974c91e64560e49d HTTP/1.1" 200 None 2026-03-13 03:32:36,237 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:32:36,261 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b35ee0974c91e64560e9fa HTTP/1.1" 200 None 2026-03-13 03:32:36,270 - lava_callback - INFO - Completed processing callback for node 69b35ee0974c91e64560e9fa INFO: 172.20.0.1:52404 - "GET / HTTP/1.0" 200 OK 2026-03-13 03:32:46,123 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:32:46,145 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:48146 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41580 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43730 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43186 - "GET / HTTP/1.0" 200 OK 2026-03-13 03:33:11,362 - lava_callback - INFO - Received callback for node 69b0e2679d19175818252f06 INFO: 172.20.0.1:43198 - "POST /node/69b0e2679d19175818252f06 HTTP/1.0" 202 Accepted 2026-03-13 03:33:11,367 - lava_callback - INFO - Starting async job submit for node 69b0e2679d19175818252f06 2026-03-13 03:33:11,368 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:33:11,385 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0e2679d19175818252f06 HTTP/1.1" 200 None 2026-03-13 03:33:11,386 - lava_callback - DEBUG - Processing node 69b0e2679d19175818252f06: baseline-arm-clabbe 2026-03-13 03:33:11,386 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0e2679d19175818252f06 2026-03-13 03:33:11,387 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:33:11,400 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0e2679d19175818252f06 HTTP/1.1" 200 None 2026-03-13 03:33:11,402 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:33:11,420 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afd3a3c9209b146d8e1d68 HTTP/1.1" 200 None 2026-03-13 03:33:11,421 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:33:11,454 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0e2679d19175818252f06 HTTP/1.1" 200 None 2026-03-13 03:33:11,456 - lava_callback - ERROR - Error processing callback for node 69b0e2679d19175818252f06: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:43206 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43218 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38518 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58922 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57288 - "GET / HTTP/1.0" 200 OK 2026-03-13 03:33:51,422 - lava_callback - INFO - Received callback for node 69b0e28c9d19175818252f1c INFO: 172.20.0.1:39184 - "POST /node/69b0e28c9d19175818252f1c HTTP/1.0" 202 Accepted 2026-03-13 03:33:51,427 - lava_callback - INFO - Starting async job submit for node 69b0e28c9d19175818252f1c 2026-03-13 03:33:51,428 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:33:51,443 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0e28c9d19175818252f1c HTTP/1.1" 200 None 2026-03-13 03:33:51,444 - lava_callback - DEBUG - Processing node 69b0e28c9d19175818252f1c: baseline-arm-clabbe 2026-03-13 03:33:51,444 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0e28c9d19175818252f1c 2026-03-13 03:33:51,445 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:33:51,458 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0e28c9d19175818252f1c HTTP/1.1" 200 None 2026-03-13 03:33:51,459 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:33:51,481 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afd3a3c9209b146d8e1d68 HTTP/1.1" 200 None 2026-03-13 03:33:51,483 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:33:51,509 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0e28c9d19175818252f1c HTTP/1.1" 200 None 2026-03-13 03:33:51,510 - lava_callback - ERROR - Error processing callback for node 69b0e28c9d19175818252f1c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:39200 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56968 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33470 - "GET / HTTP/1.0" 200 OK 2026-03-13 03:34:11,413 - lava_callback - INFO - Received callback for node 69b0e2b09d19175818252f25 INFO: 172.20.0.1:33484 - "POST /node/69b0e2b09d19175818252f25 HTTP/1.0" 202 Accepted 2026-03-13 03:34:11,419 - lava_callback - INFO - Starting async job submit for node 69b0e2b09d19175818252f25 2026-03-13 03:34:11,419 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:34:11,434 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0e2b09d19175818252f25 HTTP/1.1" 200 None 2026-03-13 03:34:11,435 - lava_callback - DEBUG - Processing node 69b0e2b09d19175818252f25: baseline-arm-clabbe 2026-03-13 03:34:11,435 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0e2b09d19175818252f25 2026-03-13 03:34:11,436 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:34:11,463 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0e2b09d19175818252f25 HTTP/1.1" 200 None 2026-03-13 03:34:11,464 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:34:11,478 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afd3a3c9209b146d8e1d68 HTTP/1.1" 200 None 2026-03-13 03:34:11,479 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:34:11,499 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0e2b09d19175818252f25 HTTP/1.1" 200 None 2026-03-13 03:34:11,500 - lava_callback - ERROR - Error processing callback for node 69b0e2b09d19175818252f25: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:33500 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33510 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34952 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39858 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47224 - "GET / HTTP/1.0" 200 OK 2026-03-13 03:34:51,456 - lava_callback - INFO - Received callback for node 69b0e2d49d19175818252f34 INFO: 172.20.0.1:40046 - "POST /node/69b0e2d49d19175818252f34 HTTP/1.0" 202 Accepted 2026-03-13 03:34:51,462 - lava_callback - INFO - Starting async job submit for node 69b0e2d49d19175818252f34 2026-03-13 03:34:51,463 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:34:51,487 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0e2d49d19175818252f34 HTTP/1.1" 200 None 2026-03-13 03:34:51,488 - lava_callback - DEBUG - Processing node 69b0e2d49d19175818252f34: baseline-arm-clabbe 2026-03-13 03:34:51,488 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0e2d49d19175818252f34 2026-03-13 03:34:51,488 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:34:51,501 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0e2d49d19175818252f34 HTTP/1.1" 200 None 2026-03-13 03:34:51,502 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:34:51,516 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afd3a3c9209b146d8e1d68 HTTP/1.1" 200 None 2026-03-13 03:34:51,518 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:34:51,558 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0e2d49d19175818252f34 HTTP/1.1" 200 None 2026-03-13 03:34:51,564 - lava_callback - ERROR - Error processing callback for node 69b0e2d49d19175818252f34: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:40062 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56566 - "GET / HTTP/1.0" 200 OK 2026-03-13 03:35:10,268 - lava_callback - INFO - Received callback for node 69b378a9974c91e64560f72f INFO: 172.20.0.1:53926 - "POST /node/69b378a9974c91e64560f72f HTTP/1.0" 202 Accepted 2026-03-13 03:35:10,274 - lava_callback - INFO - Starting async job submit for node 69b378a9974c91e64560f72f 2026-03-13 03:35:10,332 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:35:10,346 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b378a9974c91e64560f72f HTTP/1.1" 200 None 2026-03-13 03:35:10,347 - lava_callback - DEBUG - Processing node 69b378a9974c91e64560f72f: ltp-smoke INFO: 172.20.0.1:53936 - "GET / HTTP/1.0" 200 OK 2026-03-13 03:35:10,835 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 03:35:11,000 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 03:35:11,002 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoke-69b378a9974c91e64560f72f/log.txt.gz 2026-03-13 03:35:11,328 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 03:35:11,572 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 03:35:11,574 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-smoke-69b378a9974c91e64560f72f/lava-logs.yaml 2026-03-13 03:35:11,585 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 03:35:11,749 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 03:35:11,750 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoke-69b378a9974c91e64560f72f/lava_callback.json.gz 2026-03-13 03:35:11,869 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:35:11,884 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b378a9974c91e64560f72f HTTP/1.1" 200 None 2026-03-13 03:35:11,885 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:35:11,898 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3593e974c91e64560e4a3 HTTP/1.1" 200 None 2026-03-13 03:35:11,900 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:35:11,933 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b378a9974c91e64560f72f HTTP/1.1" 200 None 2026-03-13 03:35:12,062 - lava_callback - INFO - Completed processing callback for node 69b378a9974c91e64560f72f INFO: 172.20.0.1:53944 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-13 03:35:16,148 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:35:16,164 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:53956 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37092 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48566 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50156 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50188 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34450 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47274 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47290 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47292 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49622 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53090 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51354 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33926 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45126 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39638 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39646 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39658 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47116 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49736 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59122 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36974 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45852 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56870 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56884 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56896 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48076 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39770 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60416 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48282 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54722 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39864 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39866 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39874 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35724 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40784 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48682 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57688 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44032 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36338 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36354 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36370 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38816 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41912 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51872 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48408 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54008 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49404 - "GET / HTTP/1.0" 200 OK 2026-03-13 03:41:12,947 - lava_callback - INFO - Received callback for node 69b3684c974c91e64560f0d9 INFO: 172.20.0.1:49414 - "POST /node/69b3684c974c91e64560f0d9 HTTP/1.0" 202 Accepted 2026-03-13 03:41:12,993 - lava_callback - INFO - Starting async job submit for node 69b3684c974c91e64560f0d9 2026-03-13 03:41:13,017 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:41:13,040 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3684c974c91e64560f0d9 HTTP/1.1" 200 None 2026-03-13 03:41:13,041 - lava_callback - DEBUG - Processing node 69b3684c974c91e64560f0d9: baseline-arm-pengutronix 2026-03-13 03:41:13,151 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 03:41:13,315 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 03:41:13,317 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69b3684c974c91e64560f0d9/log.txt.gz 2026-03-13 03:41:13,374 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 03:41:13,557 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 03:41:13,559 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69b3684c974c91e64560f0d9/lava-logs.yaml 2026-03-13 03:41:13,562 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 03:41:13,701 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 03:41:13,703 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69b3684c974c91e64560f0d9/lava_callback.json.gz DEBUG: Setup failed for 69b3684c974c91e64560f0d9. Transitting job node result: incomplete -> fail 2026-03-13 03:41:13,752 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:41:13,789 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3684c974c91e64560f0d9 HTTP/1.1" 200 None 2026-03-13 03:41:13,791 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:41:13,804 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35934974c91e64560e49d HTTP/1.1" 200 None 2026-03-13 03:41:13,806 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:41:13,829 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b3684c974c91e64560f0d9 HTTP/1.1" 200 None 2026-03-13 03:41:13,935 - lava_callback - INFO - Completed processing callback for node 69b3684c974c91e64560f0d9 INFO: 172.20.0.1:49430 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-13 03:41:16,167 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:41:16,183 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:49432 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56452 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35420 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34830 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46264 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51458 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44500 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44506 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44518 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57858 - "GET / HTTP/1.0" 200 OK 2026-03-13 03:42:30,511 - lava_callback - INFO - Received callback for node 69b363cf974c91e64560ed67 INFO: 172.20.0.1:52578 - "POST /node/69b363cf974c91e64560ed67 HTTP/1.0" 202 Accepted 2026-03-13 03:42:30,517 - lava_callback - INFO - Starting async job submit for node 69b363cf974c91e64560ed67 2026-03-13 03:42:30,540 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:42:30,565 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b363cf974c91e64560ed67 HTTP/1.1" 200 None 2026-03-13 03:42:30,566 - lava_callback - DEBUG - Processing node 69b363cf974c91e64560ed67: baseline-arm64-broonie 2026-03-13 03:42:30,805 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 03:42:30,961 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 03:42:30,963 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69b363cf974c91e64560ed67/log.txt.gz 2026-03-13 03:42:31,108 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 03:42:31,318 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 03:42:31,320 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69b363cf974c91e64560ed67/lava-logs.yaml 2026-03-13 03:42:31,327 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 03:42:31,493 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 03:42:31,495 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69b363cf974c91e64560ed67/lava_callback.json.gz 2026-03-13 03:42:31,518 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:42:31,532 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b363cf974c91e64560ed67 HTTP/1.1" 200 None 2026-03-13 03:42:31,533 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:42:31,548 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3594d974c91e64560e4b2 HTTP/1.1" 200 None 2026-03-13 03:42:31,550 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:42:31,582 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b363cf974c91e64560ed67 HTTP/1.1" 200 None 2026-03-13 03:42:31,587 - lava_callback - INFO - Completed processing callback for node 69b363cf974c91e64560ed67 INFO: 172.20.0.1:52590 - "GET / HTTP/1.0" 200 OK 2026-03-13 03:42:46,186 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:42:46,227 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:45426 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42746 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59370 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35320 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35330 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35336 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57830 - "GET / HTTP/1.0" 200 OK 2026-03-13 03:43:26,601 - lava_callback - INFO - Received callback for node 69b38326974c91e64560f84f INFO: 172.20.0.1:57838 - "POST /node/69b38326974c91e64560f84f HTTP/1.0" 202 Accepted 2026-03-13 03:43:26,608 - lava_callback - INFO - Starting async job submit for node 69b38326974c91e64560f84f 2026-03-13 03:43:26,652 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:43:26,667 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b38326974c91e64560f84f HTTP/1.1" 200 None 2026-03-13 03:43:26,668 - lava_callback - DEBUG - Processing node 69b38326974c91e64560f84f: baseline-nfs-arm64 2026-03-13 03:43:27,736 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 03:43:27,932 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 03:43:27,934 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b38326974c91e64560f84f/log.txt.gz 2026-03-13 03:43:28,570 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 03:43:28,799 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 03:43:28,802 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b38326974c91e64560f84f/lava-logs.yaml 2026-03-13 03:43:28,824 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 03:43:28,993 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 03:43:28,995 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b38326974c91e64560f84f/lava_callback.json.gz DEBUG: Setup failed for 69b38326974c91e64560f84f. Transitting job node result: incomplete -> fail 2026-03-13 03:43:29,187 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:43:29,216 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b38326974c91e64560f84f HTTP/1.1" 200 None 2026-03-13 03:43:29,217 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:43:29,235 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3594d974c91e64560e4b2 HTTP/1.1" 200 None 2026-03-13 03:43:29,237 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:43:29,264 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b38326974c91e64560f84f HTTP/1.1" 200 None 2026-03-13 03:43:29,360 - lava_callback - INFO - Completed processing callback for node 69b38326974c91e64560f84f INFO: 172.20.0.1:35544 - "GET / HTTP/1.0" 200 OK 2026-03-13 03:43:46,230 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:43:46,246 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:34464 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57980 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55616 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42990 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42996 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43008 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42938 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58364 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50388 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48670 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43654 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44864 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44870 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44876 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37602 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49404 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45890 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38584 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33622 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35908 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35912 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35914 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57740 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40958 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36806 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56042 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39956 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46820 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46822 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46828 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44362 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44726 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41976 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46952 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55624 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55772 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55788 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55796 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60452 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59424 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53278 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44092 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60622 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47192 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47208 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47216 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51226 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44382 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46768 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47154 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48724 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39694 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39708 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39716 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52022 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33296 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57742 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58374 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48610 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41098 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41114 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41130 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59160 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35876 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35690 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50036 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47276 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35394 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35406 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35410 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50636 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32846 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59026 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56666 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40564 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49618 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49624 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49640 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43414 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49698 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51214 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44924 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50104 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57416 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57420 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57424 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34260 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56230 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34572 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54320 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56752 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43002 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43012 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43024 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39042 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41020 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40906 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54784 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39054 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35982 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35996 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36000 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41152 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57326 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47132 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49078 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49894 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38326 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38338 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38354 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34022 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36922 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53892 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48614 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50222 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39652 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39666 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39672 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53776 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43996 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46080 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43326 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60246 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32916 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32926 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:32936 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55256 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51380 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59922 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49224 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35860 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37756 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37760 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37764 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37940 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51598 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46292 - "GET / HTTP/1.0" 200 OK 2026-03-13 04:00:53,245 - lava_callback - INFO - Received callback for node 69b38767974c91e64560f8bc INFO: 172.20.0.1:40906 - "POST /node/69b38767974c91e64560f8bc HTTP/1.0" 202 Accepted 2026-03-13 04:00:53,293 - lava_callback - INFO - Starting async job submit for node 69b38767974c91e64560f8bc 2026-03-13 04:00:53,317 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:00:53,332 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b38767974c91e64560f8bc HTTP/1.1" 200 None 2026-03-13 04:00:53,333 - lava_callback - DEBUG - Processing node 69b38767974c91e64560f8bc: baseline-arm-pengutronix 2026-03-13 04:00:53,440 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 04:00:53,580 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 04:00:53,582 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69b38767974c91e64560f8bc/log.txt.gz 2026-03-13 04:00:53,641 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 04:00:53,837 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 04:00:53,839 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69b38767974c91e64560f8bc/lava-logs.yaml 2026-03-13 04:00:53,842 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 04:00:53,989 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 04:00:53,991 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69b38767974c91e64560f8bc/lava_callback.json.gz DEBUG: Setup failed for 69b38767974c91e64560f8bc. Transitting job node result: incomplete -> fail 2026-03-13 04:00:54,041 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:00:54,055 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b38767974c91e64560f8bc HTTP/1.1" 200 None 2026-03-13 04:00:54,057 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:00:54,070 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35934974c91e64560e49d HTTP/1.1" 200 None 2026-03-13 04:00:54,071 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:00:54,098 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b38767974c91e64560f8bc HTTP/1.1" 200 None 2026-03-13 04:00:54,161 - lava_callback - INFO - Completed processing callback for node 69b38767974c91e64560f8bc INFO: 172.20.0.1:40912 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53796 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39796 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39812 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-13 04:01:16,251 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:01:16,275 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:39816 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43458 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45574 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54194 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46754 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45446 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47516 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47524 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47540 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36316 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54096 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45978 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51044 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47246 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48554 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48564 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48570 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37246 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38554 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53356 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42498 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60062 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53192 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53194 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53202 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55198 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58336 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52614 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44582 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43210 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35252 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35260 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35272 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37298 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53394 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58978 - "GET / HTTP/1.0" 200 OK 2026-03-13 04:05:52,701 - lava_callback - INFO - Received callback for node 69b0ea109d19175818252fd9 INFO: 172.20.0.1:55928 - "POST /node/69b0ea109d19175818252fd9 HTTP/1.0" 202 Accepted 2026-03-13 04:05:52,706 - lava_callback - INFO - Starting async job submit for node 69b0ea109d19175818252fd9 2026-03-13 04:05:52,707 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:05:52,721 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0ea109d19175818252fd9 HTTP/1.1" 200 None 2026-03-13 04:05:52,722 - lava_callback - DEBUG - Processing node 69b0ea109d19175818252fd9: baseline-arm-clabbe 2026-03-13 04:05:52,722 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0ea109d19175818252fd9 2026-03-13 04:05:52,723 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:05:52,735 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0ea109d19175818252fd9 HTTP/1.1" 200 None 2026-03-13 04:05:52,737 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:05:52,750 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afd826ae557f2600f0f036 HTTP/1.1" 200 None 2026-03-13 04:05:52,751 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:05:52,775 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0ea109d19175818252fd9 HTTP/1.1" 200 None 2026-03-13 04:05:52,779 - lava_callback - ERROR - Error processing callback for node 69b0ea109d19175818252fd9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:55930 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40458 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46954 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46964 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46980 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54878 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55874 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42884 - "GET / HTTP/1.0" 200 OK 2026-03-13 04:06:52,722 - lava_callback - INFO - Received callback for node 69b0ea4c9d19175818252fec INFO: 172.20.0.1:57136 - "POST /node/69b0ea4c9d19175818252fec HTTP/1.0" 202 Accepted 2026-03-13 04:06:52,727 - lava_callback - INFO - Starting async job submit for node 69b0ea4c9d19175818252fec 2026-03-13 04:06:52,728 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:06:52,743 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0ea4c9d19175818252fec HTTP/1.1" 200 None 2026-03-13 04:06:52,743 - lava_callback - DEBUG - Processing node 69b0ea4c9d19175818252fec: baseline-arm-clabbe 2026-03-13 04:06:52,744 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0ea4c9d19175818252fec 2026-03-13 04:06:52,744 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:06:52,775 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0ea4c9d19175818252fec HTTP/1.1" 200 None 2026-03-13 04:06:52,776 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:06:52,789 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afd826ae557f2600f0f036 HTTP/1.1" 200 None 2026-03-13 04:06:52,791 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:06:52,821 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0ea4c9d19175818252fec HTTP/1.1" 200 None 2026-03-13 04:06:52,823 - lava_callback - ERROR - Error processing callback for node 69b0ea4c9d19175818252fec: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:57148 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41536 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55408 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55414 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55430 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58892 - "GET / HTTP/1.0" 200 OK 2026-03-13 04:07:32,807 - lava_callback - INFO - Received callback for node 69b0ea719d19175818252ffa INFO: 172.20.0.1:51044 - "POST /node/69b0ea719d19175818252ffa HTTP/1.0" 202 Accepted 2026-03-13 04:07:32,812 - lava_callback - INFO - Starting async job submit for node 69b0ea719d19175818252ffa 2026-03-13 04:07:32,813 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:07:32,841 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0ea719d19175818252ffa HTTP/1.1" 200 None 2026-03-13 04:07:32,842 - lava_callback - DEBUG - Processing node 69b0ea719d19175818252ffa: baseline-arm-clabbe 2026-03-13 04:07:32,843 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0ea719d19175818252ffa 2026-03-13 04:07:32,843 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:07:32,859 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0ea719d19175818252ffa HTTP/1.1" 200 None 2026-03-13 04:07:32,860 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:07:32,874 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afd826ae557f2600f0f036 HTTP/1.1" 200 None 2026-03-13 04:07:32,876 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:07:32,896 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0ea719d19175818252ffa HTTP/1.1" 200 None 2026-03-13 04:07:32,898 - lava_callback - ERROR - Error processing callback for node 69b0ea719d19175818252ffa: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:51060 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50574 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49832 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57906 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50594 - "GET / HTTP/1.0" 200 OK 2026-03-13 04:08:12,857 - lava_callback - INFO - Received callback for node 69b0ea989d1917581825300a INFO: 172.20.0.1:50610 - "POST /node/69b0ea989d1917581825300a HTTP/1.0" 202 Accepted 2026-03-13 04:08:12,863 - lava_callback - INFO - Starting async job submit for node 69b0ea989d1917581825300a 2026-03-13 04:08:12,864 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:08:12,878 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0ea989d1917581825300a HTTP/1.1" 200 None 2026-03-13 04:08:12,879 - lava_callback - DEBUG - Processing node 69b0ea989d1917581825300a: baseline-arm-clabbe 2026-03-13 04:08:12,879 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0ea989d1917581825300a 2026-03-13 04:08:12,880 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:08:12,893 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0ea989d1917581825300a HTTP/1.1" 200 None 2026-03-13 04:08:12,894 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:08:12,908 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afd826ae557f2600f0f036 HTTP/1.1" 200 None 2026-03-13 04:08:12,909 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:08:12,929 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0ea989d1917581825300a HTTP/1.1" 200 None 2026-03-13 04:08:12,930 - lava_callback - ERROR - Error processing callback for node 69b0ea989d1917581825300a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:50614 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50628 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37640 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50500 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39500 - "GET / HTTP/1.0" 200 OK 2026-03-13 04:08:52,833 - lava_callback - INFO - Received callback for node 69b0eac59d1917581825301d INFO: 172.20.0.1:46092 - "POST /node/69b0eac59d1917581825301d HTTP/1.0" 202 Accepted 2026-03-13 04:08:52,839 - lava_callback - INFO - Starting async job submit for node 69b0eac59d1917581825301d 2026-03-13 04:08:52,839 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:08:52,854 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0eac59d1917581825301d HTTP/1.1" 200 None 2026-03-13 04:08:52,854 - lava_callback - DEBUG - Processing node 69b0eac59d1917581825301d: baseline-arm-clabbe 2026-03-13 04:08:52,855 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0eac59d1917581825301d 2026-03-13 04:08:52,855 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:08:52,869 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0eac59d1917581825301d HTTP/1.1" 200 None 2026-03-13 04:08:52,871 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:08:52,884 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afd826ae557f2600f0f036 HTTP/1.1" 200 None 2026-03-13 04:08:52,885 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:08:52,909 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0eac59d1917581825301d HTTP/1.1" 200 None 2026-03-13 04:08:52,912 - lava_callback - ERROR - Error processing callback for node 69b0eac59d1917581825301d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:46108 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51270 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40986 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40988 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40994 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46220 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35358 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49124 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40194 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36400 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60358 - "GET / HTTP/1.0" 200 OK 2026-03-13 04:10:11,317 - lava_callback - INFO - Received callback for node 69b385f1974c91e64560f8a1 INFO: 172.20.0.1:60372 - "POST /node/69b385f1974c91e64560f8a1 HTTP/1.0" 202 Accepted 2026-03-13 04:10:11,324 - lava_callback - INFO - Starting async job submit for node 69b385f1974c91e64560f8a1 2026-03-13 04:10:11,393 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:10:11,406 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b385f1974c91e64560f8a1 HTTP/1.1" 200 None 2026-03-13 04:10:11,407 - lava_callback - DEBUG - Processing node 69b385f1974c91e64560f8a1: ltp-smoke INFO: 172.20.0.1:60382 - "GET / HTTP/1.0" 200 OK 2026-03-13 04:10:11,911 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 04:10:12,072 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 04:10:12,074 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoke-69b385f1974c91e64560f8a1/log.txt.gz 2026-03-13 04:10:12,358 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 04:10:12,635 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 04:10:12,637 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-smoke-69b385f1974c91e64560f8a1/lava-logs.yaml 2026-03-13 04:10:12,651 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 04:10:12,870 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 04:10:12,872 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoke-69b385f1974c91e64560f8a1/lava_callback.json.gz 2026-03-13 04:10:12,991 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:10:13,006 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b385f1974c91e64560f8a1 HTTP/1.1" 200 None 2026-03-13 04:10:13,007 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:10:13,021 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3593e974c91e64560e4a3 HTTP/1.1" 200 None 2026-03-13 04:10:13,022 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:10:13,047 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b385f1974c91e64560f8a1 HTTP/1.1" 200 None 2026-03-13 04:10:13,163 - lava_callback - INFO - Completed processing callback for node 69b385f1974c91e64560f8a1 INFO: 172.20.0.1:60392 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-13 04:10:16,279 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:10:16,296 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:60402 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40320 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37988 - "GET /favicon.ico HTTP/1.0" 404 Not Found INFO: 172.20.0.1:38002 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50602 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49308 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44688 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38104 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38108 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38122 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49640 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52588 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41778 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41436 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50482 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50866 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50872 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50884 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57380 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57192 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46974 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48392 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55870 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47184 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47188 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47194 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35748 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55328 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48206 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44094 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33386 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33146 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33152 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33156 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36186 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37682 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58546 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36666 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32882 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48300 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48314 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48326 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47858 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55896 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37010 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34118 - "GET / HTTP/1.0" 200 OK 2026-03-13 04:15:56,820 - lava_callback - INFO - Received callback for node 69b363cc974c91e64560ed61 INFO: 172.20.0.1:34126 - "POST /node/69b363cc974c91e64560ed61 HTTP/1.0" 202 Accepted 2026-03-13 04:15:56,826 - lava_callback - INFO - Starting async job submit for node 69b363cc974c91e64560ed61 2026-03-13 04:15:56,848 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:15:56,890 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b363cc974c91e64560ed61 HTTP/1.1" 200 None 2026-03-13 04:15:56,890 - lava_callback - DEBUG - Processing node 69b363cc974c91e64560ed61: baseline-arm64-broonie 2026-03-13 04:15:57,169 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 04:15:57,319 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 04:15:57,321 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69b363cc974c91e64560ed61/log.txt.gz 2026-03-13 04:15:57,482 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 04:15:57,756 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 04:15:57,759 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69b363cc974c91e64560ed61/lava-logs.yaml 2026-03-13 04:15:57,766 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 04:15:57,944 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 04:15:57,947 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69b363cc974c91e64560ed61/lava_callback.json.gz 2026-03-13 04:15:57,969 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:15:57,995 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b363cc974c91e64560ed61 HTTP/1.1" 200 None 2026-03-13 04:15:57,997 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:15:58,034 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3594d974c91e64560e4b2 HTTP/1.1" 200 None 2026-03-13 04:15:58,036 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:15:58,068 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b363cc974c91e64560ed61 HTTP/1.1" 200 None 2026-03-13 04:15:58,069 - lava_callback - INFO - Completed processing callback for node 69b363cc974c91e64560ed61 INFO: 172.20.0.1:40224 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54046 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54056 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-13 04:16:16,299 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:16:16,324 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:54072 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44092 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44974 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46988 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60254 - "GET / HTTP/1.0" 200 OK 2026-03-13 04:16:57,260 - lava_callback - INFO - Received callback for node 69b0ecaa9d19175818253058 INFO: 172.20.0.1:60262 - "POST /node/69b0ecaa9d19175818253058 HTTP/1.0" 202 Accepted 2026-03-13 04:16:57,265 - lava_callback - INFO - Starting async job submit for node 69b0ecaa9d19175818253058 2026-03-13 04:16:57,266 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:16:57,283 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0ecaa9d19175818253058 HTTP/1.1" 200 None 2026-03-13 04:16:57,284 - lava_callback - DEBUG - Processing node 69b0ecaa9d19175818253058: baseline-arm64-baylibre 2026-03-13 04:16:57,285 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0ecaa9d19175818253058 2026-03-13 04:16:57,285 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:16:57,298 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0ecaa9d19175818253058 HTTP/1.1" 200 None 2026-03-13 04:16:57,299 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:16:57,321 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afd839ae557f2600f0f045 HTTP/1.1" 200 None 2026-03-13 04:16:57,322 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:16:57,352 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0ecaa9d19175818253058 HTTP/1.1" 200 None 2026-03-13 04:16:57,357 - lava_callback - ERROR - Error processing callback for node 69b0ecaa9d19175818253058: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:59496 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45238 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45244 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45260 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50618 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38856 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39042 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60788 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38664 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51014 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51028 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51030 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37274 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45544 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40554 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44832 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37054 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37742 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37756 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37758 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50194 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51008 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52804 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49834 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42238 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40380 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40394 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40402 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40178 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51962 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42082 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58372 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46116 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48220 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48236 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48246 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53376 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42402 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48942 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60158 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60936 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59932 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59942 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59954 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43660 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55680 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58700 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59394 - "GET / HTTP/1.0" 200 OK 2026-03-13 04:22:57,543 - lava_callback - INFO - Received callback for node 69b0ee169d19175818253066 INFO: 172.20.0.1:59404 - "POST /node/69b0ee169d19175818253066 HTTP/1.0" 202 Accepted 2026-03-13 04:22:57,549 - lava_callback - INFO - Starting async job submit for node 69b0ee169d19175818253066 2026-03-13 04:22:57,549 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:22:57,566 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0ee169d19175818253066 HTTP/1.1" 200 None 2026-03-13 04:22:57,567 - lava_callback - DEBUG - Processing node 69b0ee169d19175818253066: baseline-arm64-baylibre 2026-03-13 04:22:57,567 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0ee169d19175818253066 2026-03-13 04:22:57,567 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:22:57,589 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0ee169d19175818253066 HTTP/1.1" 200 None 2026-03-13 04:22:57,591 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:22:57,608 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afd839ae557f2600f0f045 HTTP/1.1" 200 None 2026-03-13 04:22:57,610 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:22:57,630 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0ee169d19175818253066 HTTP/1.1" 200 None 2026-03-13 04:22:57,635 - lava_callback - ERROR - Error processing callback for node 69b0ee169d19175818253066: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:32954 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60488 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60494 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60496 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33870 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48792 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43040 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37684 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39150 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39714 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39728 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39742 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55124 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39042 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46218 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46448 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39400 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58772 - "GET / HTTP/1.0" 200 OK 2026-03-13 04:25:13,496 - lava_callback - INFO - Received callback for node 69b0ee949d19175818253075 INFO: 172.20.0.1:58788 - "POST /node/69b0ee949d19175818253075 HTTP/1.0" 202 Accepted 2026-03-13 04:25:13,501 - lava_callback - INFO - Starting async job submit for node 69b0ee949d19175818253075 2026-03-13 04:25:13,502 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:25:13,517 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0ee949d19175818253075 HTTP/1.1" 200 None 2026-03-13 04:25:13,518 - lava_callback - DEBUG - Processing node 69b0ee949d19175818253075: baseline-arm64-clabbe 2026-03-13 04:25:13,518 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0ee949d19175818253075 2026-03-13 04:25:13,519 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:25:13,540 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0ee949d19175818253075 HTTP/1.1" 200 None 2026-03-13 04:25:13,542 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:25:13,555 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afd839ae557f2600f0f045 HTTP/1.1" 200 None 2026-03-13 04:25:13,556 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:25:13,577 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0ee949d19175818253075 HTTP/1.1" 200 None 2026-03-13 04:25:13,579 - lava_callback - ERROR - Error processing callback for node 69b0ee949d19175818253075: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:58796 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58808 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49662 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57042 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57522 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50694 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33892 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40176 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40180 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40182 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51926 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35326 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47566 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55132 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37660 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57348 - "GET / HTTP/1.0" 200 OK 2026-03-13 04:27:13,609 - lava_callback - INFO - Received callback for node 69b0ef0e9d19175818253084 INFO: 172.20.0.1:57358 - "POST /node/69b0ef0e9d19175818253084 HTTP/1.0" 202 Accepted 2026-03-13 04:27:13,614 - lava_callback - INFO - Starting async job submit for node 69b0ef0e9d19175818253084 2026-03-13 04:27:13,615 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:27:13,631 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0ef0e9d19175818253084 HTTP/1.1" 200 None 2026-03-13 04:27:13,632 - lava_callback - DEBUG - Processing node 69b0ef0e9d19175818253084: baseline-arm64-clabbe 2026-03-13 04:27:13,632 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0ef0e9d19175818253084 2026-03-13 04:27:13,632 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:27:13,677 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0ef0e9d19175818253084 HTTP/1.1" 200 None 2026-03-13 04:27:13,679 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:27:13,697 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afd839ae557f2600f0f045 HTTP/1.1" 200 None 2026-03-13 04:27:13,699 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:27:13,719 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0ef0e9d19175818253084 HTTP/1.1" 200 None 2026-03-13 04:27:13,721 - lava_callback - ERROR - Error processing callback for node 69b0ef0e9d19175818253084: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:57372 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57380 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45986 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37026 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37930 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37628 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50586 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49376 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49390 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49394 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50192 - "GET / HTTP/1.0" 200 OK 2026-03-13 04:28:33,676 - lava_callback - INFO - Received callback for node 69b0ef5f9d19175818253095 INFO: 172.20.0.1:46074 - "POST /node/69b0ef5f9d19175818253095 HTTP/1.0" 202 Accepted 2026-03-13 04:28:33,681 - lava_callback - INFO - Starting async job submit for node 69b0ef5f9d19175818253095 2026-03-13 04:28:33,682 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:28:33,702 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0ef5f9d19175818253095 HTTP/1.1" 200 None 2026-03-13 04:28:33,702 - lava_callback - DEBUG - Processing node 69b0ef5f9d19175818253095: baseline-arm64-clabbe 2026-03-13 04:28:33,703 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0ef5f9d19175818253095 2026-03-13 04:28:33,703 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:28:33,716 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0ef5f9d19175818253095 HTTP/1.1" 200 None 2026-03-13 04:28:33,718 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:28:33,730 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afd839ae557f2600f0f045 HTTP/1.1" 200 None 2026-03-13 04:28:33,732 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:28:33,760 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0ef5f9d19175818253095 HTTP/1.1" 200 None 2026-03-13 04:28:33,762 - lava_callback - ERROR - Error processing callback for node 69b0ef5f9d19175818253095: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:46086 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48950 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35108 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56844 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36482 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36498 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36510 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44460 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44648 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49762 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45640 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53494 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37046 - "GET / HTTP/1.0" 200 OK 2026-03-13 04:30:13,739 - lava_callback - INFO - Received callback for node 69b0efca9d191758182530aa INFO: 172.20.0.1:37062 - "POST /node/69b0efca9d191758182530aa HTTP/1.0" 202 Accepted 2026-03-13 04:30:13,745 - lava_callback - INFO - Starting async job submit for node 69b0efca9d191758182530aa 2026-03-13 04:30:13,746 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:30:13,775 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0efca9d191758182530aa HTTP/1.1" 200 None 2026-03-13 04:30:13,776 - lava_callback - DEBUG - Processing node 69b0efca9d191758182530aa: baseline-arm64-clabbe 2026-03-13 04:30:13,776 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0efca9d191758182530aa 2026-03-13 04:30:13,777 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:30:13,790 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0efca9d191758182530aa HTTP/1.1" 200 None 2026-03-13 04:30:13,791 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:30:13,804 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afd839ae557f2600f0f045 HTTP/1.1" 200 None 2026-03-13 04:30:13,806 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:30:13,835 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0efca9d191758182530aa HTTP/1.1" 200 None 2026-03-13 04:30:13,836 - lava_callback - ERROR - Error processing callback for node 69b0efca9d191758182530aa: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:37072 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37088 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36876 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43530 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56780 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60728 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46052 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36128 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36132 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36140 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49348 - "GET / HTTP/1.0" 200 OK 2026-03-13 04:31:33,804 - lava_callback - INFO - Received callback for node 69b0f0219d191758182530b2 INFO: 172.20.0.1:34384 - "POST /node/69b0f0219d191758182530b2 HTTP/1.0" 202 Accepted 2026-03-13 04:31:33,809 - lava_callback - INFO - Starting async job submit for node 69b0f0219d191758182530b2 2026-03-13 04:31:33,810 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:31:33,825 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0f0219d191758182530b2 HTTP/1.1" 200 None 2026-03-13 04:31:33,826 - lava_callback - DEBUG - Processing node 69b0f0219d191758182530b2: baseline-arm64-clabbe 2026-03-13 04:31:33,826 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0f0219d191758182530b2 2026-03-13 04:31:33,826 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:31:33,846 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0f0219d191758182530b2 HTTP/1.1" 200 None 2026-03-13 04:31:33,847 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:31:33,861 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afd839ae557f2600f0f045 HTTP/1.1" 200 None 2026-03-13 04:31:33,863 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:31:33,882 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0f0219d191758182530b2 HTTP/1.1" 200 None 2026-03-13 04:31:33,883 - lava_callback - ERROR - Error processing callback for node 69b0f0219d191758182530b2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:34396 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42308 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56900 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33744 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38980 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38984 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38990 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44054 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54600 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40648 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42460 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58474 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43268 - "GET / HTTP/1.0" 200 OK 2026-03-13 04:33:13,845 - lava_callback - INFO - Received callback for node 69b0f07e9d191758182530c8 INFO: 172.20.0.1:43284 - "POST /node/69b0f07e9d191758182530c8 HTTP/1.0" 202 Accepted 2026-03-13 04:33:13,850 - lava_callback - INFO - Starting async job submit for node 69b0f07e9d191758182530c8 2026-03-13 04:33:13,851 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:33:13,866 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0f07e9d191758182530c8 HTTP/1.1" 200 None 2026-03-13 04:33:13,866 - lava_callback - DEBUG - Processing node 69b0f07e9d191758182530c8: baseline-arm64-clabbe 2026-03-13 04:33:13,867 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0f07e9d191758182530c8 2026-03-13 04:33:13,867 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:33:13,905 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0f07e9d191758182530c8 HTTP/1.1" 200 None 2026-03-13 04:33:13,906 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:33:13,930 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afd839ae557f2600f0f045 HTTP/1.1" 200 None 2026-03-13 04:33:13,932 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:33:13,957 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0f07e9d191758182530c8 HTTP/1.1" 200 None 2026-03-13 04:33:13,958 - lava_callback - ERROR - Error processing callback for node 69b0f07e9d191758182530c8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:43292 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43298 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54002 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36534 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45786 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49226 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37804 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55560 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55570 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55574 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41784 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38368 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34844 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40496 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48776 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45100 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45110 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45114 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35690 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46176 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38906 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54660 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54654 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56358 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56366 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56380 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45572 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44520 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40234 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47194 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41048 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45010 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45018 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45024 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54182 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42796 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55394 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53910 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56372 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54966 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54972 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54976 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53590 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60770 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58946 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50810 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36670 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34482 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34490 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34492 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38436 - "GET / HTTP/1.0" 200 OK 2026-03-13 04:39:35,300 - lava_callback - INFO - Received callback for node 69b38c03974c91e64560f949 INFO: 172.20.0.1:50474 - "POST /node/69b38c03974c91e64560f949 HTTP/1.0" 202 Accepted 2026-03-13 04:39:35,349 - lava_callback - INFO - Starting async job submit for node 69b38c03974c91e64560f949 2026-03-13 04:39:35,396 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:39:35,444 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b38c03974c91e64560f949 HTTP/1.1" 200 None 2026-03-13 04:39:35,445 - lava_callback - DEBUG - Processing node 69b38c03974c91e64560f949: baseline-arm-pengutronix 2026-03-13 04:39:35,562 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 04:39:35,726 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 04:39:35,728 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69b38c03974c91e64560f949/log.txt.gz 2026-03-13 04:39:35,786 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 04:39:35,992 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 04:39:35,994 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69b38c03974c91e64560f949/lava-logs.yaml 2026-03-13 04:39:35,998 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 04:39:36,131 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 04:39:36,133 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69b38c03974c91e64560f949/lava_callback.json.gz DEBUG: Setup failed for 69b38c03974c91e64560f949. Transitting job node result: incomplete -> fail 2026-03-13 04:39:36,182 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:39:36,197 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b38c03974c91e64560f949 HTTP/1.1" 200 None 2026-03-13 04:39:36,198 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:39:36,231 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35934974c91e64560e49d HTTP/1.1" 200 None 2026-03-13 04:39:36,233 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:39:36,257 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b38c03974c91e64560f949 HTTP/1.1" 200 None 2026-03-13 04:39:36,302 - lava_callback - INFO - Completed processing callback for node 69b38c03974c91e64560f949 INFO: 172.20.0.1:50488 - "GET / HTTP/1.0" 200 OK 2026-03-13 04:39:46,330 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:39:46,370 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:47098 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59962 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52530 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34754 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34762 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34776 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45148 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43884 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48606 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38236 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58036 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46956 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46962 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46964 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45648 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35272 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56100 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40668 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41752 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34218 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34226 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34240 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60980 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43486 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53754 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45904 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43418 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43358 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43370 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43380 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46194 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55672 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45628 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60422 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39024 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54160 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54170 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54184 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39946 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42340 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40594 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50636 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34072 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51624 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51636 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51650 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52146 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54566 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34088 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57422 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50284 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37056 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37064 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37078 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42764 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38558 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57922 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41644 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34506 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40242 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40250 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40260 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57912 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50964 - "GET / HTTP/1.0" 200 OK 2026-03-13 04:47:39,068 - lava_callback - INFO - Received callback for node 69b0f3dd9d191758182530e2 INFO: 172.20.0.1:50974 - "POST /node/69b0f3dd9d191758182530e2 HTTP/1.0" 202 Accepted 2026-03-13 04:47:39,073 - lava_callback - INFO - Starting async job submit for node 69b0f3dd9d191758182530e2 2026-03-13 04:47:39,075 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:47:39,094 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0f3dd9d191758182530e2 HTTP/1.1" 200 None 2026-03-13 04:47:39,096 - lava_callback - DEBUG - Processing node 69b0f3dd9d191758182530e2: baseline-arm-baylibre 2026-03-13 04:47:39,096 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0f3dd9d191758182530e2 2026-03-13 04:47:39,097 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:47:39,143 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0f3dd9d191758182530e2 HTTP/1.1" 200 None 2026-03-13 04:47:39,145 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:47:39,159 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ada13b8fc97e5ba07d4d75 HTTP/1.1" 200 None 2026-03-13 04:47:39,161 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:47:39,184 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0f3dd9d191758182530e2 HTTP/1.1" 200 None 2026-03-13 04:47:39,190 - lava_callback - ERROR - Error processing callback for node 69b0f3dd9d191758182530e2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:49130 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37872 - "GET / HTTP/1.0" 200 OK 2026-03-13 04:47:57,711 - lava_callback - INFO - Received callback for node 69b363d3974c91e64560ed6d INFO: 172.20.0.1:37882 - "POST /node/69b363d3974c91e64560ed6d HTTP/1.0" 202 Accepted 2026-03-13 04:47:57,717 - lava_callback - INFO - Starting async job submit for node 69b363d3974c91e64560ed6d 2026-03-13 04:47:57,739 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:47:57,754 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b363d3974c91e64560ed6d HTTP/1.1" 200 None 2026-03-13 04:47:57,755 - lava_callback - DEBUG - Processing node 69b363d3974c91e64560ed6d: baseline-arm64-broonie 2026-03-13 04:47:57,989 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 04:47:58,152 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 04:47:58,154 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69b363d3974c91e64560ed6d/log.txt.gz 2026-03-13 04:47:58,301 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 04:47:58,512 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 04:47:58,514 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69b363d3974c91e64560ed6d/lava-logs.yaml 2026-03-13 04:47:58,520 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 04:47:58,680 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 04:47:58,682 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69b363d3974c91e64560ed6d/lava_callback.json.gz 2026-03-13 04:47:58,705 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:47:58,720 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b363d3974c91e64560ed6d HTTP/1.1" 200 None 2026-03-13 04:47:58,721 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:47:58,735 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3594d974c91e64560e4b2 HTTP/1.1" 200 None 2026-03-13 04:47:58,736 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:47:58,771 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b363d3974c91e64560ed6d HTTP/1.1" 200 None 2026-03-13 04:47:58,772 - lava_callback - INFO - Completed processing callback for node 69b363d3974c91e64560ed6d INFO: 172.20.0.1:50548 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35360 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35372 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-13 04:48:16,373 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:48:16,409 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:35382 - "GET / HTTP/1.0" 200 OK 2026-03-13 04:48:18,967 - lava_callback - INFO - Received callback for node 69b0f4049d191758182530f8 INFO: 172.20.0.1:35384 - "POST /node/69b0f4049d191758182530f8 HTTP/1.0" 202 Accepted 2026-03-13 04:48:18,972 - lava_callback - INFO - Starting async job submit for node 69b0f4049d191758182530f8 2026-03-13 04:48:18,973 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:48:18,993 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0f4049d191758182530f8 HTTP/1.1" 200 None 2026-03-13 04:48:18,994 - lava_callback - DEBUG - Processing node 69b0f4049d191758182530f8: baseline-arm-baylibre 2026-03-13 04:48:18,994 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0f4049d191758182530f8 2026-03-13 04:48:18,995 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:48:19,019 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0f4049d191758182530f8 HTTP/1.1" 200 None 2026-03-13 04:48:19,020 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:48:19,053 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ada13b8fc97e5ba07d4d75 HTTP/1.1" 200 None 2026-03-13 04:48:19,055 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:48:19,075 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0f4049d191758182530f8 HTTP/1.1" 200 None 2026-03-13 04:48:19,079 - lava_callback - ERROR - Error processing callback for node 69b0f4049d191758182530f8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:41260 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56788 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49506 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48294 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59460 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44320 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44328 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44338 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36750 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45728 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47458 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49008 - "GET / HTTP/1.0" 200 OK 2026-03-13 04:49:59,084 - lava_callback - INFO - Received callback for node 69b0f4729d19175818253122 INFO: 172.20.0.1:49014 - "POST /node/69b0f4729d19175818253122 HTTP/1.0" 202 Accepted 2026-03-13 04:49:59,089 - lava_callback - INFO - Starting async job submit for node 69b0f4729d19175818253122 2026-03-13 04:49:59,090 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:49:59,104 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0f4729d19175818253122 HTTP/1.1" 200 None 2026-03-13 04:49:59,105 - lava_callback - DEBUG - Processing node 69b0f4729d19175818253122: baseline-arm-baylibre 2026-03-13 04:49:59,105 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0f4729d19175818253122 2026-03-13 04:49:59,106 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:49:59,130 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0f4729d19175818253122 HTTP/1.1" 200 None 2026-03-13 04:49:59,132 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:49:59,145 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ada13b8fc97e5ba07d4d75 HTTP/1.1" 200 None 2026-03-13 04:49:59,146 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:49:59,172 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0f4729d19175818253122 HTTP/1.1" 200 None 2026-03-13 04:49:59,173 - lava_callback - ERROR - Error processing callback for node 69b0f4729d19175818253122: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:50132 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47166 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47170 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47180 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50740 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49100 - "GET / HTTP/1.0" 200 OK 2026-03-13 04:50:39,077 - lava_callback - INFO - Received callback for node 69b0f4969d19175818253131 INFO: 172.20.0.1:49114 - "POST /node/69b0f4969d19175818253131 HTTP/1.0" 202 Accepted 2026-03-13 04:50:39,082 - lava_callback - INFO - Starting async job submit for node 69b0f4969d19175818253131 2026-03-13 04:50:39,083 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:50:39,097 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0f4969d19175818253131 HTTP/1.1" 200 None 2026-03-13 04:50:39,098 - lava_callback - DEBUG - Processing node 69b0f4969d19175818253131: baseline-arm-baylibre 2026-03-13 04:50:39,098 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0f4969d19175818253131 2026-03-13 04:50:39,098 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:50:39,113 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0f4969d19175818253131 HTTP/1.1" 200 None 2026-03-13 04:50:39,115 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:50:39,138 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ada13b8fc97e5ba07d4d75 HTTP/1.1" 200 None 2026-03-13 04:50:39,140 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:50:39,175 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0f4969d19175818253131 HTTP/1.1" 200 None 2026-03-13 04:50:39,180 - lava_callback - ERROR - Error processing callback for node 69b0f4969d19175818253131: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:53704 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48942 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52394 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46344 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46348 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46358 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54774 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37682 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45922 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58340 - "GET / HTTP/1.0" 200 OK 2026-03-13 04:51:59,205 - lava_callback - INFO - Received callback for node 69b0f4e19d1917581825314e INFO: 172.20.0.1:58352 - "POST /node/69b0f4e19d1917581825314e HTTP/1.0" 202 Accepted 2026-03-13 04:51:59,211 - lava_callback - INFO - Starting async job submit for node 69b0f4e19d1917581825314e 2026-03-13 04:51:59,212 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:51:59,228 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0f4e19d1917581825314e HTTP/1.1" 200 None 2026-03-13 04:51:59,229 - lava_callback - DEBUG - Processing node 69b0f4e19d1917581825314e: baseline-arm-baylibre 2026-03-13 04:51:59,229 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0f4e19d1917581825314e 2026-03-13 04:51:59,230 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:51:59,244 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0f4e19d1917581825314e HTTP/1.1" 200 None 2026-03-13 04:51:59,246 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:51:59,259 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ada13b8fc97e5ba07d4d75 HTTP/1.1" 200 None 2026-03-13 04:51:59,260 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:51:59,282 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0f4e19d1917581825314e HTTP/1.1" 200 None 2026-03-13 04:51:59,283 - lava_callback - ERROR - Error processing callback for node 69b0f4e19d1917581825314e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:50324 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43732 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43742 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43746 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37668 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58774 - "GET / HTTP/1.0" 200 OK 2026-03-13 04:52:39,201 - lava_callback - INFO - Received callback for node 69b0f5069d1917581825315c INFO: 172.20.0.1:58778 - "POST /node/69b0f5069d1917581825315c HTTP/1.0" 202 Accepted 2026-03-13 04:52:39,206 - lava_callback - INFO - Starting async job submit for node 69b0f5069d1917581825315c 2026-03-13 04:52:39,207 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:52:39,232 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0f5069d1917581825315c HTTP/1.1" 200 None 2026-03-13 04:52:39,233 - lava_callback - DEBUG - Processing node 69b0f5069d1917581825315c: baseline-arm-baylibre 2026-03-13 04:52:39,233 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0f5069d1917581825315c 2026-03-13 04:52:39,234 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:52:39,256 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0f5069d1917581825315c HTTP/1.1" 200 None 2026-03-13 04:52:39,257 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:52:39,271 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ada13b8fc97e5ba07d4d75 HTTP/1.1" 200 None 2026-03-13 04:52:39,272 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:52:39,305 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0f5069d1917581825315c HTTP/1.1" 200 None 2026-03-13 04:52:39,315 - lava_callback - ERROR - Error processing callback for node 69b0f5069d1917581825315c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:33314 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44568 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49802 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48404 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48420 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48426 - "GET / HTTP/1.0" 200 OK 2026-03-13 04:53:19,268 - lava_callback - INFO - Received callback for node 69b0f52a9d1917581825316d INFO: 172.20.0.1:48442 - "POST /node/69b0f52a9d1917581825316d HTTP/1.0" 202 Accepted 2026-03-13 04:53:19,273 - lava_callback - INFO - Starting async job submit for node 69b0f52a9d1917581825316d 2026-03-13 04:53:19,274 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:53:19,289 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0f52a9d1917581825316d HTTP/1.1" 200 None 2026-03-13 04:53:19,290 - lava_callback - DEBUG - Processing node 69b0f52a9d1917581825316d: baseline-arm-baylibre 2026-03-13 04:53:19,290 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0f52a9d1917581825316d 2026-03-13 04:53:19,290 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:53:19,303 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0f52a9d1917581825316d HTTP/1.1" 200 None 2026-03-13 04:53:19,305 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:53:19,320 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ada13b8fc97e5ba07d4d75 HTTP/1.1" 200 None 2026-03-13 04:53:19,322 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:53:19,342 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0f52a9d1917581825316d HTTP/1.1" 200 None 2026-03-13 04:53:19,348 - lava_callback - ERROR - Error processing callback for node 69b0f52a9d1917581825316d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:34664 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47014 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40230 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37658 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40954 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58974 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58976 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58980 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52008 - "GET / HTTP/1.0" 200 OK 2026-03-13 04:54:34,749 - lava_callback - INFO - Received callback for node 69b0f5779d1917581825318d INFO: 172.20.0.1:45238 - "POST /node/69b0f5779d1917581825318d HTTP/1.0" 202 Accepted 2026-03-13 04:54:34,755 - lava_callback - INFO - Starting async job submit for node 69b0f5779d1917581825318d 2026-03-13 04:54:34,756 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:54:34,770 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0f5779d1917581825318d HTTP/1.1" 200 None 2026-03-13 04:54:34,771 - lava_callback - DEBUG - Processing node 69b0f5779d1917581825318d: baseline-arm-clabbe 2026-03-13 04:54:34,772 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0f5779d1917581825318d 2026-03-13 04:54:34,772 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:54:34,786 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0f5779d1917581825318d HTTP/1.1" 200 None 2026-03-13 04:54:34,787 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:54:34,800 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ada13b8fc97e5ba07d4d75 HTTP/1.1" 200 None 2026-03-13 04:54:34,801 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:54:34,821 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0f5779d1917581825318d HTTP/1.1" 200 None 2026-03-13 04:54:34,823 - lava_callback - ERROR - Error processing callback for node 69b0f5779d1917581825318d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:45254 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54600 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46624 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48080 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55374 - "GET / HTTP/1.0" 200 OK 2026-03-13 04:55:14,588 - lava_callback - INFO - Received callback for node 69b0f59d9d1917581825319f INFO: 172.20.0.1:55386 - "POST /node/69b0f59d9d1917581825319f HTTP/1.0" 202 Accepted 2026-03-13 04:55:14,593 - lava_callback - INFO - Starting async job submit for node 69b0f59d9d1917581825319f 2026-03-13 04:55:14,594 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:55:14,608 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0f59d9d1917581825319f HTTP/1.1" 200 None 2026-03-13 04:55:14,609 - lava_callback - DEBUG - Processing node 69b0f59d9d1917581825319f: baseline-arm-clabbe 2026-03-13 04:55:14,609 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0f59d9d1917581825319f 2026-03-13 04:55:14,610 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:55:14,622 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0f59d9d1917581825319f HTTP/1.1" 200 None 2026-03-13 04:55:14,624 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:55:14,638 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ada13b8fc97e5ba07d4d75 HTTP/1.1" 200 None 2026-03-13 04:55:14,639 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:55:14,668 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0f59d9d1917581825319f HTTP/1.1" 200 None 2026-03-13 04:55:14,674 - lava_callback - ERROR - Error processing callback for node 69b0f59d9d1917581825319f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:55400 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55412 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59188 - "GET / HTTP/1.0" 200 OK 2026-03-13 04:55:34,743 - lava_callback - INFO - Received callback for node 69b0f5c39d191758182531af INFO: 172.20.0.1:34116 - "POST /node/69b0f5c39d191758182531af HTTP/1.0" 202 Accepted 2026-03-13 04:55:34,749 - lava_callback - INFO - Starting async job submit for node 69b0f5c39d191758182531af 2026-03-13 04:55:34,749 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:55:34,765 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0f5c39d191758182531af HTTP/1.1" 200 None 2026-03-13 04:55:34,766 - lava_callback - DEBUG - Processing node 69b0f5c39d191758182531af: baseline-arm-clabbe 2026-03-13 04:55:34,766 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0f5c39d191758182531af 2026-03-13 04:55:34,767 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:55:34,780 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0f5c39d191758182531af HTTP/1.1" 200 None 2026-03-13 04:55:34,781 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:55:34,806 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ada13b8fc97e5ba07d4d75 HTTP/1.1" 200 None 2026-03-13 04:55:34,807 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:55:34,832 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0f5c39d191758182531af HTTP/1.1" 200 None 2026-03-13 04:55:34,833 - lava_callback - ERROR - Error processing callback for node 69b0f5c39d191758182531af: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:34132 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49254 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35830 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35742 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44666 - "GET / HTTP/1.0" 200 OK 2026-03-13 04:56:14,793 - lava_callback - INFO - Received callback for node 69b0f5e89d191758182531c4 INFO: 172.20.0.1:44678 - "POST /node/69b0f5e89d191758182531c4 HTTP/1.0" 202 Accepted 2026-03-13 04:56:14,798 - lava_callback - INFO - Starting async job submit for node 69b0f5e89d191758182531c4 2026-03-13 04:56:14,799 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:56:14,814 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0f5e89d191758182531c4 HTTP/1.1" 200 None 2026-03-13 04:56:14,815 - lava_callback - DEBUG - Processing node 69b0f5e89d191758182531c4: baseline-arm-clabbe 2026-03-13 04:56:14,815 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0f5e89d191758182531c4 2026-03-13 04:56:14,816 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:56:14,830 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0f5e89d191758182531c4 HTTP/1.1" 200 None 2026-03-13 04:56:14,831 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:56:14,845 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ada13b8fc97e5ba07d4d75 HTTP/1.1" 200 None 2026-03-13 04:56:14,846 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:56:14,870 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0f5e89d191758182531c4 HTTP/1.1" 200 None 2026-03-13 04:56:14,876 - lava_callback - ERROR - Error processing callback for node 69b0f5e89d191758182531c4: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:44682 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44690 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59236 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49792 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37488 - "GET / HTTP/1.0" 200 OK 2026-03-13 04:56:54,803 - lava_callback - INFO - Received callback for node 69b0f60e9d191758182531d6 INFO: 172.20.0.1:60256 - "POST /node/69b0f60e9d191758182531d6 HTTP/1.0" 202 Accepted 2026-03-13 04:56:54,809 - lava_callback - INFO - Starting async job submit for node 69b0f60e9d191758182531d6 2026-03-13 04:56:54,810 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:56:54,824 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0f60e9d191758182531d6 HTTP/1.1" 200 None 2026-03-13 04:56:54,825 - lava_callback - DEBUG - Processing node 69b0f60e9d191758182531d6: baseline-arm-clabbe 2026-03-13 04:56:54,825 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0f60e9d191758182531d6 2026-03-13 04:56:54,826 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:56:54,839 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0f60e9d191758182531d6 HTTP/1.1" 200 None 2026-03-13 04:56:54,840 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:56:54,858 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ada13b8fc97e5ba07d4d75 HTTP/1.1" 200 None 2026-03-13 04:56:54,859 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:56:54,899 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0f60e9d191758182531d6 HTTP/1.1" 200 None 2026-03-13 04:56:54,903 - lava_callback - ERROR - Error processing callback for node 69b0f60e9d191758182531d6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:60272 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40210 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51884 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51886 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51896 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49614 - "GET / HTTP/1.0" 200 OK 2026-03-13 04:57:34,812 - lava_callback - INFO - Received callback for node 69b0f6339d191758182531e7 INFO: 172.20.0.1:49972 - "POST /node/69b0f6339d191758182531e7 HTTP/1.0" 202 Accepted 2026-03-13 04:57:34,817 - lava_callback - INFO - Starting async job submit for node 69b0f6339d191758182531e7 2026-03-13 04:57:34,818 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:57:34,832 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0f6339d191758182531e7 HTTP/1.1" 200 None 2026-03-13 04:57:34,832 - lava_callback - DEBUG - Processing node 69b0f6339d191758182531e7: baseline-arm-clabbe 2026-03-13 04:57:34,833 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0f6339d191758182531e7 2026-03-13 04:57:34,833 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:57:34,847 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0f6339d191758182531e7 HTTP/1.1" 200 None 2026-03-13 04:57:34,848 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:57:34,861 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ada13b8fc97e5ba07d4d75 HTTP/1.1" 200 None 2026-03-13 04:57:34,863 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:57:34,894 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0f6339d191758182531e7 HTTP/1.1" 200 None 2026-03-13 04:57:34,897 - lava_callback - ERROR - Error processing callback for node 69b0f6339d191758182531e7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:49978 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52524 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36810 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49340 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39612 - "GET / HTTP/1.0" 200 OK 2026-03-13 04:58:14,840 - lava_callback - INFO - Received callback for node 69b0f6579d191758182531f9 INFO: 172.20.0.1:39624 - "POST /node/69b0f6579d191758182531f9 HTTP/1.0" 202 Accepted 2026-03-13 04:58:14,845 - lava_callback - INFO - Starting async job submit for node 69b0f6579d191758182531f9 2026-03-13 04:58:14,846 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:58:14,871 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0f6579d191758182531f9 HTTP/1.1" 200 None 2026-03-13 04:58:14,872 - lava_callback - DEBUG - Processing node 69b0f6579d191758182531f9: baseline-arm-clabbe 2026-03-13 04:58:14,872 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0f6579d191758182531f9 2026-03-13 04:58:14,873 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:58:14,895 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0f6579d191758182531f9 HTTP/1.1" 200 None 2026-03-13 04:58:14,896 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:58:14,909 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ada13b8fc97e5ba07d4d75 HTTP/1.1" 200 None 2026-03-13 04:58:14,911 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:58:14,931 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0f6579d191758182531f9 HTTP/1.1" 200 None 2026-03-13 04:58:14,937 - lava_callback - ERROR - Error processing callback for node 69b0f6579d191758182531f9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:39638 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39646 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48540 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57324 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43462 - "GET / HTTP/1.0" 200 OK 2026-03-13 04:58:54,850 - lava_callback - INFO - Received callback for node 69b0f67c9d1917581825320c INFO: 172.20.0.1:52074 - "POST /node/69b0f67c9d1917581825320c HTTP/1.0" 202 Accepted 2026-03-13 04:58:54,855 - lava_callback - INFO - Starting async job submit for node 69b0f67c9d1917581825320c 2026-03-13 04:58:54,856 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:58:54,871 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0f67c9d1917581825320c HTTP/1.1" 200 None 2026-03-13 04:58:54,871 - lava_callback - DEBUG - Processing node 69b0f67c9d1917581825320c: baseline-arm-clabbe 2026-03-13 04:58:54,872 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0f67c9d1917581825320c 2026-03-13 04:58:54,872 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:58:54,885 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0f67c9d1917581825320c HTTP/1.1" 200 None 2026-03-13 04:58:54,886 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:58:54,899 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ada13b8fc97e5ba07d4d75 HTTP/1.1" 200 None 2026-03-13 04:58:54,901 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:58:54,921 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0f67c9d1917581825320c HTTP/1.1" 200 None 2026-03-13 04:58:54,922 - lava_callback - ERROR - Error processing callback for node 69b0f67c9d1917581825320c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:52080 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41960 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34476 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34486 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34494 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54470 - "GET / HTTP/1.0" 200 OK 2026-03-13 04:59:34,949 - lava_callback - INFO - Received callback for node 69b0f6a09d19175818253217 INFO: 172.20.0.1:38734 - "POST /node/69b0f6a09d19175818253217 HTTP/1.0" 202 Accepted 2026-03-13 04:59:34,954 - lava_callback - INFO - Starting async job submit for node 69b0f6a09d19175818253217 2026-03-13 04:59:34,955 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:59:34,974 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0f6a09d19175818253217 HTTP/1.1" 200 None 2026-03-13 04:59:34,975 - lava_callback - DEBUG - Processing node 69b0f6a09d19175818253217: baseline-arm-clabbe 2026-03-13 04:59:34,975 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0f6a09d19175818253217 2026-03-13 04:59:34,976 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:59:35,011 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0f6a09d19175818253217 HTTP/1.1" 200 None 2026-03-13 04:59:35,012 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:59:35,025 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ada13b8fc97e5ba07d4d75 HTTP/1.1" 200 None 2026-03-13 04:59:35,027 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:59:35,052 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0f6a09d19175818253217 HTTP/1.1" 200 None 2026-03-13 04:59:35,061 - lava_callback - ERROR - Error processing callback for node 69b0f6a09d19175818253217: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:38740 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58674 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36302 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50374 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43034 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43042 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43056 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55016 - "GET / HTTP/1.0" 200 OK 2026-03-13 05:00:34,958 - lava_callback - INFO - Received callback for node 69b0f6e99d19175818253231 INFO: 172.20.0.1:34834 - "POST /node/69b0f6e99d19175818253231 HTTP/1.0" 202 Accepted 2026-03-13 05:00:34,963 - lava_callback - INFO - Starting async job submit for node 69b0f6e99d19175818253231 2026-03-13 05:00:34,964 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:00:35,032 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0f6e99d19175818253231 HTTP/1.1" 200 None 2026-03-13 05:00:35,033 - lava_callback - DEBUG - Processing node 69b0f6e99d19175818253231: baseline-arm-clabbe 2026-03-13 05:00:35,033 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0f6e99d19175818253231 2026-03-13 05:00:35,034 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:00:35,058 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0f6e99d19175818253231 HTTP/1.1" 200 None 2026-03-13 05:00:35,059 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:00:35,072 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ada13b8fc97e5ba07d4d75 HTTP/1.1" 200 None 2026-03-13 05:00:35,074 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:00:35,118 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0f6e99d19175818253231 HTTP/1.1" 200 None 2026-03-13 05:00:35,129 - lava_callback - ERROR - Error processing callback for node 69b0f6e99d19175818253231: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:34850 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54696 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44742 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34656 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39548 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39550 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39560 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42478 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38162 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43302 - "GET / HTTP/1.0" 200 OK 2026-03-13 05:01:54,991 - lava_callback - INFO - Received callback for node 69b0f7359d19175818253244 INFO: 172.20.0.1:45344 - "POST /node/69b0f7359d19175818253244 HTTP/1.0" 202 Accepted 2026-03-13 05:01:54,997 - lava_callback - INFO - Starting async job submit for node 69b0f7359d19175818253244 2026-03-13 05:01:54,997 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:01:55,013 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0f7359d19175818253244 HTTP/1.1" 200 None 2026-03-13 05:01:55,014 - lava_callback - DEBUG - Processing node 69b0f7359d19175818253244: baseline-arm-clabbe 2026-03-13 05:01:55,014 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0f7359d19175818253244 2026-03-13 05:01:55,014 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:01:55,027 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0f7359d19175818253244 HTTP/1.1" 200 None 2026-03-13 05:01:55,029 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:01:55,042 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ada13b8fc97e5ba07d4d75 HTTP/1.1" 200 None 2026-03-13 05:01:55,043 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:01:55,064 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0f7359d19175818253244 HTTP/1.1" 200 None 2026-03-13 05:01:55,065 - lava_callback - ERROR - Error processing callback for node 69b0f7359d19175818253244: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:45352 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52118 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53900 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53916 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53918 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45112 - "GET / HTTP/1.0" 200 OK 2026-03-13 05:02:35,000 - lava_callback - INFO - Received callback for node 69b0f75a9d19175818253255 INFO: 172.20.0.1:33506 - "POST /node/69b0f75a9d19175818253255 HTTP/1.0" 202 Accepted 2026-03-13 05:02:35,007 - lava_callback - INFO - Starting async job submit for node 69b0f75a9d19175818253255 2026-03-13 05:02:35,008 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:02:35,022 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0f75a9d19175818253255 HTTP/1.1" 200 None 2026-03-13 05:02:35,023 - lava_callback - DEBUG - Processing node 69b0f75a9d19175818253255: baseline-arm-clabbe 2026-03-13 05:02:35,023 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0f75a9d19175818253255 2026-03-13 05:02:35,024 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:02:35,037 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0f75a9d19175818253255 HTTP/1.1" 200 None 2026-03-13 05:02:35,038 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:02:35,057 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ada13b8fc97e5ba07d4d75 HTTP/1.1" 200 None 2026-03-13 05:02:35,059 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:02:35,084 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0f75a9d19175818253255 HTTP/1.1" 200 None 2026-03-13 05:02:35,085 - lava_callback - ERROR - Error processing callback for node 69b0f75a9d19175818253255: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:33512 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51370 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34066 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35942 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56812 - "GET / HTTP/1.0" 200 OK 2026-03-13 05:03:15,073 - lava_callback - INFO - Received callback for node 69b0f77f9d1917581825326b INFO: 172.20.0.1:56822 - "POST /node/69b0f77f9d1917581825326b HTTP/1.0" 202 Accepted 2026-03-13 05:03:15,078 - lava_callback - INFO - Starting async job submit for node 69b0f77f9d1917581825326b 2026-03-13 05:03:15,079 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:03:15,097 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0f77f9d1917581825326b HTTP/1.1" 200 None 2026-03-13 05:03:15,097 - lava_callback - DEBUG - Processing node 69b0f77f9d1917581825326b: baseline-arm-clabbe 2026-03-13 05:03:15,098 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0f77f9d1917581825326b 2026-03-13 05:03:15,098 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:03:15,111 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0f77f9d1917581825326b HTTP/1.1" 200 None 2026-03-13 05:03:15,113 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:03:15,133 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ada13b8fc97e5ba07d4d75 HTTP/1.1" 200 None 2026-03-13 05:03:15,135 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.20.0.1:56832 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-13 05:03:15,158 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0f77f9d1917581825326b HTTP/1.1" 200 None 2026-03-13 05:03:15,159 - lava_callback - ERROR - Error processing callback for node 69b0f77f9d1917581825326b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:56846 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34770 - "GET / HTTP/1.0" 200 OK 2026-03-13 05:03:35,091 - lava_callback - INFO - Received callback for node 69b0f7a49d19175818253281 INFO: 172.20.0.1:57976 - "POST /node/69b0f7a49d19175818253281 HTTP/1.0" 202 Accepted 2026-03-13 05:03:35,096 - lava_callback - INFO - Starting async job submit for node 69b0f7a49d19175818253281 2026-03-13 05:03:35,097 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:03:35,112 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0f7a49d19175818253281 HTTP/1.1" 200 None 2026-03-13 05:03:35,113 - lava_callback - DEBUG - Processing node 69b0f7a49d19175818253281: baseline-arm-clabbe 2026-03-13 05:03:35,114 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0f7a49d19175818253281 2026-03-13 05:03:35,114 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:03:35,153 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0f7a49d19175818253281 HTTP/1.1" 200 None 2026-03-13 05:03:35,155 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:03:35,168 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ada13b8fc97e5ba07d4d75 HTTP/1.1" 200 None 2026-03-13 05:03:35,170 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:03:35,199 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0f7a49d19175818253281 HTTP/1.1" 200 None 2026-03-13 05:03:35,204 - lava_callback - ERROR - Error processing callback for node 69b0f7a49d19175818253281: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:57978 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54796 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56242 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42104 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39414 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39420 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-13 05:04:15,408 - lava_callback - INFO - Received callback for node 69b0f7c89d1917581825328c INFO: 172.20.0.1:39422 - "POST /node/69b0f7c89d1917581825328c HTTP/1.0" 202 Accepted 2026-03-13 05:04:15,414 - lava_callback - INFO - Starting async job submit for node 69b0f7c89d1917581825328c 2026-03-13 05:04:15,416 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:04:15,432 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0f7c89d1917581825328c HTTP/1.1" 200 None 2026-03-13 05:04:15,433 - lava_callback - DEBUG - Processing node 69b0f7c89d1917581825328c: baseline-arm-clabbe 2026-03-13 05:04:15,433 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0f7c89d1917581825328c 2026-03-13 05:04:15,434 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:04:16,242 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0f7c89d1917581825328c HTTP/1.1" 200 None 2026-03-13 05:04:16,245 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:04:16,260 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ada13b8fc97e5ba07d4d75 HTTP/1.1" 200 None 2026-03-13 05:04:16,262 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:04:16,285 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0f7c89d1917581825328c HTTP/1.1" 200 None 2026-03-13 05:04:16,287 - lava_callback - ERROR - Error processing callback for node 69b0f7c89d1917581825328c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:39426 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43268 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53434 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49258 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42192 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35612 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44288 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44300 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44306 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59374 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36774 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33184 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57370 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55866 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44218 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44220 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44234 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55226 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51156 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48100 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52964 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37098 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53380 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53394 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53404 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60286 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43224 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37928 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46156 - "GET / HTTP/1.0" 200 OK 2026-03-13 05:08:00,086 - lava_callback - INFO - Received callback for node 69b0f8a59d191758182532a8 INFO: 172.20.0.1:36522 - "POST /node/69b0f8a59d191758182532a8 HTTP/1.0" 202 Accepted 2026-03-13 05:08:00,092 - lava_callback - INFO - Starting async job submit for node 69b0f8a59d191758182532a8 2026-03-13 05:08:00,092 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:08:00,106 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0f8a59d191758182532a8 HTTP/1.1" 200 None 2026-03-13 05:08:00,107 - lava_callback - DEBUG - Processing node 69b0f8a59d191758182532a8: baseline-arm64-baylibre 2026-03-13 05:08:00,108 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0f8a59d191758182532a8 2026-03-13 05:08:00,108 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:08:00,127 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0f8a59d191758182532a8 HTTP/1.1" 200 None 2026-03-13 05:08:00,129 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:08:00,143 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ada1488fc97e5ba07d4d8d HTTP/1.1" 200 None 2026-03-13 05:08:00,145 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:08:00,171 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0f8a59d191758182532a8 HTTP/1.1" 200 None 2026-03-13 05:08:00,172 - lava_callback - ERROR - Error processing callback for node 69b0f8a59d191758182532a8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:36528 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46468 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46480 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46492 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51068 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37080 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49008 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33416 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59094 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59374 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59376 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59378 - "GET / HTTP/1.0" 200 OK 2026-03-13 05:09:20,200 - lava_callback - INFO - Received callback for node 69b0f8f59d191758182532bd INFO: 172.20.0.1:34198 - "POST /node/69b0f8f59d191758182532bd HTTP/1.0" 202 Accepted 2026-03-13 05:09:20,207 - lava_callback - INFO - Starting async job submit for node 69b0f8f59d191758182532bd 2026-03-13 05:09:20,208 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:09:20,222 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0f8f59d191758182532bd HTTP/1.1" 200 None 2026-03-13 05:09:20,223 - lava_callback - DEBUG - Processing node 69b0f8f59d191758182532bd: baseline-arm64-baylibre 2026-03-13 05:09:20,224 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0f8f59d191758182532bd 2026-03-13 05:09:20,224 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:09:20,239 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0f8f59d191758182532bd HTTP/1.1" 200 None 2026-03-13 05:09:20,241 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:09:20,255 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ada1488fc97e5ba07d4d8d HTTP/1.1" 200 None 2026-03-13 05:09:20,257 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:09:20,279 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0f8f59d191758182532bd HTTP/1.1" 200 None 2026-03-13 05:09:20,281 - lava_callback - ERROR - Error processing callback for node 69b0f8f59d191758182532bd: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:34210 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44296 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59046 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44470 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36940 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35040 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35042 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35056 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50028 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57368 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34562 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49034 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57460 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47892 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47894 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47906 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45776 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44222 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43680 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40318 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46198 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49054 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49058 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-13 05:12:15,436 - lava_callback - INFO - Received callback for node 69b0f99f9d191758182532da INFO: 172.20.0.1:49062 - "POST /node/69b0f99f9d191758182532da HTTP/1.0" 202 Accepted 2026-03-13 05:12:15,441 - lava_callback - INFO - Starting async job submit for node 69b0f99f9d191758182532da 2026-03-13 05:12:15,443 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:12:15,478 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0f99f9d191758182532da HTTP/1.1" 200 None 2026-03-13 05:12:15,478 - lava_callback - DEBUG - Processing node 69b0f99f9d191758182532da: baseline-arm64-clabbe 2026-03-13 05:12:15,479 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0f99f9d191758182532da 2026-03-13 05:12:15,479 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:12:15,494 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0f99f9d191758182532da HTTP/1.1" 200 None 2026-03-13 05:12:15,495 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:12:15,509 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ada1488fc97e5ba07d4d8d HTTP/1.1" 200 None 2026-03-13 05:12:15,510 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:12:15,531 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0f99f9d191758182532da HTTP/1.1" 200 None 2026-03-13 05:12:15,536 - lava_callback - ERROR - Error processing callback for node 69b0f99f9d191758182532da: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:49074 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40820 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50530 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39904 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42796 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41642 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35148 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35152 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35166 - "GET / HTTP/1.0" 200 OK 2026-03-13 05:13:20,376 - lava_callback - INFO - Received callback for node 69b0f9e99d191758182532ee INFO: 172.20.0.1:56422 - "POST /node/69b0f9e99d191758182532ee HTTP/1.0" 202 Accepted 2026-03-13 05:13:20,381 - lava_callback - INFO - Starting async job submit for node 69b0f9e99d191758182532ee 2026-03-13 05:13:20,382 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:13:20,424 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0f9e99d191758182532ee HTTP/1.1" 200 None 2026-03-13 05:13:20,426 - lava_callback - DEBUG - Processing node 69b0f9e99d191758182532ee: baseline-arm64-baylibre 2026-03-13 05:13:20,426 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0f9e99d191758182532ee 2026-03-13 05:13:20,427 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:13:20,449 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0f9e99d191758182532ee HTTP/1.1" 200 None 2026-03-13 05:13:20,451 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:13:20,470 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ada1488fc97e5ba07d4d8d HTTP/1.1" 200 None 2026-03-13 05:13:20,472 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:13:20,492 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0f9e99d191758182532ee HTTP/1.1" 200 None 2026-03-13 05:13:20,500 - lava_callback - ERROR - Error processing callback for node 69b0f9e99d191758182532ee: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:56434 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55534 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39666 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35100 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42604 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45218 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45220 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45236 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49386 - "GET / HTTP/1.0" 200 OK 2026-03-13 05:14:35,544 - lava_callback - INFO - Received callback for node 69b0fa389d19175818253306 INFO: 172.20.0.1:38308 - "POST /node/69b0fa389d19175818253306 HTTP/1.0" 202 Accepted 2026-03-13 05:14:35,550 - lava_callback - INFO - Starting async job submit for node 69b0fa389d19175818253306 2026-03-13 05:14:35,551 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:14:35,568 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0fa389d19175818253306 HTTP/1.1" 200 None 2026-03-13 05:14:35,569 - lava_callback - DEBUG - Processing node 69b0fa389d19175818253306: baseline-arm64-clabbe 2026-03-13 05:14:35,569 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0fa389d19175818253306 2026-03-13 05:14:35,570 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:14:35,600 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0fa389d19175818253306 HTTP/1.1" 200 None 2026-03-13 05:14:35,602 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:14:35,620 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ada1488fc97e5ba07d4d8d HTTP/1.1" 200 None 2026-03-13 05:14:35,622 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:14:35,642 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0fa389d19175818253306 HTTP/1.1" 200 None 2026-03-13 05:14:35,644 - lava_callback - ERROR - Error processing callback for node 69b0fa389d19175818253306: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:38318 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48434 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47004 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59230 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44806 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44822 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44834 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46076 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58466 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51414 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58456 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33814 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47122 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47128 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47136 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48498 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34024 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42096 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58706 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50118 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52318 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52322 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-13 05:17:15,660 - lava_callback - INFO - Received callback for node 69b0fad39d19175818253326 INFO: 172.20.0.1:52330 - "POST /node/69b0fad39d19175818253326 HTTP/1.0" 202 Accepted 2026-03-13 05:17:15,665 - lava_callback - INFO - Starting async job submit for node 69b0fad39d19175818253326 2026-03-13 05:17:15,666 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:17:15,681 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0fad39d19175818253326 HTTP/1.1" 200 None 2026-03-13 05:17:15,682 - lava_callback - DEBUG - Processing node 69b0fad39d19175818253326: baseline-arm64-clabbe 2026-03-13 05:17:15,682 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0fad39d19175818253326 2026-03-13 05:17:15,683 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:17:15,718 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0fad39d19175818253326 HTTP/1.1" 200 None 2026-03-13 05:17:15,720 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:17:15,734 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ada1488fc97e5ba07d4d8d HTTP/1.1" 200 None 2026-03-13 05:17:15,735 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:17:15,762 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0fad39d19175818253326 HTTP/1.1" 200 None 2026-03-13 05:17:15,768 - lava_callback - ERROR - Error processing callback for node 69b0fad39d19175818253326: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:52334 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56358 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60354 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50472 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45884 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51282 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39710 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39726 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39732 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58478 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41820 - "GET / HTTP/1.0" 200 OK 2026-03-13 05:18:40,730 - lava_callback - INFO - Received callback for node 69b0fb1f9d19175818253330 INFO: 172.20.0.1:60988 - "POST /node/69b0fb1f9d19175818253330 HTTP/1.0" 202 Accepted 2026-03-13 05:18:40,738 - lava_callback - INFO - Starting async job submit for node 69b0fb1f9d19175818253330 2026-03-13 05:18:40,739 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:18:40,753 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0fb1f9d19175818253330 HTTP/1.1" 200 None 2026-03-13 05:18:40,754 - lava_callback - DEBUG - Processing node 69b0fb1f9d19175818253330: baseline-arm64-baylibre 2026-03-13 05:18:40,754 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0fb1f9d19175818253330 2026-03-13 05:18:40,754 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:18:40,789 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0fb1f9d19175818253330 HTTP/1.1" 200 None 2026-03-13 05:18:40,790 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:18:40,804 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ada1488fc97e5ba07d4d8d HTTP/1.1" 200 None 2026-03-13 05:18:40,806 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:18:40,830 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0fb1f9d19175818253330 HTTP/1.1" 200 None 2026-03-13 05:18:40,836 - lava_callback - ERROR - Error processing callback for node 69b0fb1f9d19175818253330: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:60994 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36358 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49848 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47094 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47096 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47102 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55708 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47450 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60088 - "GET / HTTP/1.0" 200 OK 2026-03-13 05:19:55,789 - lava_callback - INFO - Received callback for node 69b0fb759d19175818253341 INFO: 172.20.0.1:35716 - "POST /node/69b0fb759d19175818253341 HTTP/1.0" 202 Accepted 2026-03-13 05:19:55,795 - lava_callback - INFO - Starting async job submit for node 69b0fb759d19175818253341 2026-03-13 05:19:55,796 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:19:55,810 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0fb759d19175818253341 HTTP/1.1" 200 None 2026-03-13 05:19:55,811 - lava_callback - DEBUG - Processing node 69b0fb759d19175818253341: baseline-arm64-clabbe 2026-03-13 05:19:55,811 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0fb759d19175818253341 2026-03-13 05:19:55,811 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:19:55,843 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0fb759d19175818253341 HTTP/1.1" 200 None 2026-03-13 05:19:55,844 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:19:55,861 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ada1488fc97e5ba07d4d8d HTTP/1.1" 200 None 2026-03-13 05:19:55,863 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:19:55,892 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0fb759d19175818253341 HTTP/1.1" 200 None 2026-03-13 05:19:55,894 - lava_callback - ERROR - Error processing callback for node 69b0fb759d19175818253341: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:35724 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49640 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34962 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34970 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34978 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38406 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47158 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42492 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36422 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48028 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41216 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41230 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41242 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54662 - "GET / HTTP/1.0" 200 OK 2026-03-13 05:21:35,854 - lava_callback - INFO - Received callback for node 69b0fbcb9d19175818253353 INFO: 172.20.0.1:55068 - "POST /node/69b0fbcb9d19175818253353 HTTP/1.0" 202 Accepted 2026-03-13 05:21:35,859 - lava_callback - INFO - Starting async job submit for node 69b0fbcb9d19175818253353 2026-03-13 05:21:35,860 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:21:35,874 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0fbcb9d19175818253353 HTTP/1.1" 200 None 2026-03-13 05:21:35,875 - lava_callback - DEBUG - Processing node 69b0fbcb9d19175818253353: baseline-arm64-clabbe 2026-03-13 05:21:35,875 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0fbcb9d19175818253353 2026-03-13 05:21:35,876 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:21:35,891 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0fbcb9d19175818253353 HTTP/1.1" 200 None 2026-03-13 05:21:35,893 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:21:35,908 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ada1488fc97e5ba07d4d8d HTTP/1.1" 200 None 2026-03-13 05:21:35,909 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:21:35,943 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0fbcb9d19175818253353 HTTP/1.1" 200 None 2026-03-13 05:21:35,945 - lava_callback - ERROR - Error processing callback for node 69b0fbcb9d19175818253353: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:55070 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49022 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58590 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38510 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56426 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56440 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56446 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56556 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56856 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44876 - "GET / HTTP/1.0" 200 OK 2026-03-13 05:22:55,896 - lava_callback - INFO - Received callback for node 69b0fc1c9d19175818253368 INFO: 172.20.0.1:54106 - "POST /node/69b0fc1c9d19175818253368 HTTP/1.0" 202 Accepted 2026-03-13 05:22:55,901 - lava_callback - INFO - Starting async job submit for node 69b0fc1c9d19175818253368 2026-03-13 05:22:55,902 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:22:55,917 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0fc1c9d19175818253368 HTTP/1.1" 200 None 2026-03-13 05:22:55,918 - lava_callback - DEBUG - Processing node 69b0fc1c9d19175818253368: baseline-arm64-clabbe 2026-03-13 05:22:55,918 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0fc1c9d19175818253368 2026-03-13 05:22:55,919 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:22:55,962 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0fc1c9d19175818253368 HTTP/1.1" 200 None 2026-03-13 05:22:55,964 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:22:55,978 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ada1488fc97e5ba07d4d8d HTTP/1.1" 200 None 2026-03-13 05:22:55,979 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:22:56,000 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0fc1c9d19175818253368 HTTP/1.1" 200 None 2026-03-13 05:22:56,006 - lava_callback - ERROR - Error processing callback for node 69b0fc1c9d19175818253368: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:54118 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37718 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45208 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45218 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45228 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48388 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55970 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54798 - "GET / HTTP/1.0" 200 OK 2026-03-13 05:23:55,964 - lava_callback - INFO - Received callback for node 69b0fc699d19175818253370 INFO: 172.20.0.1:39822 - "POST /node/69b0fc699d19175818253370 HTTP/1.0" 202 Accepted 2026-03-13 05:23:55,969 - lava_callback - INFO - Starting async job submit for node 69b0fc699d19175818253370 2026-03-13 05:23:55,970 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:23:56,009 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0fc699d19175818253370 HTTP/1.1" 200 None 2026-03-13 05:23:56,010 - lava_callback - DEBUG - Processing node 69b0fc699d19175818253370: baseline-arm64-clabbe 2026-03-13 05:23:56,010 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0fc699d19175818253370 2026-03-13 05:23:56,011 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:23:56,025 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0fc699d19175818253370 HTTP/1.1" 200 None 2026-03-13 05:23:56,026 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:23:56,039 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ada1488fc97e5ba07d4d8d HTTP/1.1" 200 None 2026-03-13 05:23:56,040 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:23:56,061 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0fc699d19175818253370 HTTP/1.1" 200 None 2026-03-13 05:23:56,062 - lava_callback - ERROR - Error processing callback for node 69b0fc699d19175818253370: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:39830 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57594 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33348 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33350 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33352 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60914 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41216 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40902 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60786 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44008 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39504 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39514 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-13 05:25:16,285 - lava_callback - INFO - Received callback for node 69b0fcb39d19175818253389 INFO: 172.20.0.1:39524 - "POST /node/69b0fcb39d19175818253389 HTTP/1.0" 202 Accepted 2026-03-13 05:25:16,290 - lava_callback - INFO - Starting async job submit for node 69b0fcb39d19175818253389 2026-03-13 05:25:16,291 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:25:16,313 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0fcb39d19175818253389 HTTP/1.1" 200 None 2026-03-13 05:25:16,314 - lava_callback - DEBUG - Processing node 69b0fcb39d19175818253389: baseline-arm64-clabbe 2026-03-13 05:25:16,314 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0fcb39d19175818253389 2026-03-13 05:25:16,315 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:25:16,347 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0fcb39d19175818253389 HTTP/1.1" 200 None 2026-03-13 05:25:16,348 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:25:16,370 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ada1488fc97e5ba07d4d8d HTTP/1.1" 200 None 2026-03-13 05:25:16,371 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:25:16,405 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0fcb39d19175818253389 HTTP/1.1" 200 None 2026-03-13 05:25:16,417 - lava_callback - ERROR - Error processing callback for node 69b0fcb39d19175818253389: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:39528 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48708 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38952 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47260 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46590 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53524 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56128 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56138 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56140 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35158 - "GET / HTTP/1.0" 200 OK 2026-03-13 05:26:36,048 - lava_callback - INFO - Received callback for node 69b0fd019d19175818253393 INFO: 172.20.0.1:48026 - "POST /node/69b0fd019d19175818253393 HTTP/1.0" 202 Accepted 2026-03-13 05:26:36,055 - lava_callback - INFO - Starting async job submit for node 69b0fd019d19175818253393 2026-03-13 05:26:36,056 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:26:36,092 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0fd019d19175818253393 HTTP/1.1" 200 None 2026-03-13 05:26:36,093 - lava_callback - DEBUG - Processing node 69b0fd019d19175818253393: baseline-arm64-clabbe 2026-03-13 05:26:36,093 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0fd019d19175818253393 2026-03-13 05:26:36,094 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:26:36,108 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0fd019d19175818253393 HTTP/1.1" 200 None 2026-03-13 05:26:36,109 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:26:36,138 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ada1488fc97e5ba07d4d8d HTTP/1.1" 200 None 2026-03-13 05:26:36,140 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:26:36,161 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0fd019d19175818253393 HTTP/1.1" 200 None 2026-03-13 05:26:36,166 - lava_callback - ERROR - Error processing callback for node 69b0fd019d19175818253393: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:48034 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48370 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60350 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50822 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54394 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54408 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54410 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58788 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38624 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55472 - "GET / HTTP/1.0" 200 OK 2026-03-13 05:27:56,078 - lava_callback - INFO - Received callback for node 69b0fd519d191758182533a7 INFO: 172.20.0.1:55300 - "POST /node/69b0fd519d191758182533a7 HTTP/1.0" 202 Accepted 2026-03-13 05:27:56,084 - lava_callback - INFO - Starting async job submit for node 69b0fd519d191758182533a7 2026-03-13 05:27:56,084 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:27:56,100 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0fd519d191758182533a7 HTTP/1.1" 200 None 2026-03-13 05:27:56,100 - lava_callback - DEBUG - Processing node 69b0fd519d191758182533a7: baseline-arm64-clabbe 2026-03-13 05:27:56,101 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0fd519d191758182533a7 2026-03-13 05:27:56,101 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:27:56,113 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0fd519d191758182533a7 HTTP/1.1" 200 None 2026-03-13 05:27:56,115 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:27:56,144 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ada1488fc97e5ba07d4d8d HTTP/1.1" 200 None 2026-03-13 05:27:56,145 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:27:56,168 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0fd519d191758182533a7 HTTP/1.1" 200 None 2026-03-13 05:27:56,179 - lava_callback - ERROR - Error processing callback for node 69b0fd519d191758182533a7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:55314 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43576 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35856 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35870 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35872 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55204 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57344 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52376 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40362 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48502 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37116 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37122 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-13 05:29:16,128 - lava_callback - INFO - Received callback for node 69b0fda39d191758182533be INFO: 172.20.0.1:37132 - "POST /node/69b0fda39d191758182533be HTTP/1.0" 202 Accepted 2026-03-13 05:29:16,134 - lava_callback - INFO - Starting async job submit for node 69b0fda39d191758182533be 2026-03-13 05:29:16,135 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:29:16,183 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0fda39d191758182533be HTTP/1.1" 200 None 2026-03-13 05:29:16,184 - lava_callback - DEBUG - Processing node 69b0fda39d191758182533be: baseline-arm64-clabbe 2026-03-13 05:29:16,184 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0fda39d191758182533be 2026-03-13 05:29:16,185 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:29:16,202 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0fda39d191758182533be HTTP/1.1" 200 None 2026-03-13 05:29:16,203 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:29:16,216 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ada1488fc97e5ba07d4d8d HTTP/1.1" 200 None 2026-03-13 05:29:16,218 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:29:16,246 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0fda39d191758182533be HTTP/1.1" 200 None 2026-03-13 05:29:16,247 - lava_callback - ERROR - Error processing callback for node 69b0fda39d191758182533be: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:37142 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35564 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34626 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36992 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32886 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56056 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40762 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40778 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40788 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58528 - "GET / HTTP/1.0" 200 OK 2026-03-13 05:30:36,220 - lava_callback - INFO - Received callback for node 69b0fdf49d191758182533c5 INFO: 172.20.0.1:33394 - "POST /node/69b0fdf49d191758182533c5 HTTP/1.0" 202 Accepted 2026-03-13 05:30:36,226 - lava_callback - INFO - Starting async job submit for node 69b0fdf49d191758182533c5 2026-03-13 05:30:36,227 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:30:36,245 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0fdf49d191758182533c5 HTTP/1.1" 200 None 2026-03-13 05:30:36,246 - lava_callback - DEBUG - Processing node 69b0fdf49d191758182533c5: baseline-arm64-clabbe 2026-03-13 05:30:36,246 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0fdf49d191758182533c5 2026-03-13 05:30:36,247 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:30:36,281 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0fdf49d191758182533c5 HTTP/1.1" 200 None 2026-03-13 05:30:36,283 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:30:36,321 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ada1488fc97e5ba07d4d8d HTTP/1.1" 200 None 2026-03-13 05:30:36,322 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:30:36,343 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0fdf49d191758182533c5 HTTP/1.1" 200 None 2026-03-13 05:30:36,345 - lava_callback - ERROR - Error processing callback for node 69b0fdf49d191758182533c5: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:33408 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38082 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35494 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51412 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39850 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39854 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39856 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40898 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46366 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40858 - "GET / HTTP/1.0" 200 OK 2026-03-13 05:31:56,311 - lava_callback - INFO - Received callback for node 69b0fe479d191758182533d8 INFO: 172.20.0.1:51484 - "POST /node/69b0fe479d191758182533d8 HTTP/1.0" 202 Accepted 2026-03-13 05:31:56,316 - lava_callback - INFO - Starting async job submit for node 69b0fe479d191758182533d8 2026-03-13 05:31:56,317 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:31:56,344 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0fe479d191758182533d8 HTTP/1.1" 200 None 2026-03-13 05:31:56,346 - lava_callback - DEBUG - Processing node 69b0fe479d191758182533d8: baseline-arm64-clabbe 2026-03-13 05:31:56,346 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0fe479d191758182533d8 2026-03-13 05:31:56,348 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:31:56,371 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0fe479d191758182533d8 HTTP/1.1" 200 None 2026-03-13 05:31:56,373 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:31:56,390 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ada1488fc97e5ba07d4d8d HTTP/1.1" 200 None 2026-03-13 05:31:56,392 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:31:56,413 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0fe479d191758182533d8 HTTP/1.1" 200 None 2026-03-13 05:31:56,414 - lava_callback - ERROR - Error processing callback for node 69b0fe479d191758182533d8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:51490 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35878 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51634 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51636 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51642 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42336 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34008 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57862 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52848 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36200 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45430 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45440 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-13 05:33:16,358 - lava_callback - INFO - Received callback for node 69b0fe949d191758182533ed INFO: 172.20.0.1:45448 - "POST /node/69b0fe949d191758182533ed HTTP/1.0" 202 Accepted 2026-03-13 05:33:16,364 - lava_callback - INFO - Starting async job submit for node 69b0fe949d191758182533ed 2026-03-13 05:33:16,365 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:33:16,379 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0fe949d191758182533ed HTTP/1.1" 200 None 2026-03-13 05:33:16,380 - lava_callback - DEBUG - Processing node 69b0fe949d191758182533ed: baseline-arm64-clabbe 2026-03-13 05:33:16,380 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0fe949d191758182533ed 2026-03-13 05:33:16,380 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:33:16,394 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0fe949d191758182533ed HTTP/1.1" 200 None 2026-03-13 05:33:16,395 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:33:16,408 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ada1488fc97e5ba07d4d8d HTTP/1.1" 200 None 2026-03-13 05:33:16,410 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:33:16,442 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0fe949d191758182533ed HTTP/1.1" 200 None 2026-03-13 05:33:16,444 - lava_callback - ERROR - Error processing callback for node 69b0fe949d191758182533ed: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:45454 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47050 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56888 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46792 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46062 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51844 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54092 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54094 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54096 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39318 - "GET / HTTP/1.0" 200 OK 2026-03-13 05:34:36,416 - lava_callback - INFO - Received callback for node 69b0fedd9d191758182533f5 INFO: 172.20.0.1:52692 - "POST /node/69b0fedd9d191758182533f5 HTTP/1.0" 202 Accepted 2026-03-13 05:34:36,421 - lava_callback - INFO - Starting async job submit for node 69b0fedd9d191758182533f5 2026-03-13 05:34:36,422 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:34:36,445 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0fedd9d191758182533f5 HTTP/1.1" 200 None 2026-03-13 05:34:36,446 - lava_callback - DEBUG - Processing node 69b0fedd9d191758182533f5: baseline-arm64-clabbe 2026-03-13 05:34:36,446 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0fedd9d191758182533f5 2026-03-13 05:34:36,447 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:34:36,460 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0fedd9d191758182533f5 HTTP/1.1" 200 None 2026-03-13 05:34:36,462 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:34:36,475 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ada1488fc97e5ba07d4d8d HTTP/1.1" 200 None 2026-03-13 05:34:36,477 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:34:36,497 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0fedd9d191758182533f5 HTTP/1.1" 200 None 2026-03-13 05:34:36,498 - lava_callback - ERROR - Error processing callback for node 69b0fedd9d191758182533f5: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:52706 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46968 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36202 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48284 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59510 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59520 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59528 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57990 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39482 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60314 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51028 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37414 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50630 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50634 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50636 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36200 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43764 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35436 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57830 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37020 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44636 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44650 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44656 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50732 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39464 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47056 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39614 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34766 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46150 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46158 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46166 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45760 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45754 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39856 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34354 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52314 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53042 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53048 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53060 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39442 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34062 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51754 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40560 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49068 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57342 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57344 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57354 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53830 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37986 - "GET / HTTP/1.0" 200 OK 2026-03-13 05:40:39,973 - lava_callback - INFO - Received callback for node 69b363d1974c91e64560ed6a INFO: 172.20.0.1:45764 - "POST /node/69b363d1974c91e64560ed6a HTTP/1.0" 202 Accepted 2026-03-13 05:40:39,979 - lava_callback - INFO - Starting async job submit for node 69b363d1974c91e64560ed6a 2026-03-13 05:40:40,003 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:40:40,035 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b363d1974c91e64560ed6a HTTP/1.1" 200 None 2026-03-13 05:40:40,036 - lava_callback - DEBUG - Processing node 69b363d1974c91e64560ed6a: baseline-arm64-broonie 2026-03-13 05:40:40,318 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 05:40:40,480 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 05:40:40,482 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69b363d1974c91e64560ed6a/log.txt.gz 2026-03-13 05:40:40,639 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 05:40:40,886 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 05:40:40,889 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69b363d1974c91e64560ed6a/lava-logs.yaml 2026-03-13 05:40:40,897 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 05:40:41,052 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 05:40:41,054 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69b363d1974c91e64560ed6a/lava_callback.json.gz 2026-03-13 05:40:41,077 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:40:41,091 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b363d1974c91e64560ed6a HTTP/1.1" 200 None 2026-03-13 05:40:41,093 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:40:41,106 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3594d974c91e64560e4b2 HTTP/1.1" 200 None 2026-03-13 05:40:41,108 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:40:41,139 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b363d1974c91e64560ed6a HTTP/1.1" 200 None 2026-03-13 05:40:41,143 - lava_callback - INFO - Completed processing callback for node 69b363d1974c91e64560ed6a 2026-03-13 05:40:46,422 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:40:46,439 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:45774 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46260 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43942 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38842 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38856 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38862 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47142 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52310 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47930 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40304 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60808 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56702 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56710 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56718 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58800 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54990 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33302 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49244 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48586 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60108 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60120 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60126 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37378 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51834 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40790 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42362 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55128 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47596 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47612 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47624 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40784 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46468 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54214 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56834 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39916 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49948 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49958 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49960 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54104 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38760 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46340 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42638 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34718 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55304 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55310 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55320 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40424 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55356 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42988 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52798 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59232 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56566 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56582 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56590 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41566 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44422 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36304 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41048 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42374 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38316 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38322 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38326 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33144 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49790 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55336 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38228 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36682 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36606 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36610 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36612 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58242 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35608 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60866 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44290 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46120 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53390 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53392 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53402 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49646 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48602 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33548 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50486 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58676 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51266 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51274 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51276 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55880 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42794 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59616 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47126 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34008 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42284 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42286 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42296 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49418 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46192 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57120 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44580 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40040 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37120 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37136 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37138 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44032 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36678 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49164 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44670 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36056 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56868 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56880 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56894 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57902 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48584 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34676 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38300 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43586 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52316 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52322 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52332 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35696 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55252 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57902 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42980 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43398 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48226 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48238 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48240 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58846 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58002 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57598 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45888 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43702 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35058 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35074 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35082 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34196 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52292 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46636 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37942 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59616 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38392 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38398 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38412 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46010 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43262 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40454 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48090 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52696 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33952 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33956 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33964 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59072 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49284 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58100 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59122 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45916 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47482 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47496 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47500 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58420 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58150 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48764 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37732 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41788 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48642 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48650 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48654 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34192 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48054 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34304 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45394 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42822 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45852 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45864 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45880 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56168 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49712 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35508 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43370 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33576 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47814 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47830 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47832 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53372 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38654 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39494 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58382 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33352 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53022 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53030 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53038 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40160 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48080 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54246 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55922 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36910 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37072 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37084 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37092 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38750 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43254 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54878 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38506 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54854 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41798 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41800 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41808 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35882 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47450 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52358 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50000 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48782 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38362 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38378 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38392 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48928 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42324 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35130 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51044 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51472 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55964 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55972 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55976 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51096 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53306 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36004 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50346 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43042 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42250 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42262 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42276 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55898 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58344 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39828 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53122 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44560 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33898 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33904 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33916 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52288 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33044 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58692 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48140 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41810 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37402 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37410 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37414 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50626 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49962 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36904 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56582 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40998 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53688 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53696 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53704 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40672 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40084 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40004 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54476 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42580 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47850 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47854 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47864 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51130 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44452 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54000 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50008 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41390 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52240 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52252 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52260 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60082 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54850 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38530 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49148 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60792 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38060 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38066 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38078 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43728 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40798 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57182 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38224 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46008 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56216 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56226 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56238 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39576 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55710 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35412 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42730 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44406 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45260 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45262 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45266 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51404 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39078 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37998 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47190 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52054 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40520 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40528 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40532 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39272 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49184 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59542 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40712 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53458 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41370 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41384 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41386 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44468 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37200 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41708 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51074 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54696 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39876 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39884 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39888 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40394 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57146 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56834 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35556 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60824 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55454 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55464 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55474 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34742 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45414 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55050 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45394 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40132 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60314 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60326 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60330 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46466 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53298 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45696 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50516 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58188 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48022 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48024 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48038 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38422 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41456 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41264 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58788 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47404 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46448 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46456 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46464 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49578 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49380 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54026 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38850 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58456 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52984 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52996 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52998 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58518 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56574 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48950 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59856 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37610 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60952 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60964 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60976 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60990 - "GET /node/69a62d087da105ad098359e3%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:37116 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36532 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57208 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50186 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51904 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48702 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48712 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48720 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33404 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59920 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53424 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47802 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44618 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42398 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42412 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42428 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33840 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39082 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60500 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45226 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40428 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33096 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33112 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33116 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58342 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57118 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54716 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33756 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35506 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52018 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52024 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52036 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34822 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38118 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56352 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43070 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57756 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49302 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49304 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49314 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44250 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44726 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51876 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56554 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48372 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51836 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51838 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51844 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49604 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47782 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36586 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53894 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54768 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44244 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44256 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44262 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45040 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41056 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47726 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53890 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52088 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43180 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43196 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43204 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58930 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55100 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56180 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37140 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57104 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51258 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51266 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51280 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53870 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43798 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36388 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57230 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55014 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38092 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38098 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38114 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52454 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59760 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51372 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45330 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38222 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36100 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36106 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36114 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53538 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33724 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55704 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54120 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44128 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53706 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53718 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53722 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60994 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60076 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51254 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51054 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43842 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50208 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50212 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50222 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43494 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34326 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60248 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56456 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60228 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45846 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45848 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45856 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58600 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57424 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37428 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44338 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45152 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38108 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38110 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38122 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33054 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32882 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59244 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54428 - "GET / HTTP/1.0" 200 OK 2026-03-13 06:42:05,487 - lava_callback - INFO - Received callback for node 69b10eb69d19175818253449 INFO: 172.20.0.1:58122 - "POST /node/69b10eb69d19175818253449 HTTP/1.0" 202 Accepted 2026-03-13 06:42:05,493 - lava_callback - INFO - Starting async job submit for node 69b10eb69d19175818253449 2026-03-13 06:42:05,494 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 06:42:05,509 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b10eb69d19175818253449 HTTP/1.1" 200 None 2026-03-13 06:42:05,510 - lava_callback - DEBUG - Processing node 69b10eb69d19175818253449: baseline-arm-baylibre 2026-03-13 06:42:05,510 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b10eb69d19175818253449 2026-03-13 06:42:05,511 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 06:42:05,524 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b10eb69d19175818253449 HTTP/1.1" 200 None 2026-03-13 06:42:05,526 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 06:42:05,539 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b4fc9d19175818251fd6 HTTP/1.1" 200 None 2026-03-13 06:42:05,540 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 06:42:05,561 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b10eb69d19175818253449 HTTP/1.1" 200 None 2026-03-13 06:42:05,570 - lava_callback - ERROR - Error processing callback for node 69b10eb69d19175818253449: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:58136 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43436 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43438 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43446 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58828 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44702 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41034 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43742 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38242 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41546 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41556 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41558 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38082 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49516 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40114 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49252 - "GET / HTTP/1.0" 200 OK 2026-03-13 06:43:59,341 - lava_callback - INFO - Received callback for node 69b10f249d19175818253473 INFO: 172.20.0.1:49266 - "POST /node/69b10f249d19175818253473 HTTP/1.0" 202 Accepted 2026-03-13 06:43:59,347 - lava_callback - INFO - Starting async job submit for node 69b10f249d19175818253473 2026-03-13 06:43:59,348 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 06:43:59,362 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b10f249d19175818253473 HTTP/1.1" 200 None 2026-03-13 06:43:59,363 - lava_callback - DEBUG - Processing node 69b10f249d19175818253473: baseline-arm-clabbe 2026-03-13 06:43:59,364 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b10f249d19175818253473 2026-03-13 06:43:59,364 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 06:43:59,378 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b10f249d19175818253473 HTTP/1.1" 200 None 2026-03-13 06:43:59,379 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 06:43:59,394 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b4fc9d19175818251fd6 HTTP/1.1" 200 None 2026-03-13 06:43:59,396 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 06:43:59,421 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b10f249d19175818253473 HTTP/1.1" 200 None 2026-03-13 06:43:59,422 - lava_callback - ERROR - Error processing callback for node 69b10f249d19175818253473: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:34406 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51630 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51636 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51650 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46972 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46270 - "GET / HTTP/1.0" 200 OK 2026-03-13 06:44:39,335 - lava_callback - INFO - Received callback for node 69b10f4a9d19175818253481 INFO: 172.20.0.1:46284 - "POST /node/69b10f4a9d19175818253481 HTTP/1.0" 202 Accepted 2026-03-13 06:44:39,341 - lava_callback - INFO - Starting async job submit for node 69b10f4a9d19175818253481 2026-03-13 06:44:39,341 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 06:44:39,355 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b10f4a9d19175818253481 HTTP/1.1" 200 None 2026-03-13 06:44:39,356 - lava_callback - DEBUG - Processing node 69b10f4a9d19175818253481: baseline-arm-clabbe 2026-03-13 06:44:39,357 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b10f4a9d19175818253481 2026-03-13 06:44:39,357 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 06:44:39,398 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b10f4a9d19175818253481 HTTP/1.1" 200 None 2026-03-13 06:44:39,400 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 06:44:39,415 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b4fc9d19175818251fd6 HTTP/1.1" 200 None 2026-03-13 06:44:39,417 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 06:44:39,439 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b10f4a9d19175818253481 HTTP/1.1" 200 None 2026-03-13 06:44:39,445 - lava_callback - ERROR - Error processing callback for node 69b10f4a9d19175818253481: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:39474 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47358 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40542 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56896 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56902 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56912 - "GET / HTTP/1.0" 200 OK 2026-03-13 06:45:19,383 - lava_callback - INFO - Received callback for node 69b10f6e9d19175818253491 INFO: 172.20.0.1:56924 - "POST /node/69b10f6e9d19175818253491 HTTP/1.0" 202 Accepted 2026-03-13 06:45:19,389 - lava_callback - INFO - Starting async job submit for node 69b10f6e9d19175818253491 2026-03-13 06:45:19,390 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 06:45:19,407 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b10f6e9d19175818253491 HTTP/1.1" 200 None 2026-03-13 06:45:19,408 - lava_callback - DEBUG - Processing node 69b10f6e9d19175818253491: baseline-arm-clabbe 2026-03-13 06:45:19,408 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b10f6e9d19175818253491 2026-03-13 06:45:19,408 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 06:45:19,423 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b10f6e9d19175818253491 HTTP/1.1" 200 None 2026-03-13 06:45:19,424 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 06:45:19,438 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b4fc9d19175818251fd6 HTTP/1.1" 200 None 2026-03-13 06:45:19,439 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 06:45:19,459 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b10f6e9d19175818253491 HTTP/1.1" 200 None 2026-03-13 06:45:19,461 - lava_callback - ERROR - Error processing callback for node 69b10f6e9d19175818253491: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:46378 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57132 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46674 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42722 - "GET / HTTP/1.0" 200 OK 2026-03-13 06:45:59,385 - lava_callback - INFO - Received callback for node 69b10f929d191758182534a3 INFO: 172.20.0.1:42724 - "POST /node/69b10f929d191758182534a3 HTTP/1.0" 202 Accepted 2026-03-13 06:45:59,390 - lava_callback - INFO - Starting async job submit for node 69b10f929d191758182534a3 2026-03-13 06:45:59,391 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 06:45:59,405 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b10f929d191758182534a3 HTTP/1.1" 200 None 2026-03-13 06:45:59,406 - lava_callback - DEBUG - Processing node 69b10f929d191758182534a3: baseline-arm-clabbe 2026-03-13 06:45:59,406 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b10f929d191758182534a3 2026-03-13 06:45:59,407 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 06:45:59,423 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b10f929d191758182534a3 HTTP/1.1" 200 None 2026-03-13 06:45:59,424 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 06:45:59,439 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b4fc9d19175818251fd6 HTTP/1.1" 200 None 2026-03-13 06:45:59,440 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 06:45:59,460 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b10f929d191758182534a3 HTTP/1.1" 200 None 2026-03-13 06:45:59,461 - lava_callback - ERROR - Error processing callback for node 69b10f929d191758182534a3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:40968 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55900 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55912 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55926 - "GET / HTTP/1.0" 200 OK 2026-03-13 06:46:19,606 - lava_callback - INFO - Received callback for node 69b10fb79d191758182534b0 INFO: 172.20.0.1:55930 - "POST /node/69b10fb79d191758182534b0 HTTP/1.0" 202 Accepted 2026-03-13 06:46:19,611 - lava_callback - INFO - Starting async job submit for node 69b10fb79d191758182534b0 2026-03-13 06:46:19,612 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 06:46:19,627 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b10fb79d191758182534b0 HTTP/1.1" 200 None 2026-03-13 06:46:19,627 - lava_callback - DEBUG - Processing node 69b10fb79d191758182534b0: baseline-arm-clabbe 2026-03-13 06:46:19,628 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b10fb79d191758182534b0 2026-03-13 06:46:19,628 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 06:46:19,641 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b10fb79d191758182534b0 HTTP/1.1" 200 None 2026-03-13 06:46:19,643 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 06:46:19,656 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b4fc9d19175818251fd6 HTTP/1.1" 200 None 2026-03-13 06:46:19,657 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 06:46:19,678 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b10fb79d191758182534b0 HTTP/1.1" 200 None 2026-03-13 06:46:19,682 - lava_callback - ERROR - Error processing callback for node 69b10fb79d191758182534b0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:33864 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57290 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48858 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40760 - "GET / HTTP/1.0" 200 OK 2026-03-13 06:47:05,634 - lava_callback - INFO - Received callback for node 69b10fd79d191758182534be INFO: 172.20.0.1:59644 - "POST /node/69b10fd79d191758182534be HTTP/1.0" 202 Accepted 2026-03-13 06:47:05,639 - lava_callback - INFO - Starting async job submit for node 69b10fd79d191758182534be 2026-03-13 06:47:05,640 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 06:47:05,655 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b10fd79d191758182534be HTTP/1.1" 200 None 2026-03-13 06:47:05,656 - lava_callback - DEBUG - Processing node 69b10fd79d191758182534be: baseline-arm-baylibre 2026-03-13 06:47:05,656 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b10fd79d191758182534be 2026-03-13 06:47:05,656 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 06:47:05,689 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b10fd79d191758182534be HTTP/1.1" 200 None 2026-03-13 06:47:05,690 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 06:47:05,703 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11668ef4424e487b2ea2 HTTP/1.1" 200 None 2026-03-13 06:47:05,705 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 06:47:05,725 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b10fd79d191758182534be HTTP/1.1" 200 None 2026-03-13 06:47:05,726 - lava_callback - ERROR - Error processing callback for node 69b10fd79d191758182534be: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:59656 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55688 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55702 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55710 - "GET / HTTP/1.0" 200 OK 2026-03-13 06:47:25,590 - lava_callback - INFO - Received callback for node 69b10ffa9d191758182534cb INFO: 172.20.0.1:55512 - "POST /node/69b10ffa9d191758182534cb HTTP/1.0" 202 Accepted 2026-03-13 06:47:25,596 - lava_callback - INFO - Starting async job submit for node 69b10ffa9d191758182534cb 2026-03-13 06:47:25,597 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 06:47:25,612 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b10ffa9d191758182534cb HTTP/1.1" 200 None 2026-03-13 06:47:25,613 - lava_callback - DEBUG - Processing node 69b10ffa9d191758182534cb: baseline-arm-baylibre 2026-03-13 06:47:25,613 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b10ffa9d191758182534cb 2026-03-13 06:47:25,614 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 06:47:25,632 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b10ffa9d191758182534cb HTTP/1.1" 200 None 2026-03-13 06:47:25,633 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 06:47:25,647 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11668ef4424e487b2ea2 HTTP/1.1" 200 None 2026-03-13 06:47:25,649 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 06:47:25,679 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b10ffa9d191758182534cb HTTP/1.1" 200 None 2026-03-13 06:47:25,680 - lava_callback - ERROR - Error processing callback for node 69b10ffa9d191758182534cb: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:55526 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38850 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50162 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40498 - "GET / HTTP/1.0" 200 OK 2026-03-13 06:48:05,622 - lava_callback - INFO - Received callback for node 69b1101e9d191758182534dd INFO: 172.20.0.1:55216 - "POST /node/69b1101e9d191758182534dd HTTP/1.0" 202 Accepted 2026-03-13 06:48:05,627 - lava_callback - INFO - Starting async job submit for node 69b1101e9d191758182534dd 2026-03-13 06:48:05,628 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 06:48:05,644 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b1101e9d191758182534dd HTTP/1.1" 200 None 2026-03-13 06:48:05,645 - lava_callback - DEBUG - Processing node 69b1101e9d191758182534dd: baseline-arm-baylibre 2026-03-13 06:48:05,645 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b1101e9d191758182534dd 2026-03-13 06:48:05,645 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 06:48:05,658 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b1101e9d191758182534dd HTTP/1.1" 200 None 2026-03-13 06:48:05,660 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 06:48:05,673 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11668ef4424e487b2ea2 HTTP/1.1" 200 None 2026-03-13 06:48:05,674 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 06:48:05,694 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b1101e9d191758182534dd HTTP/1.1" 200 None 2026-03-13 06:48:05,700 - lava_callback - ERROR - Error processing callback for node 69b1101e9d191758182534dd: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:55226 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47706 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47720 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47726 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43658 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54198 - "GET / HTTP/1.0" 200 OK 2026-03-13 06:48:45,724 - lava_callback - INFO - Received callback for node 69b110439d191758182534e9 INFO: 172.20.0.1:33664 - "POST /node/69b110439d191758182534e9 HTTP/1.0" 202 Accepted 2026-03-13 06:48:45,730 - lava_callback - INFO - Starting async job submit for node 69b110439d191758182534e9 2026-03-13 06:48:45,731 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 06:48:45,745 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b110439d191758182534e9 HTTP/1.1" 200 None 2026-03-13 06:48:45,747 - lava_callback - DEBUG - Processing node 69b110439d191758182534e9: baseline-arm-baylibre 2026-03-13 06:48:45,747 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b110439d191758182534e9 2026-03-13 06:48:45,748 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 06:48:45,763 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b110439d191758182534e9 HTTP/1.1" 200 None 2026-03-13 06:48:45,765 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 06:48:45,781 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11668ef4424e487b2ea2 HTTP/1.1" 200 None 2026-03-13 06:48:45,783 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 06:48:45,809 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b110439d191758182534e9 HTTP/1.1" 200 None 2026-03-13 06:48:45,815 - lava_callback - ERROR - Error processing callback for node 69b110439d191758182534e9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:33678 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51466 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47454 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56286 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56296 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56302 - "GET / HTTP/1.0" 200 OK 2026-03-13 06:49:25,694 - lava_callback - INFO - Received callback for node 69b110689d191758182534f5 INFO: 172.20.0.1:47092 - "POST /node/69b110689d191758182534f5 HTTP/1.0" 202 Accepted 2026-03-13 06:49:25,700 - lava_callback - INFO - Starting async job submit for node 69b110689d191758182534f5 2026-03-13 06:49:25,701 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 06:49:25,786 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b110689d191758182534f5 HTTP/1.1" 200 None 2026-03-13 06:49:25,787 - lava_callback - DEBUG - Processing node 69b110689d191758182534f5: baseline-arm-baylibre 2026-03-13 06:49:25,787 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b110689d191758182534f5 2026-03-13 06:49:25,788 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 06:49:25,802 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b110689d191758182534f5 HTTP/1.1" 200 None 2026-03-13 06:49:25,803 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 06:49:25,816 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11668ef4424e487b2ea2 HTTP/1.1" 200 None 2026-03-13 06:49:25,818 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 06:49:25,839 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b110689d191758182534f5 HTTP/1.1" 200 None 2026-03-13 06:49:25,843 - lava_callback - ERROR - Error processing callback for node 69b110689d191758182534f5: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:47106 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60068 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53324 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59630 - "GET / HTTP/1.0" 200 OK 2026-03-13 06:50:05,750 - lava_callback - INFO - Received callback for node 69b110899d19175818253505 INFO: 172.20.0.1:46504 - "POST /node/69b110899d19175818253505 HTTP/1.0" 202 Accepted 2026-03-13 06:50:05,756 - lava_callback - INFO - Starting async job submit for node 69b110899d19175818253505 2026-03-13 06:50:05,757 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 06:50:05,781 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b110899d19175818253505 HTTP/1.1" 200 None 2026-03-13 06:50:05,782 - lava_callback - DEBUG - Processing node 69b110899d19175818253505: baseline-arm-baylibre 2026-03-13 06:50:05,782 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b110899d19175818253505 2026-03-13 06:50:05,783 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 06:50:05,805 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b110899d19175818253505 HTTP/1.1" 200 None 2026-03-13 06:50:05,806 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 06:50:05,820 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11668ef4424e487b2ea2 HTTP/1.1" 200 None 2026-03-13 06:50:05,822 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 06:50:05,842 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b110899d19175818253505 HTTP/1.1" 200 None 2026-03-13 06:50:05,848 - lava_callback - ERROR - Error processing callback for node 69b110899d19175818253505: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:46506 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32796 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32810 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:32826 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44466 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32982 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59288 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50356 - "GET / HTTP/1.0" 200 OK 2026-03-13 06:51:05,803 - lava_callback - INFO - Received callback for node 69b110d29d19175818253525 INFO: 172.20.0.1:38760 - "POST /node/69b110d29d19175818253525 HTTP/1.0" 202 Accepted 2026-03-13 06:51:05,809 - lava_callback - INFO - Starting async job submit for node 69b110d29d19175818253525 2026-03-13 06:51:05,810 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 06:51:05,832 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b110d29d19175818253525 HTTP/1.1" 200 None 2026-03-13 06:51:05,832 - lava_callback - DEBUG - Processing node 69b110d29d19175818253525: baseline-arm-baylibre 2026-03-13 06:51:05,833 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b110d29d19175818253525 2026-03-13 06:51:05,833 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 06:51:05,846 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b110d29d19175818253525 HTTP/1.1" 200 None 2026-03-13 06:51:05,848 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 06:51:05,862 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11668ef4424e487b2ea2 HTTP/1.1" 200 None 2026-03-13 06:51:05,864 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 06:51:05,892 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b110d29d19175818253525 HTTP/1.1" 200 None 2026-03-13 06:51:05,893 - lava_callback - ERROR - Error processing callback for node 69b110d29d19175818253525: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:38762 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38092 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38104 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38106 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42112 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51310 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53214 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56222 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47332 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51586 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51602 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51614 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40726 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59700 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36326 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49564 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60704 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46964 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46974 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46986 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47294 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57898 - "GET / HTTP/1.0" 200 OK 2026-03-13 06:53:39,717 - lava_callback - INFO - Received callback for node 69b1115f9d19175818253565 INFO: 172.20.0.1:59952 - "POST /node/69b1115f9d19175818253565 HTTP/1.0" 202 Accepted 2026-03-13 06:53:39,722 - lava_callback - INFO - Starting async job submit for node 69b1115f9d19175818253565 2026-03-13 06:53:39,723 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 06:53:39,741 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b1115f9d19175818253565 HTTP/1.1" 200 None 2026-03-13 06:53:39,742 - lava_callback - DEBUG - Processing node 69b1115f9d19175818253565: baseline-arm-clabbe 2026-03-13 06:53:39,742 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b1115f9d19175818253565 2026-03-13 06:53:39,743 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 06:53:39,759 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b1115f9d19175818253565 HTTP/1.1" 200 None 2026-03-13 06:53:39,761 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 06:53:39,775 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11668ef4424e487b2ea2 HTTP/1.1" 200 None 2026-03-13 06:53:39,776 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 06:53:39,797 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b1115f9d19175818253565 HTTP/1.1" 200 None 2026-03-13 06:53:39,806 - lava_callback - ERROR - Error processing callback for node 69b1115f9d19175818253565: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:59958 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42416 - "GET / HTTP/1.0" 200 OK 2026-03-13 06:53:59,782 - lava_callback - INFO - Received callback for node 69b111849d19175818253578 INFO: 172.20.0.1:40396 - "POST /node/69b111849d19175818253578 HTTP/1.0" 202 Accepted 2026-03-13 06:53:59,788 - lava_callback - INFO - Starting async job submit for node 69b111849d19175818253578 2026-03-13 06:53:59,789 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 06:53:59,812 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b111849d19175818253578 HTTP/1.1" 200 None 2026-03-13 06:53:59,813 - lava_callback - DEBUG - Processing node 69b111849d19175818253578: baseline-arm-clabbe 2026-03-13 06:53:59,813 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b111849d19175818253578 2026-03-13 06:53:59,813 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 06:53:59,838 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b111849d19175818253578 HTTP/1.1" 200 None 2026-03-13 06:53:59,839 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 06:53:59,852 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11668ef4424e487b2ea2 HTTP/1.1" 200 None 2026-03-13 06:53:59,854 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 06:53:59,875 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b111849d19175818253578 HTTP/1.1" 200 None 2026-03-13 06:53:59,886 - lava_callback - ERROR - Error processing callback for node 69b111849d19175818253578: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:40404 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59498 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59504 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59516 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57128 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52688 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44708 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35490 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50760 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55186 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55202 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55218 - "GET / HTTP/1.0" 200 OK 2026-03-13 06:55:19,763 - lava_callback - INFO - Received callback for node 69b111cb9d1917581825359b INFO: 172.20.0.1:54638 - "POST /node/69b111cb9d1917581825359b HTTP/1.0" 202 Accepted 2026-03-13 06:55:19,768 - lava_callback - INFO - Starting async job submit for node 69b111cb9d1917581825359b 2026-03-13 06:55:19,769 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 06:55:19,783 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b111cb9d1917581825359b HTTP/1.1" 200 None 2026-03-13 06:55:19,784 - lava_callback - DEBUG - Processing node 69b111cb9d1917581825359b: baseline-arm-clabbe 2026-03-13 06:55:19,785 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b111cb9d1917581825359b 2026-03-13 06:55:19,785 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 06:55:19,807 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b111cb9d1917581825359b HTTP/1.1" 200 None 2026-03-13 06:55:19,808 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 06:55:19,822 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11668ef4424e487b2ea2 HTTP/1.1" 200 None 2026-03-13 06:55:19,823 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 06:55:19,844 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b111cb9d1917581825359b HTTP/1.1" 200 None 2026-03-13 06:55:19,845 - lava_callback - ERROR - Error processing callback for node 69b111cb9d1917581825359b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:54650 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53444 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51568 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51618 - "GET / HTTP/1.0" 200 OK 2026-03-13 06:55:59,760 - lava_callback - INFO - Received callback for node 69b111ef9d191758182535a6 INFO: 172.20.0.1:53316 - "POST /node/69b111ef9d191758182535a6 HTTP/1.0" 202 Accepted 2026-03-13 06:55:59,766 - lava_callback - INFO - Starting async job submit for node 69b111ef9d191758182535a6 2026-03-13 06:55:59,767 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 06:55:59,781 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b111ef9d191758182535a6 HTTP/1.1" 200 None 2026-03-13 06:55:59,782 - lava_callback - DEBUG - Processing node 69b111ef9d191758182535a6: baseline-arm-clabbe 2026-03-13 06:55:59,782 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b111ef9d191758182535a6 2026-03-13 06:55:59,783 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 06:55:59,801 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b111ef9d191758182535a6 HTTP/1.1" 200 None 2026-03-13 06:55:59,803 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 06:55:59,817 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11668ef4424e487b2ea2 HTTP/1.1" 200 None 2026-03-13 06:55:59,818 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 06:55:59,838 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b111ef9d191758182535a6 HTTP/1.1" 200 None 2026-03-13 06:55:59,845 - lava_callback - ERROR - Error processing callback for node 69b111ef9d191758182535a6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:53326 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34288 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34296 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34310 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47122 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56820 - "GET / HTTP/1.0" 200 OK 2026-03-13 06:56:40,257 - lava_callback - INFO - Received callback for node 69b112139d191758182535b3 INFO: 172.20.0.1:40568 - "POST /node/69b112139d191758182535b3 HTTP/1.0" 202 Accepted 2026-03-13 06:56:40,262 - lava_callback - INFO - Starting async job submit for node 69b112139d191758182535b3 2026-03-13 06:56:40,263 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 06:56:40,279 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b112139d191758182535b3 HTTP/1.1" 200 None 2026-03-13 06:56:40,280 - lava_callback - DEBUG - Processing node 69b112139d191758182535b3: baseline-arm-clabbe 2026-03-13 06:56:40,280 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b112139d191758182535b3 2026-03-13 06:56:40,281 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 06:56:40,295 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b112139d191758182535b3 HTTP/1.1" 200 None 2026-03-13 06:56:40,296 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 06:56:40,314 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11668ef4424e487b2ea2 HTTP/1.1" 200 None 2026-03-13 06:56:40,316 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 06:56:40,338 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b112139d191758182535b3 HTTP/1.1" 200 None 2026-03-13 06:56:40,341 - lava_callback - ERROR - Error processing callback for node 69b112139d191758182535b3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:40572 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48038 - "GET / HTTP/1.0" 200 OK 2026-03-13 06:56:59,760 - lava_callback - INFO - Received callback for node 69b112389d191758182535bd INFO: 172.20.0.1:38648 - "POST /node/69b112389d191758182535bd HTTP/1.0" 202 Accepted 2026-03-13 06:56:59,765 - lava_callback - INFO - Starting async job submit for node 69b112389d191758182535bd 2026-03-13 06:56:59,766 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 06:56:59,781 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b112389d191758182535bd HTTP/1.1" 200 None 2026-03-13 06:56:59,782 - lava_callback - DEBUG - Processing node 69b112389d191758182535bd: baseline-arm-clabbe 2026-03-13 06:56:59,782 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b112389d191758182535bd 2026-03-13 06:56:59,783 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 06:56:59,799 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b112389d191758182535bd HTTP/1.1" 200 None 2026-03-13 06:56:59,800 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 06:56:59,813 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11668ef4424e487b2ea2 HTTP/1.1" 200 None 2026-03-13 06:56:59,815 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 06:56:59,841 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b112389d191758182535bd HTTP/1.1" 200 None 2026-03-13 06:56:59,842 - lava_callback - ERROR - Error processing callback for node 69b112389d191758182535bd: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:38660 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50426 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50434 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50450 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59280 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54158 - "GET / HTTP/1.0" 200 OK 2026-03-13 06:57:39,843 - lava_callback - INFO - Received callback for node 69b1125c9d191758182535c7 INFO: 172.20.0.1:41108 - "POST /node/69b1125c9d191758182535c7 HTTP/1.0" 202 Accepted 2026-03-13 06:57:39,849 - lava_callback - INFO - Starting async job submit for node 69b1125c9d191758182535c7 2026-03-13 06:57:39,850 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 06:57:39,864 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b1125c9d191758182535c7 HTTP/1.1" 200 None 2026-03-13 06:57:39,865 - lava_callback - DEBUG - Processing node 69b1125c9d191758182535c7: baseline-arm-clabbe 2026-03-13 06:57:39,865 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b1125c9d191758182535c7 2026-03-13 06:57:39,866 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 06:57:39,905 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b1125c9d191758182535c7 HTTP/1.1" 200 None 2026-03-13 06:57:39,907 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 06:57:39,920 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11668ef4424e487b2ea2 HTTP/1.1" 200 None 2026-03-13 06:57:39,921 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 06:57:39,949 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b1125c9d191758182535c7 HTTP/1.1" 200 None 2026-03-13 06:57:39,951 - lava_callback - ERROR - Error processing callback for node 69b1125c9d191758182535c7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:41114 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43668 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59194 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48194 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48202 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48208 - "GET / HTTP/1.0" 200 OK 2026-03-13 06:58:19,845 - lava_callback - INFO - Received callback for node 69b112809d191758182535e5 INFO: 172.20.0.1:37036 - "POST /node/69b112809d191758182535e5 HTTP/1.0" 202 Accepted 2026-03-13 06:58:19,851 - lava_callback - INFO - Starting async job submit for node 69b112809d191758182535e5 2026-03-13 06:58:19,852 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 06:58:19,865 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b112809d191758182535e5 HTTP/1.1" 200 None 2026-03-13 06:58:19,866 - lava_callback - DEBUG - Processing node 69b112809d191758182535e5: baseline-arm-clabbe 2026-03-13 06:58:19,866 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b112809d191758182535e5 2026-03-13 06:58:19,867 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 06:58:19,884 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b112809d191758182535e5 HTTP/1.1" 200 None 2026-03-13 06:58:19,886 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 06:58:19,899 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11668ef4424e487b2ea2 HTTP/1.1" 200 None 2026-03-13 06:58:19,901 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 06:58:19,921 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b112809d191758182535e5 HTTP/1.1" 200 None 2026-03-13 06:58:19,930 - lava_callback - ERROR - Error processing callback for node 69b112809d191758182535e5: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:37048 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34634 - "GET / HTTP/1.0" 200 OK 2026-03-13 06:58:46,242 - lava_callback - INFO - Received callback for node 69b112a09d191758182535ec INFO: 172.20.0.1:40594 - "POST /node/69b112a09d191758182535ec HTTP/1.0" 202 Accepted 2026-03-13 06:58:46,248 - lava_callback - INFO - Starting async job submit for node 69b112a09d191758182535ec 2026-03-13 06:58:46,248 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 06:58:46,269 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b112a09d191758182535ec HTTP/1.1" 200 None 2026-03-13 06:58:46,270 - lava_callback - DEBUG - Processing node 69b112a09d191758182535ec: baseline-arm-baylibre 2026-03-13 06:58:46,270 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b112a09d191758182535ec 2026-03-13 06:58:46,271 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 06:58:46,285 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b112a09d191758182535ec HTTP/1.1" 200 None 2026-03-13 06:58:46,286 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 06:58:46,300 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e34efa461f55e59cdb7 HTTP/1.1" 200 None 2026-03-13 06:58:46,301 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 06:58:46,321 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b112a09d191758182535ec HTTP/1.1" 200 None 2026-03-13 06:58:46,328 - lava_callback - ERROR - Error processing callback for node 69b112a09d191758182535ec: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:40596 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39450 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46536 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42604 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42610 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42626 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35156 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58108 - "GET / HTTP/1.0" 200 OK 2026-03-13 06:59:39,914 - lava_callback - INFO - Received callback for node 69b112c89d1917581825360a INFO: 172.20.0.1:38090 - "POST /node/69b112c89d1917581825360a HTTP/1.0" 202 Accepted 2026-03-13 06:59:39,919 - lava_callback - INFO - Starting async job submit for node 69b112c89d1917581825360a 2026-03-13 06:59:39,920 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 06:59:39,935 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b112c89d1917581825360a HTTP/1.1" 200 None 2026-03-13 06:59:39,935 - lava_callback - DEBUG - Processing node 69b112c89d1917581825360a: baseline-arm-clabbe 2026-03-13 06:59:39,936 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b112c89d1917581825360a 2026-03-13 06:59:39,936 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 06:59:39,949 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b112c89d1917581825360a HTTP/1.1" 200 None 2026-03-13 06:59:39,950 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 06:59:39,964 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11668ef4424e487b2ea2 HTTP/1.1" 200 None 2026-03-13 06:59:39,966 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 06:59:39,991 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b112c89d1917581825360a HTTP/1.1" 200 None 2026-03-13 06:59:39,992 - lava_callback - ERROR - Error processing callback for node 69b112c89d1917581825360a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:38096 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59154 - "GET / HTTP/1.0" 200 OK 2026-03-13 07:00:06,274 - lava_callback - INFO - Received callback for node 69b112e89d19175818253621 INFO: 172.20.0.1:35962 - "POST /node/69b112e89d19175818253621 HTTP/1.0" 202 Accepted 2026-03-13 07:00:06,280 - lava_callback - INFO - Starting async job submit for node 69b112e89d19175818253621 2026-03-13 07:00:06,281 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:00:06,296 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b112e89d19175818253621 HTTP/1.1" 200 None 2026-03-13 07:00:06,296 - lava_callback - DEBUG - Processing node 69b112e89d19175818253621: baseline-arm-baylibre 2026-03-13 07:00:06,297 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b112e89d19175818253621 2026-03-13 07:00:06,297 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:00:06,311 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b112e89d19175818253621 HTTP/1.1" 200 None 2026-03-13 07:00:06,312 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:00:06,326 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e34efa461f55e59cdb7 HTTP/1.1" 200 None 2026-03-13 07:00:06,328 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:00:06,360 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b112e89d19175818253621 HTTP/1.1" 200 None 2026-03-13 07:00:06,361 - lava_callback - ERROR - Error processing callback for node 69b112e89d19175818253621: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:35976 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35042 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35050 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35060 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45224 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53194 - "GET / HTTP/1.0" 200 OK 2026-03-13 07:00:46,283 - lava_callback - INFO - Received callback for node 69b1130c9d19175818253634 INFO: 172.20.0.1:40368 - "POST /node/69b1130c9d19175818253634 HTTP/1.0" 202 Accepted 2026-03-13 07:00:46,289 - lava_callback - INFO - Starting async job submit for node 69b1130c9d19175818253634 2026-03-13 07:00:46,289 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:00:46,310 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b1130c9d19175818253634 HTTP/1.1" 200 None 2026-03-13 07:00:46,311 - lava_callback - DEBUG - Processing node 69b1130c9d19175818253634: baseline-arm-baylibre 2026-03-13 07:00:46,311 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b1130c9d19175818253634 2026-03-13 07:00:46,312 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:00:46,325 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b1130c9d19175818253634 HTTP/1.1" 200 None 2026-03-13 07:00:46,327 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:00:46,341 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e34efa461f55e59cdb7 HTTP/1.1" 200 None 2026-03-13 07:00:46,342 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:00:46,369 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b1130c9d19175818253634 HTTP/1.1" 200 None 2026-03-13 07:00:46,370 - lava_callback - ERROR - Error processing callback for node 69b1130c9d19175818253634: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:40374 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34560 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50502 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35022 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35036 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35048 - "GET / HTTP/1.0" 200 OK 2026-03-13 07:01:20,052 - lava_callback - INFO - Received callback for node 69b113349d19175818253650 INFO: 172.20.0.1:32910 - "POST /node/69b113349d19175818253650 HTTP/1.0" 202 Accepted 2026-03-13 07:01:20,058 - lava_callback - INFO - Starting async job submit for node 69b113349d19175818253650 2026-03-13 07:01:20,059 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:01:20,073 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b113349d19175818253650 HTTP/1.1" 200 None 2026-03-13 07:01:20,074 - lava_callback - DEBUG - Processing node 69b113349d19175818253650: baseline-arm-clabbe 2026-03-13 07:01:20,074 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b113349d19175818253650 2026-03-13 07:01:20,075 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:01:20,088 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b113349d19175818253650 HTTP/1.1" 200 None 2026-03-13 07:01:20,090 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:01:20,104 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11668ef4424e487b2ea2 HTTP/1.1" 200 None 2026-03-13 07:01:20,106 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:01:20,132 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b113349d19175818253650 HTTP/1.1" 200 None 2026-03-13 07:01:20,133 - lava_callback - ERROR - Error processing callback for node 69b113349d19175818253650: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:32920 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42888 - "GET / HTTP/1.0" 200 OK 2026-03-13 07:01:42,838 - lava_callback - INFO - Received callback for node 69b363ca974c91e64560ed5e INFO: 172.20.0.1:33656 - "POST /node/69b363ca974c91e64560ed5e HTTP/1.0" 202 Accepted 2026-03-13 07:01:42,844 - lava_callback - INFO - Starting async job submit for node 69b363ca974c91e64560ed5e 2026-03-13 07:01:42,866 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:01:42,891 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b363ca974c91e64560ed5e HTTP/1.1" 200 None 2026-03-13 07:01:42,892 - lava_callback - DEBUG - Processing node 69b363ca974c91e64560ed5e: baseline-arm64-broonie 2026-03-13 07:01:43,165 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 07:01:43,326 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 07:01:43,327 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69b363ca974c91e64560ed5e/log.txt.gz 2026-03-13 07:01:43,497 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 07:01:43,836 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 07:01:43,838 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69b363ca974c91e64560ed5e/lava-logs.yaml 2026-03-13 07:01:43,845 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 07:01:44,150 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 07:01:44,152 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69b363ca974c91e64560ed5e/lava_callback.json.gz 2026-03-13 07:01:44,174 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:01:44,188 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b363ca974c91e64560ed5e HTTP/1.1" 200 None 2026-03-13 07:01:44,190 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:01:44,207 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3594d974c91e64560e4b2 HTTP/1.1" 200 None 2026-03-13 07:01:44,209 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:01:44,233 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b363ca974c91e64560ed5e HTTP/1.1" 200 None 2026-03-13 07:01:44,239 - lava_callback - INFO - Completed processing callback for node 69b363ca974c91e64560ed5e 2026-03-13 07:01:46,379 - lava_callback - INFO - Received callback for node 69b113559d1917581825365b INFO: 172.20.0.1:33672 - "POST /node/69b113559d1917581825365b HTTP/1.0" 202 Accepted 2026-03-13 07:01:46,384 - lava_callback - INFO - Starting async job submit for node 69b113559d1917581825365b 2026-03-13 07:01:46,385 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:01:46,411 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b113559d1917581825365b HTTP/1.1" 200 None 2026-03-13 07:01:46,412 - lava_callback - DEBUG - Processing node 69b113559d1917581825365b: baseline-arm-baylibre 2026-03-13 07:01:46,412 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b113559d1917581825365b 2026-03-13 07:01:46,413 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:01:46,436 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b113559d1917581825365b HTTP/1.1" 200 None 2026-03-13 07:01:46,437 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:01:46,451 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e34efa461f55e59cdb7 HTTP/1.1" 200 None 2026-03-13 07:01:46,452 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:01:46,458 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:01:46,478 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-13 07:01:46,484 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b113559d1917581825365b HTTP/1.1" 200 None 2026-03-13 07:01:46,486 - lava_callback - ERROR - Error processing callback for node 69b113559d1917581825365b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:33678 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44134 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51144 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50662 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50674 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50684 - "GET / HTTP/1.0" 200 OK 2026-03-13 07:02:26,388 - lava_callback - INFO - Received callback for node 69b113799d19175818253668 INFO: 172.20.0.1:43164 - "POST /node/69b113799d19175818253668 HTTP/1.0" 202 Accepted 2026-03-13 07:02:26,393 - lava_callback - INFO - Starting async job submit for node 69b113799d19175818253668 2026-03-13 07:02:26,394 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:02:26,409 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b113799d19175818253668 HTTP/1.1" 200 None 2026-03-13 07:02:26,409 - lava_callback - DEBUG - Processing node 69b113799d19175818253668: baseline-arm-baylibre 2026-03-13 07:02:26,410 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b113799d19175818253668 2026-03-13 07:02:26,410 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:02:26,424 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b113799d19175818253668 HTTP/1.1" 200 None 2026-03-13 07:02:26,426 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:02:26,439 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e34efa461f55e59cdb7 HTTP/1.1" 200 None 2026-03-13 07:02:26,441 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:02:26,469 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b113799d19175818253668 HTTP/1.1" 200 None 2026-03-13 07:02:26,470 - lava_callback - ERROR - Error processing callback for node 69b113799d19175818253668: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:43172 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36570 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50340 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49500 - "GET / HTTP/1.0" 200 OK 2026-03-13 07:03:00,068 - lava_callback - INFO - Received callback for node 69b1139e9d19175818253679 INFO: 172.20.0.1:59066 - "POST /node/69b1139e9d19175818253679 HTTP/1.0" 202 Accepted 2026-03-13 07:03:00,074 - lava_callback - INFO - Starting async job submit for node 69b1139e9d19175818253679 2026-03-13 07:03:00,075 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:03:00,091 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b1139e9d19175818253679 HTTP/1.1" 200 None 2026-03-13 07:03:00,092 - lava_callback - DEBUG - Processing node 69b1139e9d19175818253679: baseline-arm-clabbe 2026-03-13 07:03:00,093 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b1139e9d19175818253679 2026-03-13 07:03:00,093 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:03:00,107 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b1139e9d19175818253679 HTTP/1.1" 200 None 2026-03-13 07:03:00,108 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:03:00,121 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e34efa461f55e59cdb7 HTTP/1.1" 200 None 2026-03-13 07:03:00,123 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:03:00,151 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b1139e9d19175818253679 HTTP/1.1" 200 None 2026-03-13 07:03:00,153 - lava_callback - ERROR - Error processing callback for node 69b1139e9d19175818253679: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:59078 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51330 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51342 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51350 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34048 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54850 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33916 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40170 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52450 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50648 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50652 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50654 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45878 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38912 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35822 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33024 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49010 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41936 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41946 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41948 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45916 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42230 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55838 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44656 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56172 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55132 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55144 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55148 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43864 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40110 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40142 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34178 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37300 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36684 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36686 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36692 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42102 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45842 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47274 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57636 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50940 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54848 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54860 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54866 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39628 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41528 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54568 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53886 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43402 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41682 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41696 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41710 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35120 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57394 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44968 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39368 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50418 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50266 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50270 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50274 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47278 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35242 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52954 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56272 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33668 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36946 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36960 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36962 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60716 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43960 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51308 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49190 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34062 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59604 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59616 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59622 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49664 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46254 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41354 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54318 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60876 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38440 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38444 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38458 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37292 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51776 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56884 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47700 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44230 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58512 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58526 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58530 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60324 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46566 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48180 - "GET / HTTP/1.0" 200 OK 2026-03-13 07:14:47,129 - lava_callback - INFO - Received callback for node 69b116629d19175818253708 INFO: 172.20.0.1:48188 - "POST /node/69b116629d19175818253708 HTTP/1.0" 202 Accepted 2026-03-13 07:14:47,137 - lava_callback - INFO - Starting async job submit for node 69b116629d19175818253708 2026-03-13 07:14:47,139 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:14:47,156 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b116629d19175818253708 HTTP/1.1" 200 None 2026-03-13 07:14:47,158 - lava_callback - DEBUG - Processing node 69b116629d19175818253708: baseline-arm64-baylibre 2026-03-13 07:14:47,158 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b116629d19175818253708 2026-03-13 07:14:47,159 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:14:47,174 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b116629d19175818253708 HTTP/1.1" 200 None 2026-03-13 07:14:47,176 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:14:47,189 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b5149d19175818251feb HTTP/1.1" 200 None 2026-03-13 07:14:47,190 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:14:47,210 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b116629d19175818253708 HTTP/1.1" 200 None 2026-03-13 07:14:47,212 - lava_callback - ERROR - Error processing callback for node 69b116629d19175818253708: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:52350 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51462 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38846 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38850 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38858 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46846 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35614 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60946 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54444 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37000 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46094 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46102 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46104 - "GET / HTTP/1.0" 200 OK 2026-03-13 07:16:20,645 - lava_callback - INFO - Received callback for node 69b116b39d19175818253715 INFO: 172.20.0.1:49102 - "POST /node/69b116b39d19175818253715 HTTP/1.0" 202 Accepted 2026-03-13 07:16:20,650 - lava_callback - INFO - Starting async job submit for node 69b116b39d19175818253715 2026-03-13 07:16:20,651 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:16:20,665 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b116b39d19175818253715 HTTP/1.1" 200 None 2026-03-13 07:16:20,666 - lava_callback - DEBUG - Processing node 69b116b39d19175818253715: baseline-arm64-clabbe 2026-03-13 07:16:20,667 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b116b39d19175818253715 2026-03-13 07:16:20,667 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:16:20,682 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b116b39d19175818253715 HTTP/1.1" 200 None 2026-03-13 07:16:20,683 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:16:20,702 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b5149d19175818251feb HTTP/1.1" 200 None 2026-03-13 07:16:20,704 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:16:20,727 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b116b39d19175818253715 HTTP/1.1" 200 None 2026-03-13 07:16:20,737 - lava_callback - ERROR - Error processing callback for node 69b116b39d19175818253715: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:49104 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49224 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46110 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57312 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36490 - "GET / HTTP/1.0" 200 OK 2026-03-13 07:17:07,317 - lava_callback - INFO - Received callback for node 69b116eb9d19175818253722 INFO: 172.20.0.1:36502 - "POST /node/69b116eb9d19175818253722 HTTP/1.0" 202 Accepted 2026-03-13 07:17:07,323 - lava_callback - INFO - Starting async job submit for node 69b116eb9d19175818253722 2026-03-13 07:17:07,324 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:17:07,338 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b116eb9d19175818253722 HTTP/1.1" 200 None 2026-03-13 07:17:07,339 - lava_callback - DEBUG - Processing node 69b116eb9d19175818253722: baseline-arm-baylibre 2026-03-13 07:17:07,340 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b116eb9d19175818253722 2026-03-13 07:17:07,340 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:17:07,357 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b116eb9d19175818253722 HTTP/1.1" 200 None 2026-03-13 07:17:07,358 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:17:07,371 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e34efa461f55e59cdb7 HTTP/1.1" 200 None 2026-03-13 07:17:07,373 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:17:07,395 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b116eb9d19175818253722 HTTP/1.1" 200 None 2026-03-13 07:17:07,398 - lava_callback - ERROR - Error processing callback for node 69b116eb9d19175818253722: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:60402 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60418 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60430 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42330 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36676 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40046 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45682 - "GET / HTTP/1.0" 200 OK 2026-03-13 07:18:00,661 - lava_callback - INFO - Received callback for node 69b117219d1917581825372f INFO: 172.20.0.1:46018 - "POST /node/69b117219d1917581825372f HTTP/1.0" 202 Accepted 2026-03-13 07:18:00,667 - lava_callback - INFO - Starting async job submit for node 69b117219d1917581825372f 2026-03-13 07:18:00,668 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:18:00,698 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b117219d1917581825372f HTTP/1.1" 200 None 2026-03-13 07:18:00,699 - lava_callback - DEBUG - Processing node 69b117219d1917581825372f: baseline-arm64-clabbe 2026-03-13 07:18:00,699 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b117219d1917581825372f 2026-03-13 07:18:00,700 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:18:00,717 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b117219d1917581825372f HTTP/1.1" 200 None 2026-03-13 07:18:00,718 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:18:00,732 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b5149d19175818251feb HTTP/1.1" 200 None 2026-03-13 07:18:00,734 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:18:00,755 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b117219d1917581825372f HTTP/1.1" 200 None 2026-03-13 07:18:00,762 - lava_callback - ERROR - Error processing callback for node 69b117219d1917581825372f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:46032 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46698 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46712 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46722 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39334 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44136 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35518 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32998 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32934 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41994 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42002 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42004 - "GET / HTTP/1.0" 200 OK 2026-03-13 07:19:20,745 - lava_callback - INFO - Received callback for node 69b1176d9d19175818253744 INFO: 172.20.0.1:52912 - "POST /node/69b1176d9d19175818253744 HTTP/1.0" 202 Accepted 2026-03-13 07:19:20,750 - lava_callback - INFO - Starting async job submit for node 69b1176d9d19175818253744 2026-03-13 07:19:20,751 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:19:20,775 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b1176d9d19175818253744 HTTP/1.1" 200 None 2026-03-13 07:19:20,776 - lava_callback - DEBUG - Processing node 69b1176d9d19175818253744: baseline-arm64-clabbe 2026-03-13 07:19:20,777 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b1176d9d19175818253744 2026-03-13 07:19:20,777 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:19:20,791 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b1176d9d19175818253744 HTTP/1.1" 200 None 2026-03-13 07:19:20,793 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:19:20,806 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b5149d19175818251feb HTTP/1.1" 200 None 2026-03-13 07:19:20,808 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:19:20,828 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b1176d9d19175818253744 HTTP/1.1" 200 None 2026-03-13 07:19:20,830 - lava_callback - ERROR - Error processing callback for node 69b1176d9d19175818253744: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:52920 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41966 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32810 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48918 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40850 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34992 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35004 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35018 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55608 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46610 - "GET / HTTP/1.0" 200 OK 2026-03-13 07:20:40,729 - lava_callback - INFO - Received callback for node 69b117b89d1917581825374a INFO: 172.20.0.1:44280 - "POST /node/69b117b89d1917581825374a HTTP/1.0" 202 Accepted 2026-03-13 07:20:40,734 - lava_callback - INFO - Starting async job submit for node 69b117b89d1917581825374a 2026-03-13 07:20:40,735 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:20:40,751 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b117b89d1917581825374a HTTP/1.1" 200 None 2026-03-13 07:20:40,752 - lava_callback - DEBUG - Processing node 69b117b89d1917581825374a: baseline-arm64-clabbe 2026-03-13 07:20:40,752 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b117b89d1917581825374a 2026-03-13 07:20:40,753 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:20:40,780 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b117b89d1917581825374a HTTP/1.1" 200 None 2026-03-13 07:20:40,782 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:20:40,805 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b5149d19175818251feb HTTP/1.1" 200 None 2026-03-13 07:20:40,808 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:20:40,830 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b117b89d1917581825374a HTTP/1.1" 200 None 2026-03-13 07:20:40,840 - lava_callback - ERROR - Error processing callback for node 69b117b89d1917581825374a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:44288 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58042 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55200 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52136 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52146 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52156 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34432 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57714 - "GET / HTTP/1.0" 200 OK 2026-03-13 07:21:40,839 - lava_callback - INFO - Received callback for node 69b118029d1917581825375e INFO: 172.20.0.1:46052 - "POST /node/69b118029d1917581825375e HTTP/1.0" 202 Accepted 2026-03-13 07:21:40,845 - lava_callback - INFO - Starting async job submit for node 69b118029d1917581825375e 2026-03-13 07:21:40,846 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:21:40,861 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b118029d1917581825375e HTTP/1.1" 200 None 2026-03-13 07:21:40,862 - lava_callback - DEBUG - Processing node 69b118029d1917581825375e: baseline-arm64-clabbe 2026-03-13 07:21:40,862 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b118029d1917581825375e 2026-03-13 07:21:40,863 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:21:40,876 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b118029d1917581825375e HTTP/1.1" 200 None 2026-03-13 07:21:40,878 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:21:40,894 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b5149d19175818251feb HTTP/1.1" 200 None 2026-03-13 07:21:40,896 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:21:40,934 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b118029d1917581825375e HTTP/1.1" 200 None 2026-03-13 07:21:40,936 - lava_callback - ERROR - Error processing callback for node 69b118029d1917581825375e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:46058 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52116 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40600 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51976 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51978 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51992 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42140 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46622 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48474 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40254 - "GET / HTTP/1.0" 200 OK 2026-03-13 07:23:00,877 - lava_callback - INFO - Received callback for node 69b1184d9d19175818253769 INFO: 172.20.0.1:34670 - "POST /node/69b1184d9d19175818253769 HTTP/1.0" 202 Accepted 2026-03-13 07:23:00,883 - lava_callback - INFO - Starting async job submit for node 69b1184d9d19175818253769 2026-03-13 07:23:00,883 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:23:00,898 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b1184d9d19175818253769 HTTP/1.1" 200 None 2026-03-13 07:23:00,899 - lava_callback - DEBUG - Processing node 69b1184d9d19175818253769: baseline-arm64-clabbe 2026-03-13 07:23:00,900 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b1184d9d19175818253769 2026-03-13 07:23:00,900 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:23:00,926 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b1184d9d19175818253769 HTTP/1.1" 200 None 2026-03-13 07:23:00,928 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:23:00,941 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b5149d19175818251feb HTTP/1.1" 200 None 2026-03-13 07:23:00,942 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:23:00,962 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b1184d9d19175818253769 HTTP/1.1" 200 None 2026-03-13 07:23:00,964 - lava_callback - ERROR - Error processing callback for node 69b1184d9d19175818253769: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:34674 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38762 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38766 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38782 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52596 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47918 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36388 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51938 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50892 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37344 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37358 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37366 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60876 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58518 - "GET / HTTP/1.0" 200 OK 2026-03-13 07:24:40,953 - lava_callback - INFO - Received callback for node 69b118ab9d1917581825377d INFO: 172.20.0.1:38698 - "POST /node/69b118ab9d1917581825377d HTTP/1.0" 202 Accepted 2026-03-13 07:24:40,959 - lava_callback - INFO - Starting async job submit for node 69b118ab9d1917581825377d 2026-03-13 07:24:40,959 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:24:40,975 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b118ab9d1917581825377d HTTP/1.1" 200 None 2026-03-13 07:24:40,976 - lava_callback - DEBUG - Processing node 69b118ab9d1917581825377d: baseline-arm-clabbe 2026-03-13 07:24:40,976 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b118ab9d1917581825377d 2026-03-13 07:24:40,977 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:24:40,991 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b118ab9d1917581825377d HTTP/1.1" 200 None 2026-03-13 07:24:40,992 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:24:41,008 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e34efa461f55e59cdb7 HTTP/1.1" 200 None 2026-03-13 07:24:41,010 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:24:41,039 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b118ab9d1917581825377d HTTP/1.1" 200 None 2026-03-13 07:24:41,042 - lava_callback - ERROR - Error processing callback for node 69b118ab9d1917581825377d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:38700 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34448 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36440 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40920 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40924 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40930 - "GET / HTTP/1.0" 200 OK 2026-03-13 07:25:20,924 - lava_callback - INFO - Received callback for node 69b118d09d1917581825378c INFO: 172.20.0.1:44786 - "POST /node/69b118d09d1917581825378c HTTP/1.0" 202 Accepted 2026-03-13 07:25:20,929 - lava_callback - INFO - Starting async job submit for node 69b118d09d1917581825378c 2026-03-13 07:25:20,930 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:25:20,945 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b118d09d1917581825378c HTTP/1.1" 200 None 2026-03-13 07:25:20,946 - lava_callback - DEBUG - Processing node 69b118d09d1917581825378c: baseline-arm-clabbe 2026-03-13 07:25:20,946 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b118d09d1917581825378c 2026-03-13 07:25:20,946 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:25:20,959 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b118d09d1917581825378c HTTP/1.1" 200 None 2026-03-13 07:25:20,961 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:25:21,002 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e34efa461f55e59cdb7 HTTP/1.1" 200 None 2026-03-13 07:25:21,004 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:25:21,039 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b118d09d1917581825378c HTTP/1.1" 200 None 2026-03-13 07:25:21,044 - lava_callback - ERROR - Error processing callback for node 69b118d09d1917581825378c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:44802 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35656 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33834 - "GET / HTTP/1.0" 200 OK 2026-03-13 07:25:47,806 - lava_callback - INFO - Received callback for node 69b118ef9d19175818253799 INFO: 172.20.0.1:33846 - "POST /node/69b118ef9d19175818253799 HTTP/1.0" 202 Accepted 2026-03-13 07:25:47,812 - lava_callback - INFO - Starting async job submit for node 69b118ef9d19175818253799 2026-03-13 07:25:47,813 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:25:47,827 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b118ef9d19175818253799 HTTP/1.1" 200 None 2026-03-13 07:25:47,828 - lava_callback - DEBUG - Processing node 69b118ef9d19175818253799: baseline-arm-baylibre 2026-03-13 07:25:47,829 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b118ef9d19175818253799 2026-03-13 07:25:47,829 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:25:47,844 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b118ef9d19175818253799 HTTP/1.1" 200 None 2026-03-13 07:25:47,846 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:25:47,860 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e34efa461f55e59cdb7 HTTP/1.1" 200 None 2026-03-13 07:25:47,862 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:25:47,890 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b118ef9d19175818253799 HTTP/1.1" 200 None 2026-03-13 07:25:47,892 - lava_callback - ERROR - Error processing callback for node 69b118ef9d19175818253799: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:36426 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40604 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42872 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42882 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42886 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39508 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58220 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60312 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58378 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44326 - "GET / HTTP/1.0" 200 OK 2026-03-13 07:27:07,847 - lava_callback - INFO - Received callback for node 69b119369d191758182537ac INFO: 172.20.0.1:44330 - "POST /node/69b119369d191758182537ac HTTP/1.0" 202 Accepted 2026-03-13 07:27:07,852 - lava_callback - INFO - Starting async job submit for node 69b119369d191758182537ac 2026-03-13 07:27:07,853 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:27:07,874 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b119369d191758182537ac HTTP/1.1" 200 None 2026-03-13 07:27:07,875 - lava_callback - DEBUG - Processing node 69b119369d191758182537ac: baseline-arm-baylibre 2026-03-13 07:27:07,875 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b119369d191758182537ac 2026-03-13 07:27:07,876 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:27:07,889 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b119369d191758182537ac HTTP/1.1" 200 None 2026-03-13 07:27:07,890 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:27:07,903 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e34efa461f55e59cdb7 HTTP/1.1" 200 None 2026-03-13 07:27:07,905 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:27:07,939 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b119369d191758182537ac HTTP/1.1" 200 None 2026-03-13 07:27:07,941 - lava_callback - ERROR - Error processing callback for node 69b119369d191758182537ac: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:40626 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40632 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40638 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53402 - "GET / HTTP/1.0" 200 OK 2026-03-13 07:27:27,832 - lava_callback - INFO - Received callback for node 69b1195b9d191758182537b8 INFO: 172.20.0.1:53410 - "POST /node/69b1195b9d191758182537b8 HTTP/1.0" 202 Accepted 2026-03-13 07:27:27,837 - lava_callback - INFO - Starting async job submit for node 69b1195b9d191758182537b8 2026-03-13 07:27:27,838 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:27:27,852 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b1195b9d191758182537b8 HTTP/1.1" 200 None 2026-03-13 07:27:27,853 - lava_callback - DEBUG - Processing node 69b1195b9d191758182537b8: baseline-arm-baylibre 2026-03-13 07:27:27,853 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b1195b9d191758182537b8 2026-03-13 07:27:27,854 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:27:27,867 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b1195b9d191758182537b8 HTTP/1.1" 200 None 2026-03-13 07:27:27,869 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:27:27,882 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e34efa461f55e59cdb7 HTTP/1.1" 200 None 2026-03-13 07:27:27,883 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:27:27,908 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b1195b9d191758182537b8 HTTP/1.1" 200 None 2026-03-13 07:27:27,910 - lava_callback - ERROR - Error processing callback for node 69b1195b9d191758182537b8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:35846 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58372 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35984 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47246 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41520 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41534 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41544 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35040 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33542 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38000 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56200 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52798 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44184 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44196 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44198 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58080 - "GET / HTTP/1.0" 200 OK 2026-03-13 07:29:27,959 - lava_callback - INFO - Received callback for node 69b119c49d191758182537df INFO: 172.20.0.1:58082 - "POST /node/69b119c49d191758182537df HTTP/1.0" 202 Accepted 2026-03-13 07:29:27,964 - lava_callback - INFO - Starting async job submit for node 69b119c49d191758182537df 2026-03-13 07:29:27,965 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:29:27,979 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b119c49d191758182537df HTTP/1.1" 200 None 2026-03-13 07:29:27,980 - lava_callback - DEBUG - Processing node 69b119c49d191758182537df: baseline-arm-baylibre 2026-03-13 07:29:27,981 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b119c49d191758182537df 2026-03-13 07:29:27,981 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:29:27,995 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b119c49d191758182537df HTTP/1.1" 200 None 2026-03-13 07:29:27,996 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:29:28,017 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e34efa461f55e59cdb7 HTTP/1.1" 200 None 2026-03-13 07:29:28,019 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:29:28,039 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b119c49d191758182537df HTTP/1.1" 200 None 2026-03-13 07:29:28,041 - lava_callback - ERROR - Error processing callback for node 69b119c49d191758182537df: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:42244 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60402 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57816 - "GET / HTTP/1.0" 200 OK 2026-03-13 07:30:01,626 - lava_callback - INFO - Received callback for node 69b119ea9d191758182537eb INFO: 172.20.0.1:45710 - "POST /node/69b119ea9d191758182537eb HTTP/1.0" 202 Accepted 2026-03-13 07:30:01,632 - lava_callback - INFO - Starting async job submit for node 69b119ea9d191758182537eb 2026-03-13 07:30:01,633 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:30:01,647 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b119ea9d191758182537eb HTTP/1.1" 200 None 2026-03-13 07:30:01,648 - lava_callback - DEBUG - Processing node 69b119ea9d191758182537eb: baseline-arm-clabbe 2026-03-13 07:30:01,649 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b119ea9d191758182537eb 2026-03-13 07:30:01,649 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:30:01,663 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b119ea9d191758182537eb HTTP/1.1" 200 None 2026-03-13 07:30:01,664 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:30:01,678 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e34efa461f55e59cdb7 HTTP/1.1" 200 None 2026-03-13 07:30:01,679 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:30:01,701 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b119ea9d191758182537eb HTTP/1.1" 200 None 2026-03-13 07:30:01,703 - lava_callback - ERROR - Error processing callback for node 69b119ea9d191758182537eb: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:45716 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45636 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45640 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45642 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52184 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48956 - "GET / HTTP/1.0" 200 OK 2026-03-13 07:30:41,119 - lava_callback - INFO - Received callback for node 69b11a0e9d191758182537ff INFO: 172.20.0.1:46476 - "POST /node/69b11a0e9d191758182537ff HTTP/1.0" 202 Accepted 2026-03-13 07:30:41,124 - lava_callback - INFO - Starting async job submit for node 69b11a0e9d191758182537ff 2026-03-13 07:30:41,125 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:30:41,144 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b11a0e9d191758182537ff HTTP/1.1" 200 None 2026-03-13 07:30:41,145 - lava_callback - DEBUG - Processing node 69b11a0e9d191758182537ff: baseline-arm-clabbe 2026-03-13 07:30:41,145 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b11a0e9d191758182537ff 2026-03-13 07:30:41,146 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:30:41,160 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b11a0e9d191758182537ff HTTP/1.1" 200 None 2026-03-13 07:30:41,162 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:30:41,181 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e34efa461f55e59cdb7 HTTP/1.1" 200 None 2026-03-13 07:30:41,183 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:30:41,204 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b11a0e9d191758182537ff HTTP/1.1" 200 None 2026-03-13 07:30:41,205 - lava_callback - ERROR - Error processing callback for node 69b11a0e9d191758182537ff: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:46484 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35602 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49072 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50472 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50488 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50494 - "GET / HTTP/1.0" 200 OK 2026-03-13 07:31:21,184 - lava_callback - INFO - Received callback for node 69b11a339d19175818253811 INFO: 172.20.0.1:42860 - "POST /node/69b11a339d19175818253811 HTTP/1.0" 202 Accepted 2026-03-13 07:31:21,190 - lava_callback - INFO - Starting async job submit for node 69b11a339d19175818253811 2026-03-13 07:31:21,191 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:31:21,211 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b11a339d19175818253811 HTTP/1.1" 200 None 2026-03-13 07:31:21,213 - lava_callback - DEBUG - Processing node 69b11a339d19175818253811: baseline-arm-clabbe 2026-03-13 07:31:21,213 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b11a339d19175818253811 2026-03-13 07:31:21,213 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:31:21,228 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b11a339d19175818253811 HTTP/1.1" 200 None 2026-03-13 07:31:21,230 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:31:21,245 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e34efa461f55e59cdb7 HTTP/1.1" 200 None 2026-03-13 07:31:21,246 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:31:21,268 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b11a339d19175818253811 HTTP/1.1" 200 None 2026-03-13 07:31:21,274 - lava_callback - ERROR - Error processing callback for node 69b11a339d19175818253811: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:42868 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56724 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52216 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47070 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47444 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59714 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59724 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59734 - "GET / HTTP/1.0" 200 OK 2026-03-13 07:32:21,238 - lava_callback - INFO - Received callback for node 69b11a7b9d1917581825382f INFO: 172.20.0.1:35182 - "POST /node/69b11a7b9d1917581825382f HTTP/1.0" 202 Accepted 2026-03-13 07:32:21,243 - lava_callback - INFO - Starting async job submit for node 69b11a7b9d1917581825382f 2026-03-13 07:32:21,244 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:32:21,258 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b11a7b9d1917581825382f HTTP/1.1" 200 None 2026-03-13 07:32:21,259 - lava_callback - DEBUG - Processing node 69b11a7b9d1917581825382f: baseline-arm-clabbe 2026-03-13 07:32:21,259 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b11a7b9d1917581825382f 2026-03-13 07:32:21,260 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:32:21,273 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b11a7b9d1917581825382f HTTP/1.1" 200 None 2026-03-13 07:32:21,275 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:32:21,358 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e34efa461f55e59cdb7 HTTP/1.1" 200 None 2026-03-13 07:32:21,360 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:32:21,380 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b11a7b9d1917581825382f HTTP/1.1" 200 None 2026-03-13 07:32:21,388 - lava_callback - ERROR - Error processing callback for node 69b11a7b9d1917581825382f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:35190 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34172 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56992 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36326 - "GET / HTTP/1.0" 200 OK 2026-03-13 07:33:01,218 - lava_callback - INFO - Received callback for node 69b11aa09d1917581825383e INFO: 172.20.0.1:49550 - "POST /node/69b11aa09d1917581825383e HTTP/1.0" 202 Accepted 2026-03-13 07:33:01,224 - lava_callback - INFO - Starting async job submit for node 69b11aa09d1917581825383e 2026-03-13 07:33:01,225 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:33:01,239 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b11aa09d1917581825383e HTTP/1.1" 200 None 2026-03-13 07:33:01,240 - lava_callback - DEBUG - Processing node 69b11aa09d1917581825383e: baseline-arm-clabbe 2026-03-13 07:33:01,240 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b11aa09d1917581825383e 2026-03-13 07:33:01,241 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:33:01,254 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b11aa09d1917581825383e HTTP/1.1" 200 None 2026-03-13 07:33:01,255 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:33:01,277 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e34efa461f55e59cdb7 HTTP/1.1" 200 None 2026-03-13 07:33:01,279 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:33:01,311 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b11aa09d1917581825383e HTTP/1.1" 200 None 2026-03-13 07:33:01,313 - lava_callback - ERROR - Error processing callback for node 69b11aa09d1917581825383e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:49552 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33178 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33192 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33194 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33834 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60038 - "GET / HTTP/1.0" 200 OK 2026-03-13 07:33:41,253 - lava_callback - INFO - Received callback for node 69b11ac49d1917581825384e INFO: 172.20.0.1:58202 - "POST /node/69b11ac49d1917581825384e HTTP/1.0" 202 Accepted 2026-03-13 07:33:41,258 - lava_callback - INFO - Starting async job submit for node 69b11ac49d1917581825384e 2026-03-13 07:33:41,259 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:33:41,274 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b11ac49d1917581825384e HTTP/1.1" 200 None 2026-03-13 07:33:41,274 - lava_callback - DEBUG - Processing node 69b11ac49d1917581825384e: baseline-arm-clabbe 2026-03-13 07:33:41,275 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b11ac49d1917581825384e 2026-03-13 07:33:41,275 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:33:41,289 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b11ac49d1917581825384e HTTP/1.1" 200 None 2026-03-13 07:33:41,290 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:33:41,304 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e34efa461f55e59cdb7 HTTP/1.1" 200 None 2026-03-13 07:33:41,306 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:33:41,326 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b11ac49d1917581825384e HTTP/1.1" 200 None 2026-03-13 07:33:41,333 - lava_callback - ERROR - Error processing callback for node 69b11ac49d1917581825384e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:58212 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46034 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60392 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48758 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48768 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48784 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58304 - "GET / HTTP/1.0" 200 OK 2026-03-13 07:34:28,205 - lava_callback - INFO - Received callback for node 69b11aef9d19175818253865 INFO: 172.20.0.1:58310 - "POST /node/69b11aef9d19175818253865 HTTP/1.0" 202 Accepted 2026-03-13 07:34:28,210 - lava_callback - INFO - Starting async job submit for node 69b11aef9d19175818253865 2026-03-13 07:34:28,211 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:34:28,225 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b11aef9d19175818253865 HTTP/1.1" 200 None 2026-03-13 07:34:28,226 - lava_callback - DEBUG - Processing node 69b11aef9d19175818253865: baseline-arm64-baylibre 2026-03-13 07:34:28,227 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b11aef9d19175818253865 2026-03-13 07:34:28,227 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:34:28,240 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b11aef9d19175818253865 HTTP/1.1" 200 None 2026-03-13 07:34:28,242 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:34:28,255 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11758ef4424e487b2eb1 HTTP/1.1" 200 None 2026-03-13 07:34:28,257 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:34:28,278 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b11aef9d19175818253865 HTTP/1.1" 200 None 2026-03-13 07:34:28,283 - lava_callback - ERROR - Error processing callback for node 69b11aef9d19175818253865: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:45924 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46702 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38494 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35882 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35616 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35632 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35638 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59700 - "GET / HTTP/1.0" 200 OK 2026-03-13 07:35:28,245 - lava_callback - INFO - Received callback for node 69b11b3a9d1917581825386d INFO: 172.20.0.1:59712 - "POST /node/69b11b3a9d1917581825386d HTTP/1.0" 202 Accepted 2026-03-13 07:35:28,251 - lava_callback - INFO - Starting async job submit for node 69b11b3a9d1917581825386d 2026-03-13 07:35:28,252 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:35:28,267 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b11b3a9d1917581825386d HTTP/1.1" 200 None 2026-03-13 07:35:28,268 - lava_callback - DEBUG - Processing node 69b11b3a9d1917581825386d: baseline-arm64-baylibre 2026-03-13 07:35:28,268 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b11b3a9d1917581825386d 2026-03-13 07:35:28,268 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:35:28,281 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b11b3a9d1917581825386d HTTP/1.1" 200 None 2026-03-13 07:35:28,283 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:35:28,296 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11758ef4424e487b2eb1 HTTP/1.1" 200 None 2026-03-13 07:35:28,298 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:35:28,324 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b11b3a9d1917581825386d HTTP/1.1" 200 None 2026-03-13 07:35:28,325 - lava_callback - ERROR - Error processing callback for node 69b11b3a9d1917581825386d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:36354 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40654 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36028 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58110 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47070 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47078 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47094 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55206 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44066 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42456 - "GET / HTTP/1.0" 200 OK 2026-03-13 07:36:48,362 - lava_callback - INFO - Received callback for node 69b11b849d1917581825387a INFO: 172.20.0.1:42466 - "POST /node/69b11b849d1917581825387a HTTP/1.0" 202 Accepted 2026-03-13 07:36:48,368 - lava_callback - INFO - Starting async job submit for node 69b11b849d1917581825387a 2026-03-13 07:36:48,369 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:36:48,385 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b11b849d1917581825387a HTTP/1.1" 200 None 2026-03-13 07:36:48,385 - lava_callback - DEBUG - Processing node 69b11b849d1917581825387a: baseline-arm64-baylibre 2026-03-13 07:36:48,386 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b11b849d1917581825387a 2026-03-13 07:36:48,386 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:36:48,405 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b11b849d1917581825387a HTTP/1.1" 200 None 2026-03-13 07:36:48,407 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:36:48,421 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11758ef4424e487b2eb1 HTTP/1.1" 200 None 2026-03-13 07:36:48,422 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:36:48,443 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b11b849d1917581825387a HTTP/1.1" 200 None 2026-03-13 07:36:48,444 - lava_callback - ERROR - Error processing callback for node 69b11b849d1917581825387a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:41940 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41664 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47556 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47570 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47586 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38860 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33936 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36998 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52134 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44834 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50828 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50838 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50852 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46708 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44996 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60440 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60074 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55384 - "GET / HTTP/1.0" 200 OK 2026-03-13 07:39:08,528 - lava_callback - INFO - Received callback for node 69b11c199d1917581825389e INFO: 172.20.0.1:55398 - "POST /node/69b11c199d1917581825389e HTTP/1.0" 202 Accepted 2026-03-13 07:39:08,534 - lava_callback - INFO - Starting async job submit for node 69b11c199d1917581825389e 2026-03-13 07:39:08,535 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:39:08,551 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b11c199d1917581825389e HTTP/1.1" 200 None 2026-03-13 07:39:08,552 - lava_callback - DEBUG - Processing node 69b11c199d1917581825389e: baseline-arm64-baylibre 2026-03-13 07:39:08,553 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b11c199d1917581825389e 2026-03-13 07:39:08,553 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:39:08,568 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b11c199d1917581825389e HTTP/1.1" 200 None 2026-03-13 07:39:08,570 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:39:08,586 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11758ef4424e487b2eb1 HTTP/1.1" 200 None 2026-03-13 07:39:08,588 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:39:08,616 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b11c199d1917581825389e HTTP/1.1" 200 None 2026-03-13 07:39:08,618 - lava_callback - ERROR - Error processing callback for node 69b11c199d1917581825389e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:52384 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52388 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52398 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41278 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38606 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39548 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37198 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50176 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58730 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58738 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58750 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55346 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60066 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37170 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48344 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54892 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39368 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39384 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39394 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38718 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56962 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38186 - "GET / HTTP/1.0" 200 OK 2026-03-13 07:41:48,626 - lava_callback - INFO - Received callback for node 69b11caf9d191758182538b2 INFO: 172.20.0.1:38192 - "POST /node/69b11caf9d191758182538b2 HTTP/1.0" 202 Accepted 2026-03-13 07:41:48,631 - lava_callback - INFO - Starting async job submit for node 69b11caf9d191758182538b2 2026-03-13 07:41:48,632 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:41:48,646 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b11caf9d191758182538b2 HTTP/1.1" 200 None 2026-03-13 07:41:48,647 - lava_callback - DEBUG - Processing node 69b11caf9d191758182538b2: baseline-arm64-baylibre 2026-03-13 07:41:48,648 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b11caf9d191758182538b2 2026-03-13 07:41:48,648 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:41:48,663 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b11caf9d191758182538b2 HTTP/1.1" 200 None 2026-03-13 07:41:48,664 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:41:48,684 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11758ef4424e487b2eb1 HTTP/1.1" 200 None 2026-03-13 07:41:48,686 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:41:48,707 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b11caf9d191758182538b2 HTTP/1.1" 200 None 2026-03-13 07:41:48,718 - lava_callback - ERROR - Error processing callback for node 69b11caf9d191758182538b2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:44310 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54538 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56318 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56328 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56344 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50658 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37842 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50718 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48154 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33834 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35302 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35312 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35318 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54964 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56926 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40866 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60680 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35734 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40076 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40080 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40090 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60224 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34336 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60870 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40794 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43706 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39488 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39494 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39504 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40606 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39190 - "GET / HTTP/1.0" 200 OK 2026-03-13 07:45:41,916 - lava_callback - INFO - Received callback for node 69b11d929d191758182538e2 INFO: 172.20.0.1:32846 - "POST /node/69b11d929d191758182538e2 HTTP/1.0" 202 Accepted 2026-03-13 07:45:41,922 - lava_callback - INFO - Starting async job submit for node 69b11d929d191758182538e2 2026-03-13 07:45:41,923 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:45:41,942 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b11d929d191758182538e2 HTTP/1.1" 200 None 2026-03-13 07:45:41,943 - lava_callback - DEBUG - Processing node 69b11d929d191758182538e2: baseline-arm64-clabbe 2026-03-13 07:45:41,944 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b11d929d191758182538e2 2026-03-13 07:45:41,944 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:45:41,959 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b11d929d191758182538e2 HTTP/1.1" 200 None 2026-03-13 07:45:41,961 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:45:41,974 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11758ef4424e487b2eb1 HTTP/1.1" 200 None 2026-03-13 07:45:41,976 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:45:41,998 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b11d929d191758182538e2 HTTP/1.1" 200 None 2026-03-13 07:45:42,000 - lava_callback - ERROR - Error processing callback for node 69b11d929d191758182538e2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:32852 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58182 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52210 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58370 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58384 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58388 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50384 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57376 - "GET / HTTP/1.0" 200 OK 2026-03-13 07:46:41,884 - lava_callback - INFO - Received callback for node 69b11ddd9d191758182538f7 INFO: 172.20.0.1:54650 - "POST /node/69b11ddd9d191758182538f7 HTTP/1.0" 202 Accepted 2026-03-13 07:46:41,889 - lava_callback - INFO - Starting async job submit for node 69b11ddd9d191758182538f7 2026-03-13 07:46:41,890 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:46:41,921 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b11ddd9d191758182538f7 HTTP/1.1" 200 None 2026-03-13 07:46:41,922 - lava_callback - DEBUG - Processing node 69b11ddd9d191758182538f7: baseline-arm64-clabbe 2026-03-13 07:46:41,922 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b11ddd9d191758182538f7 2026-03-13 07:46:41,923 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:46:41,936 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b11ddd9d191758182538f7 HTTP/1.1" 200 None 2026-03-13 07:46:41,937 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:46:41,950 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11758ef4424e487b2eb1 HTTP/1.1" 200 None 2026-03-13 07:46:41,951 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:46:41,974 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b11ddd9d191758182538f7 HTTP/1.1" 200 None 2026-03-13 07:46:41,981 - lava_callback - ERROR - Error processing callback for node 69b11ddd9d191758182538f7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:54664 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36442 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36764 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46532 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46536 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46550 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44370 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36560 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40476 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49494 - "GET / HTTP/1.0" 200 OK 2026-03-13 07:48:01,879 - lava_callback - INFO - Received callback for node 69b11e289d19175818253900 INFO: 172.20.0.1:37944 - "POST /node/69b11e289d19175818253900 HTTP/1.0" 202 Accepted 2026-03-13 07:48:01,885 - lava_callback - INFO - Starting async job submit for node 69b11e289d19175818253900 2026-03-13 07:48:01,886 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:48:01,901 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b11e289d19175818253900 HTTP/1.1" 200 None 2026-03-13 07:48:01,902 - lava_callback - DEBUG - Processing node 69b11e289d19175818253900: baseline-arm64-clabbe 2026-03-13 07:48:01,902 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b11e289d19175818253900 2026-03-13 07:48:01,903 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:48:01,919 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b11e289d19175818253900 HTTP/1.1" 200 None 2026-03-13 07:48:01,921 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:48:01,935 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11758ef4424e487b2eb1 HTTP/1.1" 200 None 2026-03-13 07:48:01,937 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:48:01,960 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b11e289d19175818253900 HTTP/1.1" 200 None 2026-03-13 07:48:01,962 - lava_callback - ERROR - Error processing callback for node 69b11e289d19175818253900: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:37958 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37952 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37960 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37972 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33558 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42068 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33688 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60880 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54002 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34554 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34558 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34570 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48416 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35238 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34226 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51414 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47708 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55462 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55470 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55474 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57100 - "GET / HTTP/1.0" 200 OK 2026-03-13 07:50:29,120 - lava_callback - INFO - Received callback for node 69b11eb89d19175818253915 INFO: 172.20.0.1:57114 - "POST /node/69b11eb89d19175818253915 HTTP/1.0" 202 Accepted 2026-03-13 07:50:29,126 - lava_callback - INFO - Starting async job submit for node 69b11eb89d19175818253915 2026-03-13 07:50:29,127 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:50:29,143 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b11eb89d19175818253915 HTTP/1.1" 200 None 2026-03-13 07:50:29,144 - lava_callback - DEBUG - Processing node 69b11eb89d19175818253915: baseline-arm64-baylibre 2026-03-13 07:50:29,144 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b11eb89d19175818253915 2026-03-13 07:50:29,145 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:50:29,159 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b11eb89d19175818253915 HTTP/1.1" 200 None 2026-03-13 07:50:29,160 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:50:29,174 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11758ef4424e487b2eb1 HTTP/1.1" 200 None 2026-03-13 07:50:29,175 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:50:29,202 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b11eb89d19175818253915 HTTP/1.1" 200 None 2026-03-13 07:50:29,205 - lava_callback - ERROR - Error processing callback for node 69b11eb89d19175818253915: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:51262 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56384 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51132 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53850 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45242 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45258 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45272 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55884 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35056 - "GET / HTTP/1.0" 200 OK 2026-03-13 07:51:42,071 - lava_callback - INFO - Received callback for node 69b11f089d19175818253926 INFO: 172.20.0.1:39132 - "POST /node/69b11f089d19175818253926 HTTP/1.0" 202 Accepted 2026-03-13 07:51:42,077 - lava_callback - INFO - Starting async job submit for node 69b11f089d19175818253926 2026-03-13 07:51:42,077 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:51:42,092 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b11f089d19175818253926 HTTP/1.1" 200 None 2026-03-13 07:51:42,092 - lava_callback - DEBUG - Processing node 69b11f089d19175818253926: baseline-arm64-clabbe 2026-03-13 07:51:42,093 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b11f089d19175818253926 2026-03-13 07:51:42,093 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:51:42,106 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b11f089d19175818253926 HTTP/1.1" 200 None 2026-03-13 07:51:42,108 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:51:42,148 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11758ef4424e487b2eb1 HTTP/1.1" 200 None 2026-03-13 07:51:42,149 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:51:42,175 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b11f089d19175818253926 HTTP/1.1" 200 None 2026-03-13 07:51:42,177 - lava_callback - ERROR - Error processing callback for node 69b11f089d19175818253926: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:39146 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51948 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43766 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35692 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35706 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35714 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45858 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58086 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54176 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50760 - "GET / HTTP/1.0" 200 OK 2026-03-13 07:53:02,160 - lava_callback - INFO - Received callback for node 69b11f539d19175818253931 INFO: 172.20.0.1:52568 - "POST /node/69b11f539d19175818253931 HTTP/1.0" 202 Accepted 2026-03-13 07:53:02,165 - lava_callback - INFO - Starting async job submit for node 69b11f539d19175818253931 2026-03-13 07:53:02,166 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:53:02,180 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b11f539d19175818253931 HTTP/1.1" 200 None 2026-03-13 07:53:02,181 - lava_callback - DEBUG - Processing node 69b11f539d19175818253931: baseline-arm64-clabbe 2026-03-13 07:53:02,181 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b11f539d19175818253931 2026-03-13 07:53:02,182 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:53:02,196 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b11f539d19175818253931 HTTP/1.1" 200 None 2026-03-13 07:53:02,197 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:53:02,221 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11758ef4424e487b2eb1 HTTP/1.1" 200 None 2026-03-13 07:53:02,222 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:53:02,243 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b11f539d19175818253931 HTTP/1.1" 200 None 2026-03-13 07:53:02,245 - lava_callback - ERROR - Error processing callback for node 69b11f539d19175818253931: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:52580 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53966 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53980 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53990 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50376 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40816 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41092 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47392 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36802 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42370 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42386 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42396 - "GET / HTTP/1.0" 200 OK 2026-03-13 07:54:22,243 - lava_callback - INFO - Received callback for node 69b11f9e9d19175818253942 INFO: 172.20.0.1:38974 - "POST /node/69b11f9e9d19175818253942 HTTP/1.0" 202 Accepted 2026-03-13 07:54:22,249 - lava_callback - INFO - Starting async job submit for node 69b11f9e9d19175818253942 2026-03-13 07:54:22,250 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:54:22,265 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b11f9e9d19175818253942 HTTP/1.1" 200 None 2026-03-13 07:54:22,265 - lava_callback - DEBUG - Processing node 69b11f9e9d19175818253942: baseline-arm64-clabbe 2026-03-13 07:54:22,266 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b11f9e9d19175818253942 2026-03-13 07:54:22,266 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:54:22,283 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b11f9e9d19175818253942 HTTP/1.1" 200 None 2026-03-13 07:54:22,285 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:54:22,298 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11758ef4424e487b2eb1 HTTP/1.1" 200 None 2026-03-13 07:54:22,300 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:54:22,323 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b11f9e9d19175818253942 HTTP/1.1" 200 None 2026-03-13 07:54:22,329 - lava_callback - ERROR - Error processing callback for node 69b11f9e9d19175818253942: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:38990 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47966 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34820 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42838 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44342 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48588 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48596 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48598 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57102 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45138 - "GET / HTTP/1.0" 200 OK 2026-03-13 07:55:42,265 - lava_callback - INFO - Received callback for node 69b11fea9d1917581825394b INFO: 172.20.0.1:40114 - "POST /node/69b11fea9d1917581825394b HTTP/1.0" 202 Accepted 2026-03-13 07:55:42,271 - lava_callback - INFO - Starting async job submit for node 69b11fea9d1917581825394b 2026-03-13 07:55:42,272 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:55:42,287 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b11fea9d1917581825394b HTTP/1.1" 200 None 2026-03-13 07:55:42,287 - lava_callback - DEBUG - Processing node 69b11fea9d1917581825394b: baseline-arm64-clabbe 2026-03-13 07:55:42,288 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b11fea9d1917581825394b 2026-03-13 07:55:42,288 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:55:42,302 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b11fea9d1917581825394b HTTP/1.1" 200 None 2026-03-13 07:55:42,303 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:55:42,316 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11758ef4424e487b2eb1 HTTP/1.1" 200 None 2026-03-13 07:55:42,317 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:55:42,339 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b11fea9d1917581825394b HTTP/1.1" 200 None 2026-03-13 07:55:42,340 - lava_callback - ERROR - Error processing callback for node 69b11fea9d1917581825394b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:40130 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38684 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50766 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50808 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50810 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50818 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36744 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35706 - "GET / HTTP/1.0" 200 OK 2026-03-13 07:56:42,264 - lava_callback - INFO - Received callback for node 69b120369d19175818253959 INFO: 172.20.0.1:39020 - "POST /node/69b120369d19175818253959 HTTP/1.0" 202 Accepted 2026-03-13 07:56:42,270 - lava_callback - INFO - Starting async job submit for node 69b120369d19175818253959 2026-03-13 07:56:42,271 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:56:42,285 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b120369d19175818253959 HTTP/1.1" 200 None 2026-03-13 07:56:42,285 - lava_callback - DEBUG - Processing node 69b120369d19175818253959: baseline-arm64-clabbe 2026-03-13 07:56:42,286 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b120369d19175818253959 2026-03-13 07:56:42,286 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:56:42,299 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b120369d19175818253959 HTTP/1.1" 200 None 2026-03-13 07:56:42,300 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:56:42,313 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11758ef4424e487b2eb1 HTTP/1.1" 200 None 2026-03-13 07:56:42,314 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:56:42,334 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b120369d19175818253959 HTTP/1.1" 200 None 2026-03-13 07:56:42,340 - lava_callback - ERROR - Error processing callback for node 69b120369d19175818253959: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:39022 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56558 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47550 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36548 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36558 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36568 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45834 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40346 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52186 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49904 - "GET / HTTP/1.0" 200 OK 2026-03-13 07:58:02,377 - lava_callback - INFO - Received callback for node 69b120839d19175818253963 INFO: 172.20.0.1:52186 - "POST /node/69b120839d19175818253963 HTTP/1.0" 202 Accepted 2026-03-13 07:58:02,382 - lava_callback - INFO - Starting async job submit for node 69b120839d19175818253963 2026-03-13 07:58:02,383 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:58:02,398 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b120839d19175818253963 HTTP/1.1" 200 None 2026-03-13 07:58:02,399 - lava_callback - DEBUG - Processing node 69b120839d19175818253963: baseline-arm64-clabbe 2026-03-13 07:58:02,399 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b120839d19175818253963 2026-03-13 07:58:02,400 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:58:02,416 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b120839d19175818253963 HTTP/1.1" 200 None 2026-03-13 07:58:02,417 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:58:02,432 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11758ef4424e487b2eb1 HTTP/1.1" 200 None 2026-03-13 07:58:02,434 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:58:02,460 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b120839d19175818253963 HTTP/1.1" 200 None 2026-03-13 07:58:02,461 - lava_callback - ERROR - Error processing callback for node 69b120839d19175818253963: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:52200 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46854 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46868 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46872 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40508 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44854 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48364 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51748 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36108 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50520 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50528 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50534 - "GET / HTTP/1.0" 200 OK 2026-03-13 07:59:22,454 - lava_callback - INFO - Received callback for node 69b120cd9d19175818253973 INFO: 172.20.0.1:34340 - "POST /node/69b120cd9d19175818253973 HTTP/1.0" 202 Accepted 2026-03-13 07:59:22,463 - lava_callback - INFO - Starting async job submit for node 69b120cd9d19175818253973 2026-03-13 07:59:22,464 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:59:22,478 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b120cd9d19175818253973 HTTP/1.1" 200 None 2026-03-13 07:59:22,479 - lava_callback - DEBUG - Processing node 69b120cd9d19175818253973: baseline-arm64-clabbe 2026-03-13 07:59:22,480 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b120cd9d19175818253973 2026-03-13 07:59:22,480 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:59:22,494 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b120cd9d19175818253973 HTTP/1.1" 200 None 2026-03-13 07:59:22,495 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:59:22,513 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11758ef4424e487b2eb1 HTTP/1.1" 200 None 2026-03-13 07:59:22,514 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:59:22,540 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b120cd9d19175818253973 HTTP/1.1" 200 None 2026-03-13 07:59:22,546 - lava_callback - ERROR - Error processing callback for node 69b120cd9d19175818253973: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:34354 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55734 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44356 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40436 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57410 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55556 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55558 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55570 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58594 - "GET / HTTP/1.0" 200 OK 2026-03-13 08:00:29,690 - lava_callback - INFO - Received callback for node 69b121109d1917581825397e INFO: 172.20.0.1:58604 - "POST /node/69b121109d1917581825397e HTTP/1.0" 202 Accepted 2026-03-13 08:00:29,695 - lava_callback - INFO - Starting async job submit for node 69b121109d1917581825397e 2026-03-13 08:00:29,696 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 08:00:29,714 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b121109d1917581825397e HTTP/1.1" 200 None 2026-03-13 08:00:29,715 - lava_callback - DEBUG - Processing node 69b121109d1917581825397e: baseline-arm64-baylibre 2026-03-13 08:00:29,716 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b121109d1917581825397e 2026-03-13 08:00:29,716 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 08:00:29,729 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b121109d1917581825397e HTTP/1.1" 200 None 2026-03-13 08:00:29,731 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 08:00:29,743 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e44efa461f55e59cdc8 HTTP/1.1" 200 None 2026-03-13 08:00:29,745 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 08:00:29,768 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b121109d1917581825397e HTTP/1.1" 200 None 2026-03-13 08:00:29,769 - lava_callback - ERROR - Error processing callback for node 69b121109d1917581825397e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:43834 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42392 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47024 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43196 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47550 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47556 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47572 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59226 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36472 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56440 - "GET / HTTP/1.0" 200 OK 2026-03-13 08:01:49,777 - lava_callback - INFO - Received callback for node 69b1215d9d19175818253986 INFO: 172.20.0.1:57094 - "POST /node/69b1215d9d19175818253986 HTTP/1.0" 202 Accepted 2026-03-13 08:01:49,784 - lava_callback - INFO - Starting async job submit for node 69b1215d9d19175818253986 2026-03-13 08:01:49,785 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 08:01:49,799 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b1215d9d19175818253986 HTTP/1.1" 200 None 2026-03-13 08:01:49,799 - lava_callback - DEBUG - Processing node 69b1215d9d19175818253986: baseline-arm64-baylibre 2026-03-13 08:01:49,800 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b1215d9d19175818253986 2026-03-13 08:01:49,800 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 08:01:49,816 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b1215d9d19175818253986 HTTP/1.1" 200 None 2026-03-13 08:01:49,817 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 08:01:49,838 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e44efa461f55e59cdc8 HTTP/1.1" 200 None 2026-03-13 08:01:49,840 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 08:01:49,861 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b1215d9d19175818253986 HTTP/1.1" 200 None 2026-03-13 08:01:49,868 - lava_callback - ERROR - Error processing callback for node 69b1215d9d19175818253986: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:57098 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43266 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38040 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38056 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38062 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35696 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44296 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39050 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45210 - "GET / HTTP/1.0" 200 OK 2026-03-13 08:03:02,622 - lava_callback - INFO - Received callback for node 69b121b39d19175818253995 INFO: 172.20.0.1:40294 - "POST /node/69b121b39d19175818253995 HTTP/1.0" 202 Accepted 2026-03-13 08:03:02,628 - lava_callback - INFO - Starting async job submit for node 69b121b39d19175818253995 2026-03-13 08:03:02,629 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 08:03:02,642 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b121b39d19175818253995 HTTP/1.1" 200 None 2026-03-13 08:03:02,643 - lava_callback - DEBUG - Processing node 69b121b39d19175818253995: baseline-arm64-clabbe 2026-03-13 08:03:02,643 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b121b39d19175818253995 2026-03-13 08:03:02,644 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 08:03:02,656 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b121b39d19175818253995 HTTP/1.1" 200 None 2026-03-13 08:03:02,658 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 08:03:02,671 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11758ef4424e487b2eb1 HTTP/1.1" 200 None 2026-03-13 08:03:02,672 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 08:03:02,692 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b121b39d19175818253995 HTTP/1.1" 200 None 2026-03-13 08:03:02,694 - lava_callback - ERROR - Error processing callback for node 69b121b39d19175818253995: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:40304 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58068 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58080 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58094 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40384 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38130 - "GET /node/6994465f5ae459b9df3d9f7b%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:38136 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58552 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36870 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38134 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41640 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41644 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41650 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57890 - "GET / HTTP/1.0" 200 OK 2026-03-13 08:04:29,913 - lava_callback - INFO - Received callback for node 69b121fa9d191758182539a1 INFO: 172.20.0.1:50866 - "POST /node/69b121fa9d191758182539a1 HTTP/1.0" 202 Accepted 2026-03-13 08:04:29,919 - lava_callback - INFO - Starting async job submit for node 69b121fa9d191758182539a1 2026-03-13 08:04:29,919 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 08:04:29,935 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b121fa9d191758182539a1 HTTP/1.1" 200 None 2026-03-13 08:04:29,936 - lava_callback - DEBUG - Processing node 69b121fa9d191758182539a1: baseline-arm64-baylibre 2026-03-13 08:04:29,936 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b121fa9d191758182539a1 2026-03-13 08:04:29,937 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 08:04:29,950 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b121fa9d191758182539a1 HTTP/1.1" 200 None 2026-03-13 08:04:29,951 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 08:04:29,978 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e44efa461f55e59cdc8 HTTP/1.1" 200 None 2026-03-13 08:04:29,980 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 08:04:30,048 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b121fa9d191758182539a1 HTTP/1.1" 200 None 2026-03-13 08:04:30,055 - lava_callback - ERROR - Error processing callback for node 69b121fa9d191758182539a1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:50872 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36776 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50798 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39422 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48522 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48528 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48540 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38062 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46660 - "GET / HTTP/1.0" 200 OK 2026-03-13 08:05:42,700 - lava_callback - INFO - Received callback for node 69b122509d191758182539a8 INFO: 172.20.0.1:36152 - "POST /node/69b122509d191758182539a8 HTTP/1.0" 202 Accepted 2026-03-13 08:05:42,705 - lava_callback - INFO - Starting async job submit for node 69b122509d191758182539a8 2026-03-13 08:05:42,706 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 08:05:42,720 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b122509d191758182539a8 HTTP/1.1" 200 None 2026-03-13 08:05:42,721 - lava_callback - DEBUG - Processing node 69b122509d191758182539a8: baseline-arm64-clabbe 2026-03-13 08:05:42,721 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b122509d191758182539a8 2026-03-13 08:05:42,721 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 08:05:42,754 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b122509d191758182539a8 HTTP/1.1" 200 None 2026-03-13 08:05:42,756 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 08:05:42,769 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11758ef4424e487b2eb1 HTTP/1.1" 200 None 2026-03-13 08:05:42,770 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 08:05:42,790 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b122509d191758182539a8 HTTP/1.1" 200 None 2026-03-13 08:05:42,795 - lava_callback - ERROR - Error processing callback for node 69b122509d191758182539a8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:36168 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55140 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42304 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52066 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52068 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52082 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37934 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39098 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55718 - "GET / HTTP/1.0" 200 OK 2026-03-13 08:06:50,083 - lava_callback - INFO - Received callback for node 69b122959d191758182539b3 INFO: 172.20.0.1:39982 - "POST /node/69b122959d191758182539b3 HTTP/1.0" 202 Accepted 2026-03-13 08:06:50,088 - lava_callback - INFO - Starting async job submit for node 69b122959d191758182539b3 2026-03-13 08:06:50,089 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 08:06:50,108 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b122959d191758182539b3 HTTP/1.1" 200 None 2026-03-13 08:06:50,109 - lava_callback - DEBUG - Processing node 69b122959d191758182539b3: baseline-arm64-baylibre 2026-03-13 08:06:50,109 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b122959d191758182539b3 2026-03-13 08:06:50,110 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 08:06:50,124 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b122959d191758182539b3 HTTP/1.1" 200 None 2026-03-13 08:06:50,126 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 08:06:50,139 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e44efa461f55e59cdc8 HTTP/1.1" 200 None 2026-03-13 08:06:50,141 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 08:06:50,198 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b122959d191758182539b3 HTTP/1.1" 200 None 2026-03-13 08:06:50,204 - lava_callback - ERROR - Error processing callback for node 69b122959d191758182539b3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:39988 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34118 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58178 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58184 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58188 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39556 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40616 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57718 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50294 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55130 - "GET / HTTP/1.0" 200 OK 2026-03-13 08:08:10,171 - lava_callback - INFO - Received callback for node 69b122e19d191758182539b9 INFO: 172.20.0.1:38598 - "POST /node/69b122e19d191758182539b9 HTTP/1.0" 202 Accepted 2026-03-13 08:08:10,177 - lava_callback - INFO - Starting async job submit for node 69b122e19d191758182539b9 2026-03-13 08:08:10,177 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 08:08:10,191 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b122e19d191758182539b9 HTTP/1.1" 200 None 2026-03-13 08:08:10,192 - lava_callback - DEBUG - Processing node 69b122e19d191758182539b9: baseline-arm64-baylibre 2026-03-13 08:08:10,192 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b122e19d191758182539b9 2026-03-13 08:08:10,193 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 08:08:10,206 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b122e19d191758182539b9 HTTP/1.1" 200 None 2026-03-13 08:08:10,207 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 08:08:10,221 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e44efa461f55e59cdc8 HTTP/1.1" 200 None 2026-03-13 08:08:10,223 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 08:08:10,243 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b122e19d191758182539b9 HTTP/1.1" 200 None 2026-03-13 08:08:10,244 - lava_callback - ERROR - Error processing callback for node 69b122e19d191758182539b9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:38610 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38616 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38632 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46890 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55520 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39390 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55234 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32948 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52616 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52620 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52636 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51670 - "GET / HTTP/1.0" 200 OK 2026-03-13 08:09:30,219 - lava_callback - INFO - Received callback for node 69b1232d9d191758182539c5 INFO: 172.20.0.1:38004 - "POST /node/69b1232d9d191758182539c5 HTTP/1.0" 202 Accepted 2026-03-13 08:09:30,225 - lava_callback - INFO - Starting async job submit for node 69b1232d9d191758182539c5 2026-03-13 08:09:30,225 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 08:09:30,242 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b1232d9d191758182539c5 HTTP/1.1" 200 None 2026-03-13 08:09:30,243 - lava_callback - DEBUG - Processing node 69b1232d9d191758182539c5: baseline-arm64-baylibre 2026-03-13 08:09:30,244 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b1232d9d191758182539c5 2026-03-13 08:09:30,244 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 08:09:30,257 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b1232d9d191758182539c5 HTTP/1.1" 200 None 2026-03-13 08:09:30,258 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 08:09:30,271 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e44efa461f55e59cdc8 HTTP/1.1" 200 None 2026-03-13 08:09:30,273 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 08:09:30,294 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b1232d9d191758182539c5 HTTP/1.1" 200 None 2026-03-13 08:09:30,295 - lava_callback - ERROR - Error processing callback for node 69b1232d9d191758182539c5: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:38014 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57542 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36016 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48398 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39392 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39402 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39408 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56602 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43440 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56662 - "GET / HTTP/1.0" 200 OK 2026-03-13 08:10:50,234 - lava_callback - INFO - Received callback for node 69b123799d191758182539cd INFO: 172.20.0.1:43312 - "POST /node/69b123799d191758182539cd HTTP/1.0" 202 Accepted 2026-03-13 08:10:50,239 - lava_callback - INFO - Starting async job submit for node 69b123799d191758182539cd 2026-03-13 08:10:50,240 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 08:10:50,255 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b123799d191758182539cd HTTP/1.1" 200 None 2026-03-13 08:10:50,256 - lava_callback - DEBUG - Processing node 69b123799d191758182539cd: baseline-arm64-baylibre 2026-03-13 08:10:50,256 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b123799d191758182539cd 2026-03-13 08:10:50,257 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 08:10:50,299 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b123799d191758182539cd HTTP/1.1" 200 None 2026-03-13 08:10:50,301 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 08:10:50,315 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11758ef4424e487b2eb1 HTTP/1.1" 200 None 2026-03-13 08:10:50,317 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 08:10:50,339 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b123799d191758182539cd HTTP/1.1" 200 None 2026-03-13 08:10:50,349 - lava_callback - ERROR - Error processing callback for node 69b123799d191758182539cd: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:43316 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46220 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50318 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50322 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50330 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42352 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45020 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42382 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50070 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34510 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48964 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48970 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48984 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56510 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50126 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38390 - "GET / HTTP/1.0" 200 OK INFO: Shutting down INFO: Waiting for application shutdown. INFO: Application shutdown complete. INFO: Finished server process [1] INFO: Started server process [1] INFO: Waiting for application startup. INFO: Application startup complete. INFO: Uvicorn running on http://0.0.0.0:8000 (Press CTRL+C to quit) INFO: 172.20.0.1:52828 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52832 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38608 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51004 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34800 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57026 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40526 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40542 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54528 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54530 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32860 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43384 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37906 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60908 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35822 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35826 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45474 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45478 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37494 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36598 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54298 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36430 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54272 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54284 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34304 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34308 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35624 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41152 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36090 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49960 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55816 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55822 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50538 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50542 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57386 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32958 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58058 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55762 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58552 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58554 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58236 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58238 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39408 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34260 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57084 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43262 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50880 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50886 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57902 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57904 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41938 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57494 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54098 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50400 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53912 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53920 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46090 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46104 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33162 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46680 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39872 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49530 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58752 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58756 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46742 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46750 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47908 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45380 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51428 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43110 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47866 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47876 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59534 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59540 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58436 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39004 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47420 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50742 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51392 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51398 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55452 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55460 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47892 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52400 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33396 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43962 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37866 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37880 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38150 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38152 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45390 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37000 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47306 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38750 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37884 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37886 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60456 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60464 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42584 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46098 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41920 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38802 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43564 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43576 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55182 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55196 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50502 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59498 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42082 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40058 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54058 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54068 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35376 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35386 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38692 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33754 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60556 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60776 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32776 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32780 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43350 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43364 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43950 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34582 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52298 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60964 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40726 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40740 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50854 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50870 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49488 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60658 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38324 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37482 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50292 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50298 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37112 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37114 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57168 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40430 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50870 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47166 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43750 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43756 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40290 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40296 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32776 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40972 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49786 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37196 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45400 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45412 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35736 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35752 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57664 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56516 - "GET / HTTP/1.0" 200 OK 2026-03-13 00:39:43,363 - lava_callback - INFO - Received callback for node 69b35c55974c91e64560e4e6 2026-03-13 00:39:43,369 - lava_callback - INFO - Starting async job submit for node 69b35c55974c91e64560e4e6 INFO: 172.20.0.1:32778 - "POST /node/69b35c55974c91e64560e4e6 HTTP/1.0" 202 Accepted 2026-03-13 00:39:43,393 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:39:43,413 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35c55974c91e64560e4e6 HTTP/1.1" 200 None 2026-03-13 00:39:43,414 - lava_callback - DEBUG - Processing node 69b35c55974c91e64560e4e6: baseline-arm 2026-03-13 00:39:43,615 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 00:39:43,882 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 00:39:43,884 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69b35c55974c91e64560e4e6/log.txt.gz 2026-03-13 00:39:43,999 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 00:39:44,223 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 00:39:44,225 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69b35c55974c91e64560e4e6/lava-logs.yaml 2026-03-13 00:39:44,230 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 00:39:44,386 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 00:39:44,388 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69b35c55974c91e64560e4e6/lava_callback.json.gz 2026-03-13 00:39:44,411 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:39:44,426 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35c55974c91e64560e4e6 HTTP/1.1" 200 None 2026-03-13 00:39:44,427 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:39:44,441 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3593e974c91e64560e4a3 HTTP/1.1" 200 None 2026-03-13 00:39:44,442 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:39:44,467 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b35c55974c91e64560e4e6 HTTP/1.1" 200 None 2026-03-13 00:39:44,468 - lava_callback - INFO - Completed processing callback for node 69b35c55974c91e64560e4e6 INFO: 172.20.0.1:32782 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50650 - "GET / HTTP/1.0" 200 OK 2026-03-13 00:40:05,734 - lava_callback - INFO - Received callback for node 69b0b9dd9d191758182523ff INFO: 172.20.0.1:47002 - "POST /node/69b0b9dd9d191758182523ff HTTP/1.0" 202 Accepted 2026-03-13 00:40:05,740 - lava_callback - INFO - Starting async job submit for node 69b0b9dd9d191758182523ff 2026-03-13 00:40:05,741 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:40:05,761 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b9dd9d191758182523ff HTTP/1.1" 200 None 2026-03-13 00:40:05,762 - lava_callback - DEBUG - Processing node 69b0b9dd9d191758182523ff: baseline-arm-baylibre 2026-03-13 00:40:05,762 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0b9dd9d191758182523ff 2026-03-13 00:40:05,763 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:40:05,796 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b9dd9d191758182523ff HTTP/1.1" 200 None 2026-03-13 00:40:05,797 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:40:05,812 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b4fc9d19175818251fd6 HTTP/1.1" 200 None 2026-03-13 00:40:05,813 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:40:05,865 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0b9dd9d191758182523ff HTTP/1.1" 200 None 2026-03-13 00:40:05,867 - lava_callback - ERROR - Error processing callback for node 69b0b9dd9d191758182523ff: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:47010 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47016 - "GET / HTTP/1.0" 200 OK 2026-03-13 00:40:14,469 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:40:14,487 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:41632 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41636 - "GET / HTTP/1.0" 200 OK 2026-03-13 00:40:23,744 - lava_callback - INFO - Received callback for node 69b0b9e99d19175818252422 INFO: 172.20.0.1:37512 - "POST /node/69b0b9e99d19175818252422 HTTP/1.0" 202 Accepted 2026-03-13 00:40:23,750 - lava_callback - INFO - Starting async job submit for node 69b0b9e99d19175818252422 2026-03-13 00:40:23,750 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:40:23,765 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b9e99d19175818252422 HTTP/1.1" 200 None 2026-03-13 00:40:23,766 - lava_callback - DEBUG - Processing node 69b0b9e99d19175818252422: baseline-arm-clabbe 2026-03-13 00:40:23,766 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0b9e99d19175818252422 2026-03-13 00:40:23,766 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:40:23,785 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b9e99d19175818252422 HTTP/1.1" 200 None 2026-03-13 00:40:23,786 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:40:23,800 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b4fc9d19175818251fd6 HTTP/1.1" 200 None 2026-03-13 00:40:23,801 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:40:23,823 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0b9e99d19175818252422 HTTP/1.1" 200 None 2026-03-13 00:40:23,825 - lava_callback - ERROR - Error processing callback for node 69b0b9e99d19175818252422: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-13 00:40:24,398 - lava_callback - INFO - Received callback for node 69b0b9ed9d1917581825242f INFO: 172.20.0.1:37524 - "POST /node/69b0b9ed9d1917581825242f HTTP/1.0" 202 Accepted 2026-03-13 00:40:24,403 - lava_callback - INFO - Starting async job submit for node 69b0b9ed9d1917581825242f 2026-03-13 00:40:24,404 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:40:24,419 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b9ed9d1917581825242f HTTP/1.1" 200 None 2026-03-13 00:40:24,420 - lava_callback - DEBUG - Processing node 69b0b9ed9d1917581825242f: baseline-arm-clabbe 2026-03-13 00:40:24,420 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0b9ed9d1917581825242f 2026-03-13 00:40:24,421 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:40:24,434 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b9ed9d1917581825242f HTTP/1.1" 200 None 2026-03-13 00:40:24,435 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:40:24,453 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b4fc9d19175818251fd6 HTTP/1.1" 200 None 2026-03-13 00:40:24,454 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:40:24,474 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0b9ed9d1917581825242f HTTP/1.1" 200 None 2026-03-13 00:40:24,475 - lava_callback - ERROR - Error processing callback for node 69b0b9ed9d1917581825242f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-13 00:40:25,035 - lava_callback - INFO - Received callback for node 69b0b9ee9d19175818252432 INFO: 172.20.0.1:37538 - "POST /node/69b0b9ee9d19175818252432 HTTP/1.0" 202 Accepted 2026-03-13 00:40:25,041 - lava_callback - INFO - Starting async job submit for node 69b0b9ee9d19175818252432 2026-03-13 00:40:25,042 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:40:25,057 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b9ee9d19175818252432 HTTP/1.1" 200 None 2026-03-13 00:40:25,057 - lava_callback - DEBUG - Processing node 69b0b9ee9d19175818252432: baseline-arm-clabbe 2026-03-13 00:40:25,058 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0b9ee9d19175818252432 2026-03-13 00:40:25,058 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:40:25,072 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b9ee9d19175818252432 HTTP/1.1" 200 None 2026-03-13 00:40:25,073 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:40:25,086 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b4fc9d19175818251fd6 HTTP/1.1" 200 None 2026-03-13 00:40:25,088 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:40:25,112 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0b9ee9d19175818252432 HTTP/1.1" 200 None 2026-03-13 00:40:25,113 - lava_callback - ERROR - Error processing callback for node 69b0b9ee9d19175818252432: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-13 00:40:25,692 - lava_callback - INFO - Received callback for node 69b0b9ef9d19175818252435 INFO: 172.20.0.1:37540 - "POST /node/69b0b9ef9d19175818252435 HTTP/1.0" 202 Accepted 2026-03-13 00:40:25,697 - lava_callback - INFO - Starting async job submit for node 69b0b9ef9d19175818252435 2026-03-13 00:40:25,698 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:40:25,713 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b9ef9d19175818252435 HTTP/1.1" 200 None 2026-03-13 00:40:25,713 - lava_callback - DEBUG - Processing node 69b0b9ef9d19175818252435: baseline-arm-clabbe 2026-03-13 00:40:25,714 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0b9ef9d19175818252435 2026-03-13 00:40:25,714 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:40:25,737 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b9ef9d19175818252435 HTTP/1.1" 200 None 2026-03-13 00:40:25,739 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:40:25,752 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b4fc9d19175818251fd6 HTTP/1.1" 200 None 2026-03-13 00:40:25,754 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:40:25,775 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0b9ef9d19175818252435 HTTP/1.1" 200 None 2026-03-13 00:40:25,777 - lava_callback - ERROR - Error processing callback for node 69b0b9ef9d19175818252435: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-13 00:40:26,338 - lava_callback - INFO - Received callback for node 69b0b9f19d1917581825243e INFO: 172.20.0.1:37546 - "POST /node/69b0b9f19d1917581825243e HTTP/1.0" 202 Accepted 2026-03-13 00:40:26,344 - lava_callback - INFO - Starting async job submit for node 69b0b9f19d1917581825243e 2026-03-13 00:40:26,345 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:40:26,359 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b9f19d1917581825243e HTTP/1.1" 200 None 2026-03-13 00:40:26,360 - lava_callback - DEBUG - Processing node 69b0b9f19d1917581825243e: baseline-arm-clabbe 2026-03-13 00:40:26,360 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0b9f19d1917581825243e 2026-03-13 00:40:26,361 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:40:26,381 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b9f19d1917581825243e HTTP/1.1" 200 None 2026-03-13 00:40:26,382 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:40:26,396 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b4fc9d19175818251fd6 HTTP/1.1" 200 None 2026-03-13 00:40:26,397 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:40:26,417 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0b9f19d1917581825243e HTTP/1.1" 200 None 2026-03-13 00:40:26,422 - lava_callback - ERROR - Error processing callback for node 69b0b9f19d1917581825243e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:37562 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47556 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37348 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59580 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38776 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38792 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46024 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46030 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54360 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45194 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37596 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41648 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50892 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50896 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49714 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49724 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47848 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51538 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52480 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48364 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37106 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37110 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59710 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59722 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54676 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36176 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47524 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35460 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40424 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40432 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58674 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58688 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39920 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44304 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49556 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55264 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56482 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56488 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54008 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54024 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55450 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35468 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46594 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59464 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52010 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52020 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44072 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44088 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56660 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36526 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43270 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38852 - "GET / HTTP/1.0" 200 OK 2026-03-13 00:47:05,988 - lava_callback - INFO - Received callback for node 69b0bb809d19175818252619 INFO: 172.20.0.1:60296 - "POST /node/69b0bb809d19175818252619 HTTP/1.0" 202 Accepted 2026-03-13 00:47:05,993 - lava_callback - INFO - Starting async job submit for node 69b0bb809d19175818252619 2026-03-13 00:47:05,994 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:47:06,017 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0bb809d19175818252619 HTTP/1.1" 200 None 2026-03-13 00:47:06,018 - lava_callback - DEBUG - Processing node 69b0bb809d19175818252619: baseline-arm-baylibre 2026-03-13 00:47:06,018 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0bb809d19175818252619 2026-03-13 00:47:06,019 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:47:06,034 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0bb809d19175818252619 HTTP/1.1" 200 None 2026-03-13 00:47:06,036 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:47:06,057 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11668ef4424e487b2ea2 HTTP/1.1" 200 None 2026-03-13 00:47:06,058 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:47:06,081 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0bb809d19175818252619 HTTP/1.1" 200 None 2026-03-13 00:47:06,086 - lava_callback - ERROR - Error processing callback for node 69b0bb809d19175818252619: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:60312 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60320 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36208 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36216 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43106 - "GET / HTTP/1.0" 200 OK 2026-03-13 00:47:28,213 - lava_callback - INFO - Received callback for node 69b35e8e974c91e64560e902 INFO: 172.20.0.1:43120 - "POST /node/69b35e8e974c91e64560e902 HTTP/1.0" 202 Accepted 2026-03-13 00:47:28,241 - lava_callback - INFO - Starting async job submit for node 69b35e8e974c91e64560e902 2026-03-13 00:47:28,251 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:47:28,320 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35e8e974c91e64560e902 HTTP/1.1" 200 None 2026-03-13 00:47:28,321 - lava_callback - DEBUG - Processing node 69b35e8e974c91e64560e902: baseline-arm 2026-03-13 00:47:28,336 - lava_callback - WARNING - Failed to upload log 2026-03-13 00:47:28,341 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 00:47:28,719 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 00:47:28,721 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69b35e8e974c91e64560e902/lava-logs.yaml 2026-03-13 00:47:28,722 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 00:47:29,047 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 00:47:29,050 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69b35e8e974c91e64560e902/lava_callback.json.gz 2026-03-13 00:47:29,055 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:47:29,081 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35e8e974c91e64560e902 HTTP/1.1" 200 None 2026-03-13 00:47:29,083 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:47:29,112 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35934974c91e64560e49d HTTP/1.1" 200 None 2026-03-13 00:47:29,115 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:47:29,154 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b35e8e974c91e64560e902 HTTP/1.1" 200 None 2026-03-13 00:47:29,159 - lava_callback - INFO - Completed processing callback for node 69b35e8e974c91e64560e902 INFO: 172.20.0.1:37462 - "GET / HTTP/1.0" 200 OK 2026-03-13 00:47:44,491 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:47:44,517 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-13 00:47:46,006 - lava_callback - INFO - Received callback for node 69b0bba69d19175818252627 INFO: 172.20.0.1:55920 - "POST /node/69b0bba69d19175818252627 HTTP/1.0" 202 Accepted 2026-03-13 00:47:46,017 - lava_callback - INFO - Starting async job submit for node 69b0bba69d19175818252627 2026-03-13 00:47:46,018 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:47:46,051 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0bba69d19175818252627 HTTP/1.1" 200 None 2026-03-13 00:47:46,052 - lava_callback - DEBUG - Processing node 69b0bba69d19175818252627: baseline-arm-baylibre 2026-03-13 00:47:46,054 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0bba69d19175818252627 2026-03-13 00:47:46,055 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:47:46,083 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0bba69d19175818252627 HTTP/1.1" 200 None 2026-03-13 00:47:46,085 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:47:46,122 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11668ef4424e487b2ea2 HTTP/1.1" 200 None 2026-03-13 00:47:46,125 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:47:46,170 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0bba69d19175818252627 HTTP/1.1" 200 None 2026-03-13 00:47:46,171 - lava_callback - ERROR - Error processing callback for node 69b0bba69d19175818252627: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:55922 - "GET / HTTP/1.0" 200 OK 2026-03-13 00:47:56,152 - lava_callback - INFO - Received callback for node 69b35e8f974c91e64560e906 INFO: 172.20.0.1:48214 - "POST /node/69b35e8f974c91e64560e906 HTTP/1.0" 202 Accepted 2026-03-13 00:47:56,210 - lava_callback - INFO - Starting async job submit for node 69b35e8f974c91e64560e906 2026-03-13 00:47:56,246 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:47:56,273 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35e8f974c91e64560e906 HTTP/1.1" 200 None 2026-03-13 00:47:56,274 - lava_callback - DEBUG - Processing node 69b35e8f974c91e64560e906: baseline-arm INFO: 172.20.0.1:48230 - "GET / HTTP/1.0" 200 OK 2026-03-13 00:47:56,522 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 00:47:56,864 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 00:47:56,866 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69b35e8f974c91e64560e906/log.txt.gz 2026-03-13 00:47:57,021 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 00:47:57,587 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 00:47:57,589 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69b35e8f974c91e64560e906/lava-logs.yaml 2026-03-13 00:47:57,594 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 00:47:58,186 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 00:47:58,188 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69b35e8f974c91e64560e906/lava_callback.json.gz 2026-03-13 00:47:58,225 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:47:58,254 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35e8f974c91e64560e906 HTTP/1.1" 200 None 2026-03-13 00:47:58,256 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:47:58,289 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35934974c91e64560e49d HTTP/1.1" 200 None 2026-03-13 00:47:58,292 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:47:58,326 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b35e8f974c91e64560e906 HTTP/1.1" 200 None 2026-03-13 00:47:58,328 - lava_callback - INFO - Completed processing callback for node 69b35e8f974c91e64560e906 2026-03-13 00:48:03,922 - lava_callback - INFO - Received callback for node 69b35e92974c91e64560e910 INFO: 172.20.0.1:54806 - "POST /node/69b35e92974c91e64560e910 HTTP/1.0" 202 Accepted 2026-03-13 00:48:03,990 - lava_callback - INFO - Starting async job submit for node 69b35e92974c91e64560e910 2026-03-13 00:48:04,026 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:48:04,058 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35e92974c91e64560e910 HTTP/1.1" 200 None 2026-03-13 00:48:04,059 - lava_callback - DEBUG - Processing node 69b35e92974c91e64560e910: baseline-arm 2026-03-13 00:48:04,278 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 00:48:04,548 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 00:48:04,550 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69b35e92974c91e64560e910/log.txt.gz 2026-03-13 00:48:04,673 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 00:48:05,135 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 00:48:05,138 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69b35e92974c91e64560e910/lava-logs.yaml 2026-03-13 00:48:05,143 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 00:48:05,381 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 00:48:05,384 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69b35e92974c91e64560e910/lava_callback.json.gz 2026-03-13 00:48:05,426 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:48:05,450 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35e92974c91e64560e910 HTTP/1.1" 200 None 2026-03-13 00:48:05,453 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:48:05,486 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35934974c91e64560e49d HTTP/1.1" 200 None 2026-03-13 00:48:05,489 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:48:05,531 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b35e92974c91e64560e910 HTTP/1.1" 200 None 2026-03-13 00:48:05,533 - lava_callback - INFO - Completed processing callback for node 69b35e92974c91e64560e910 INFO: 172.20.0.1:54818 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54820 - "GET / HTTP/1.0" 200 OK 2026-03-13 00:48:14,521 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:48:14,585 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:53824 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53830 - "GET / HTTP/1.0" 200 OK 2026-03-13 00:48:26,253 - lava_callback - INFO - Received callback for node 69b0bbcc9d19175818252632 INFO: 172.20.0.1:36866 - "POST /node/69b0bbcc9d19175818252632 HTTP/1.0" 202 Accepted 2026-03-13 00:48:26,262 - lava_callback - INFO - Starting async job submit for node 69b0bbcc9d19175818252632 2026-03-13 00:48:26,263 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:48:26,316 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0bbcc9d19175818252632 HTTP/1.1" 200 None 2026-03-13 00:48:26,317 - lava_callback - DEBUG - Processing node 69b0bbcc9d19175818252632: baseline-arm-baylibre 2026-03-13 00:48:26,317 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0bbcc9d19175818252632 2026-03-13 00:48:26,318 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:48:26,344 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0bbcc9d19175818252632 HTTP/1.1" 200 None 2026-03-13 00:48:26,347 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:48:26,361 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11668ef4424e487b2ea2 HTTP/1.1" 200 None 2026-03-13 00:48:26,363 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:48:26,391 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0bbcc9d19175818252632 HTTP/1.1" 200 None 2026-03-13 00:48:26,392 - lava_callback - ERROR - Error processing callback for node 69b0bbcc9d19175818252632: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:36870 - "GET / HTTP/1.0" 200 OK 2026-03-13 00:48:34,943 - lava_callback - INFO - Received callback for node 69b35e89974c91e64560e8fb INFO: 172.20.0.1:48964 - "POST /node/69b35e89974c91e64560e8fb HTTP/1.0" 202 Accepted 2026-03-13 00:48:34,950 - lava_callback - INFO - Starting async job submit for node 69b35e89974c91e64560e8fb 2026-03-13 00:48:34,976 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:48:34,990 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35e89974c91e64560e8fb HTTP/1.1" 200 None 2026-03-13 00:48:34,991 - lava_callback - DEBUG - Processing node 69b35e89974c91e64560e8fb: baseline-arm 2026-03-13 00:48:35,173 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 00:48:35,314 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 00:48:35,316 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69b35e89974c91e64560e8fb/log.txt.gz 2026-03-13 00:48:35,488 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 00:48:35,675 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 00:48:35,677 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69b35e89974c91e64560e8fb/lava-logs.yaml 2026-03-13 00:48:35,684 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 00:48:35,869 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 00:48:35,871 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69b35e89974c91e64560e8fb/lava_callback.json.gz 2026-03-13 00:48:35,895 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:48:35,909 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35e89974c91e64560e8fb HTTP/1.1" 200 None 2026-03-13 00:48:35,911 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:48:35,926 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35934974c91e64560e49d HTTP/1.1" 200 None 2026-03-13 00:48:35,927 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:48:35,954 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b35e89974c91e64560e8fb HTTP/1.1" 200 None 2026-03-13 00:48:35,956 - lava_callback - INFO - Completed processing callback for node 69b35e89974c91e64560e8fb INFO: 172.20.0.1:48972 - "GET / HTTP/1.0" 200 OK 2026-03-13 00:48:44,588 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:48:44,604 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:38390 - "GET / HTTP/1.0" 200 OK 2026-03-13 00:48:50,786 - lava_callback - INFO - Received callback for node 69b35eb2974c91e64560e990 INFO: 172.20.0.1:52154 - "POST /node/69b35eb2974c91e64560e990 HTTP/1.0" 202 Accepted 2026-03-13 00:48:50,792 - lava_callback - INFO - Starting async job submit for node 69b35eb2974c91e64560e990 2026-03-13 00:48:50,810 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:48:50,827 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35eb2974c91e64560e990 HTTP/1.1" 200 None 2026-03-13 00:48:50,828 - lava_callback - DEBUG - Processing node 69b35eb2974c91e64560e990: baseline-arm-pengutronix 2026-03-13 00:48:50,968 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 00:48:51,206 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 00:48:51,208 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69b35eb2974c91e64560e990/log.txt.gz 2026-03-13 00:48:51,291 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 00:48:51,546 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 00:48:51,548 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69b35eb2974c91e64560e990/lava-logs.yaml 2026-03-13 00:48:51,552 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 00:48:51,744 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 00:48:51,746 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69b35eb2974c91e64560e990/lava_callback.json.gz 2026-03-13 00:48:51,765 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:48:51,780 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35eb2974c91e64560e990 HTTP/1.1" 200 None 2026-03-13 00:48:51,781 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:48:51,795 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35934974c91e64560e49d HTTP/1.1" 200 None 2026-03-13 00:48:51,797 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:48:51,822 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b35eb2974c91e64560e990 HTTP/1.1" 200 None 2026-03-13 00:48:51,829 - lava_callback - INFO - Completed processing callback for node 69b35eb2974c91e64560e990 INFO: 172.20.0.1:52162 - "GET / HTTP/1.0" 200 OK 2026-03-13 00:49:03,075 - lava_callback - INFO - Received callback for node 69b35eb7974c91e64560e99c INFO: 172.20.0.1:49618 - "POST /node/69b35eb7974c91e64560e99c HTTP/1.0" 202 Accepted 2026-03-13 00:49:03,121 - lava_callback - INFO - Starting async job submit for node 69b35eb7974c91e64560e99c 2026-03-13 00:49:03,141 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:49:03,156 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35eb7974c91e64560e99c HTTP/1.1" 200 None 2026-03-13 00:49:03,157 - lava_callback - DEBUG - Processing node 69b35eb7974c91e64560e99c: baseline-arm-pengutronix 2026-03-13 00:49:03,265 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 00:49:03,376 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 00:49:03,378 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69b35eb7974c91e64560e99c/log.txt.gz 2026-03-13 00:49:03,441 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 00:49:03,618 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 00:49:03,620 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69b35eb7974c91e64560e99c/lava-logs.yaml 2026-03-13 00:49:03,624 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 00:49:03,741 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 00:49:03,742 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69b35eb7974c91e64560e99c/lava_callback.json.gz 2026-03-13 00:49:03,763 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:49:03,785 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35eb7974c91e64560e99c HTTP/1.1" 200 None 2026-03-13 00:49:03,787 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:49:03,802 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35934974c91e64560e49d HTTP/1.1" 200 None 2026-03-13 00:49:03,803 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:49:03,828 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b35eb7974c91e64560e99c HTTP/1.1" 200 None 2026-03-13 00:49:03,829 - lava_callback - INFO - Completed processing callback for node 69b35eb7974c91e64560e99c 2026-03-13 00:49:06,096 - lava_callback - INFO - Received callback for node 69b0bbf19d19175818252641 INFO: 172.20.0.1:49632 - "POST /node/69b0bbf19d19175818252641 HTTP/1.0" 202 Accepted 2026-03-13 00:49:06,101 - lava_callback - INFO - Starting async job submit for node 69b0bbf19d19175818252641 2026-03-13 00:49:06,102 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:49:06,118 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0bbf19d19175818252641 HTTP/1.1" 200 None 2026-03-13 00:49:06,119 - lava_callback - DEBUG - Processing node 69b0bbf19d19175818252641: baseline-arm-baylibre 2026-03-13 00:49:06,119 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0bbf19d19175818252641 2026-03-13 00:49:06,120 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:49:06,152 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0bbf19d19175818252641 HTTP/1.1" 200 None 2026-03-13 00:49:06,154 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:49:06,168 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11668ef4424e487b2ea2 HTTP/1.1" 200 None 2026-03-13 00:49:06,169 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:49:06,191 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0bbf19d19175818252641 HTTP/1.1" 200 None 2026-03-13 00:49:06,197 - lava_callback - ERROR - Error processing callback for node 69b0bbf19d19175818252641: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:49634 - "GET / HTTP/1.0" 200 OK 2026-03-13 00:49:07,345 - lava_callback - INFO - Received callback for node 69b35e9c974c91e64560e932 INFO: 172.20.0.1:49644 - "POST /node/69b35e9c974c91e64560e932 HTTP/1.0" 202 Accepted 2026-03-13 00:49:07,351 - lava_callback - INFO - Starting async job submit for node 69b35e9c974c91e64560e932 2026-03-13 00:49:07,373 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:49:07,390 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35e9c974c91e64560e932 HTTP/1.1" 200 None 2026-03-13 00:49:07,391 - lava_callback - DEBUG - Processing node 69b35e9c974c91e64560e932: baseline-arm-broonie 2026-03-13 00:49:07,591 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 00:49:07,732 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 00:49:07,734 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69b35e9c974c91e64560e932/log.txt.gz 2026-03-13 00:49:07,853 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 00:49:08,028 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 00:49:08,030 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69b35e9c974c91e64560e932/lava-logs.yaml 2026-03-13 00:49:08,036 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 00:49:08,040 - lava_callback - INFO - Received callback for node 69b35efd974c91e64560ea21 2026-03-13 00:49:08,046 - lava_callback - INFO - Starting async job submit for node 69b35efd974c91e64560ea21 INFO: 172.20.0.1:49658 - "POST /node/69b35efd974c91e64560ea21 HTTP/1.0" 202 Accepted 2026-03-13 00:49:08,049 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:49:08,071 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35efd974c91e64560ea21 HTTP/1.1" 200 None 2026-03-13 00:49:08,072 - lava_callback - DEBUG - Processing node 69b35efd974c91e64560ea21: baseline-arm 2026-03-13 00:49:08,077 - lava_callback - WARNING - Failed to upload log 2026-03-13 00:49:08,081 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 00:49:08,189 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 00:49:08,191 - lava_callback - INFO - Received callback for node 69b35eb4974c91e64560e993 2026-03-13 00:49:08,197 - lava_callback - INFO - Starting async job submit for node 69b35eb4974c91e64560e993 INFO: 172.20.0.1:49670 - "POST /node/69b35eb4974c91e64560e993 HTTP/1.0" 202 Accepted 2026-03-13 00:49:08,213 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69b35e9c974c91e64560e932/lava_callback.json.gz 2026-03-13 00:49:08,235 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 00:49:08,242 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:49:08,246 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:49:08,247 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69b35efd974c91e64560ea21/lava-logs.yaml 2026-03-13 00:49:08,248 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 00:49:08,267 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35e9c974c91e64560e932 HTTP/1.1" 200 None 2026-03-13 00:49:08,269 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:49:08,270 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35eb4974c91e64560e993 HTTP/1.1" 200 None 2026-03-13 00:49:08,271 - lava_callback - DEBUG - Processing node 69b35eb4974c91e64560e993: baseline-arm-pengutronix INFO: 172.20.0.1:49676 - "GET / HTTP/1.0" 200 OK 2026-03-13 00:49:08,338 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35934974c91e64560e49d HTTP/1.1" 200 None 2026-03-13 00:49:08,361 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:49:08,414 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 00:49:08,467 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69b35efd974c91e64560ea21/lava_callback.json.gz 2026-03-13 00:49:08,475 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:49:08,485 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 00:49:08,499 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b35e9c974c91e64560e932 HTTP/1.1" 200 None 2026-03-13 00:49:08,509 - lava_callback - INFO - Completed processing callback for node 69b35e9c974c91e64560e932 2026-03-13 00:49:08,511 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35efd974c91e64560ea21 HTTP/1.1" 200 None 2026-03-13 00:49:08,535 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:49:08,563 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35934974c91e64560e49d HTTP/1.1" 200 None 2026-03-13 00:49:08,567 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:49:08,621 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b35efd974c91e64560ea21 HTTP/1.1" 200 None 2026-03-13 00:49:08,627 - lava_callback - INFO - Completed processing callback for node 69b35efd974c91e64560ea21 2026-03-13 00:49:08,728 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 00:49:08,730 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69b35eb4974c91e64560e993/log.txt.gz 2026-03-13 00:49:08,857 - lava_callback - INFO - Received callback for node 69b35e9e974c91e64560e935 INFO: 172.20.0.1:49690 - "POST /node/69b35e9e974c91e64560e935 HTTP/1.0" 202 Accepted 2026-03-13 00:49:08,869 - lava_callback - INFO - Starting async job submit for node 69b35e9e974c91e64560e935 2026-03-13 00:49:08,915 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:49:08,928 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 00:49:08,949 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35e9e974c91e64560e935 HTTP/1.1" 200 None 2026-03-13 00:49:08,951 - lava_callback - DEBUG - Processing node 69b35e9e974c91e64560e935: baseline-arm-broonie 2026-03-13 00:49:09,132 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 00:49:09,180 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69b35eb4974c91e64560e993/lava-logs.yaml 2026-03-13 00:49:09,210 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 00:49:09,211 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 00:49:09,349 - lava_callback - INFO - Received callback for node 69b35ea0974c91e64560e938 INFO: 172.20.0.1:49704 - "POST /node/69b35ea0974c91e64560e938 HTTP/1.0" 202 Accepted 2026-03-13 00:49:09,357 - lava_callback - INFO - Starting async job submit for node 69b35ea0974c91e64560e938 2026-03-13 00:49:09,375 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 00:49:09,386 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:49:09,387 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69b35e9e974c91e64560e935/log.txt.gz 2026-03-13 00:49:09,407 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 00:49:09,467 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69b35eb4974c91e64560e993/lava_callback.json.gz 2026-03-13 00:49:09,509 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:49:09,525 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35ea0974c91e64560e938 HTTP/1.1" 200 None 2026-03-13 00:49:09,531 - lava_callback - DEBUG - Processing node 69b35ea0974c91e64560e938: baseline-arm-broonie 2026-03-13 00:49:09,580 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 00:49:09,602 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35eb4974c91e64560e993 HTTP/1.1" 200 None 2026-03-13 00:49:09,630 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:49:09,723 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35934974c91e64560e49d HTTP/1.1" 200 None 2026-03-13 00:49:09,735 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:49:09,837 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b35eb4974c91e64560e993 HTTP/1.1" 200 None 2026-03-13 00:49:09,839 - lava_callback - INFO - Completed processing callback for node 69b35eb4974c91e64560e993 2026-03-13 00:49:09,845 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 00:49:09,983 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 00:49:09,986 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69b35e9e974c91e64560e935/lava-logs.yaml 2026-03-13 00:49:09,993 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 00:49:10,011 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 00:49:10,014 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69b35ea0974c91e64560e938/log.txt.gz 2026-03-13 00:49:10,178 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 00:49:10,199 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 00:49:10,199 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69b35e9e974c91e64560e935/lava_callback.json.gz 2026-03-13 00:49:10,233 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:49:10,254 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35e9e974c91e64560e935 HTTP/1.1" 200 None 2026-03-13 00:49:10,256 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:49:10,272 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35934974c91e64560e49d HTTP/1.1" 200 None 2026-03-13 00:49:10,275 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:49:10,304 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b35e9e974c91e64560e935 HTTP/1.1" 200 None 2026-03-13 00:49:10,315 - lava_callback - INFO - Completed processing callback for node 69b35e9e974c91e64560e935 2026-03-13 00:49:10,482 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 00:49:10,485 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69b35ea0974c91e64560e938/lava-logs.yaml 2026-03-13 00:49:10,493 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 00:49:10,669 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 00:49:10,671 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69b35ea0974c91e64560e938/lava_callback.json.gz 2026-03-13 00:49:10,704 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:49:10,730 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35ea0974c91e64560e938 HTTP/1.1" 200 None 2026-03-13 00:49:10,731 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:49:10,749 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35934974c91e64560e49d HTTP/1.1" 200 None 2026-03-13 00:49:10,752 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:49:10,776 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b35ea0974c91e64560e938 HTTP/1.1" 200 None 2026-03-13 00:49:10,778 - lava_callback - INFO - Completed processing callback for node 69b35ea0974c91e64560e938 2026-03-13 00:49:13,836 - lava_callback - INFO - Received callback for node 69b35ea5974c91e64560e949 INFO: 172.20.0.1:57872 - "POST /node/69b35ea5974c91e64560e949 HTTP/1.0" 202 Accepted 2026-03-13 00:49:13,842 - lava_callback - INFO - Starting async job submit for node 69b35ea5974c91e64560e949 2026-03-13 00:49:13,866 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:49:13,880 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35ea5974c91e64560e949 HTTP/1.1" 200 None 2026-03-13 00:49:13,881 - lava_callback - DEBUG - Processing node 69b35ea5974c91e64560e949: baseline-arm-clabbe 2026-03-13 00:49:14,114 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 00:49:14,240 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 00:49:14,242 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69b35ea5974c91e64560e949/log.txt.gz 2026-03-13 00:49:14,372 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 00:49:14,554 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 00:49:14,556 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69b35ea5974c91e64560e949/lava-logs.yaml 2026-03-13 00:49:14,562 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 00:49:14,606 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:49:14,629 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 2026-03-13 00:49:14,716 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 00:49:14,718 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69b35ea5974c91e64560e949/lava_callback.json.gz 2026-03-13 00:49:14,740 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:49:14,758 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35ea5974c91e64560e949 HTTP/1.1" 200 None 2026-03-13 00:49:14,760 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:49:14,774 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35934974c91e64560e49d HTTP/1.1" 200 None 2026-03-13 00:49:14,776 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:49:14,801 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b35ea5974c91e64560e949 HTTP/1.1" 200 None 2026-03-13 00:49:14,802 - lava_callback - INFO - Completed processing callback for node 69b35ea5974c91e64560e949 INFO: 172.20.0.1:57874 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57890 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33834 - "GET / HTTP/1.0" 200 OK 2026-03-13 00:49:30,299 - lava_callback - INFO - Received callback for node 69b35e9a974c91e64560e92f INFO: 172.20.0.1:60710 - "POST /node/69b35e9a974c91e64560e92f HTTP/1.0" 202 Accepted 2026-03-13 00:49:30,305 - lava_callback - INFO - Starting async job submit for node 69b35e9a974c91e64560e92f 2026-03-13 00:49:30,328 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:49:30,342 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35e9a974c91e64560e92f HTTP/1.1" 200 None 2026-03-13 00:49:30,343 - lava_callback - DEBUG - Processing node 69b35e9a974c91e64560e92f: baseline-arm-broonie 2026-03-13 00:49:30,546 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 00:49:30,700 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 00:49:30,702 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69b35e9a974c91e64560e92f/log.txt.gz 2026-03-13 00:49:30,825 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 00:49:31,016 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 00:49:31,018 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69b35e9a974c91e64560e92f/lava-logs.yaml 2026-03-13 00:49:31,024 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 00:49:31,179 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 00:49:31,180 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69b35e9a974c91e64560e92f/lava_callback.json.gz 2026-03-13 00:49:31,203 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:49:31,240 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35e9a974c91e64560e92f HTTP/1.1" 200 None 2026-03-13 00:49:31,242 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:49:31,259 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35934974c91e64560e49d HTTP/1.1" 200 None 2026-03-13 00:49:31,261 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:49:31,285 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b35e9a974c91e64560e92f HTTP/1.1" 200 None 2026-03-13 00:49:31,291 - lava_callback - INFO - Completed processing callback for node 69b35e9a974c91e64560e92f 2026-03-13 00:49:33,222 - lava_callback - INFO - Received callback for node 69b35eaf974c91e64560e963 INFO: 172.20.0.1:60714 - "POST /node/69b35eaf974c91e64560e963 HTTP/1.0" 202 Accepted 2026-03-13 00:49:33,228 - lava_callback - INFO - Starting async job submit for node 69b35eaf974c91e64560e963 2026-03-13 00:49:33,250 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:49:33,265 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35eaf974c91e64560e963 HTTP/1.1" 200 None 2026-03-13 00:49:33,267 - lava_callback - DEBUG - Processing node 69b35eaf974c91e64560e963: baseline-arm-kontron 2026-03-13 00:49:33,446 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 00:49:33,581 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 00:49:33,582 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-kontron-69b35eaf974c91e64560e963/log.txt.gz 2026-03-13 00:49:33,684 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 00:49:33,868 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 00:49:33,870 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-kontron-69b35eaf974c91e64560e963/lava-logs.yaml 2026-03-13 00:49:33,874 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 00:49:34,001 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 00:49:34,003 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-kontron-69b35eaf974c91e64560e963/lava_callback.json.gz 2026-03-13 00:49:34,026 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:49:34,040 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35eaf974c91e64560e963 HTTP/1.1" 200 None 2026-03-13 00:49:34,042 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:49:34,056 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35934974c91e64560e49d HTTP/1.1" 200 None 2026-03-13 00:49:34,058 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:49:34,083 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b35eaf974c91e64560e963 HTTP/1.1" 200 None 2026-03-13 00:49:34,088 - lava_callback - INFO - Completed processing callback for node 69b35eaf974c91e64560e963 INFO: 172.20.0.1:60728 - "GET / HTTP/1.0" 200 OK 2026-03-13 00:49:44,631 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:49:44,658 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-13 00:49:46,156 - lava_callback - INFO - Received callback for node 69b0bc179d19175818252669 INFO: 172.20.0.1:45342 - "POST /node/69b0bc179d19175818252669 HTTP/1.0" 202 Accepted 2026-03-13 00:49:46,166 - lava_callback - INFO - Starting async job submit for node 69b0bc179d19175818252669 2026-03-13 00:49:46,166 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:49:46,181 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0bc179d19175818252669 HTTP/1.1" 200 None 2026-03-13 00:49:46,182 - lava_callback - DEBUG - Processing node 69b0bc179d19175818252669: baseline-arm-baylibre 2026-03-13 00:49:46,182 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0bc179d19175818252669 2026-03-13 00:49:46,183 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:49:46,196 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0bc179d19175818252669 HTTP/1.1" 200 None 2026-03-13 00:49:46,198 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:49:46,216 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11668ef4424e487b2ea2 HTTP/1.1" 200 None 2026-03-13 00:49:46,218 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:49:46,241 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0bc179d19175818252669 HTTP/1.1" 200 None 2026-03-13 00:49:46,250 - lava_callback - ERROR - Error processing callback for node 69b0bc179d19175818252669: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:45348 - "GET / HTTP/1.0" 200 OK 2026-03-13 00:49:46,611 - lava_callback - INFO - Received callback for node 69b35e91974c91e64560e90d INFO: 172.20.0.1:45362 - "POST /node/69b35e91974c91e64560e90d HTTP/1.0" 202 Accepted 2026-03-13 00:49:46,624 - lava_callback - INFO - Starting async job submit for node 69b35e91974c91e64560e90d 2026-03-13 00:49:46,680 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:49:46,697 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35e91974c91e64560e90d HTTP/1.1" 200 None 2026-03-13 00:49:46,698 - lava_callback - DEBUG - Processing node 69b35e91974c91e64560e90d: baseline-arm 2026-03-13 00:49:46,912 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 00:49:47,055 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 00:49:47,057 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69b35e91974c91e64560e90d/log.txt.gz 2026-03-13 00:49:47,187 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 00:49:47,359 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 00:49:47,361 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69b35e91974c91e64560e90d/lava-logs.yaml 2026-03-13 00:49:47,366 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 00:49:47,509 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 00:49:47,511 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69b35e91974c91e64560e90d/lava_callback.json.gz 2026-03-13 00:49:47,533 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:49:47,548 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35e91974c91e64560e90d HTTP/1.1" 200 None 2026-03-13 00:49:47,549 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:49:47,564 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35934974c91e64560e49d HTTP/1.1" 200 None 2026-03-13 00:49:47,565 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:49:47,597 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b35e91974c91e64560e90d HTTP/1.1" 200 None 2026-03-13 00:49:47,603 - lava_callback - INFO - Completed processing callback for node 69b35e91974c91e64560e90d INFO: 172.20.0.1:60868 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34474 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34488 - "GET / HTTP/1.0" 200 OK 2026-03-13 00:50:14,660 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:50:14,680 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:48872 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48880 - "GET / HTTP/1.0" 200 OK 2026-03-13 00:50:26,169 - lava_callback - INFO - Received callback for node 69b0bc3d9d19175818252670 INFO: 172.20.0.1:39728 - "POST /node/69b0bc3d9d19175818252670 HTTP/1.0" 202 Accepted 2026-03-13 00:50:26,175 - lava_callback - INFO - Starting async job submit for node 69b0bc3d9d19175818252670 2026-03-13 00:50:26,175 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:50:26,190 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0bc3d9d19175818252670 HTTP/1.1" 200 None 2026-03-13 00:50:26,191 - lava_callback - DEBUG - Processing node 69b0bc3d9d19175818252670: baseline-arm-baylibre 2026-03-13 00:50:26,192 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0bc3d9d19175818252670 2026-03-13 00:50:26,192 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:50:26,205 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0bc3d9d19175818252670 HTTP/1.1" 200 None 2026-03-13 00:50:26,207 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:50:26,226 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11668ef4424e487b2ea2 HTTP/1.1" 200 None 2026-03-13 00:50:26,228 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:50:26,259 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0bc3d9d19175818252670 HTTP/1.1" 200 None 2026-03-13 00:50:26,261 - lava_callback - ERROR - Error processing callback for node 69b0bc3d9d19175818252670: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:39742 - "GET / HTTP/1.0" 200 OK 2026-03-13 00:50:28,011 - lava_callback - INFO - Received callback for node 69b35ed0974c91e64560e9d4 INFO: 172.20.0.1:39748 - "POST /node/69b35ed0974c91e64560e9d4 HTTP/1.0" 202 Accepted 2026-03-13 00:50:28,017 - lava_callback - INFO - Starting async job submit for node 69b35ed0974c91e64560e9d4 2026-03-13 00:50:28,040 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:50:28,056 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35ed0974c91e64560e9d4 HTTP/1.1" 200 None 2026-03-13 00:50:28,057 - lava_callback - DEBUG - Processing node 69b35ed0974c91e64560e9d4: ltp-dio 2026-03-13 00:50:28,423 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 00:50:28,579 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 00:50:28,581 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-dio-69b35ed0974c91e64560e9d4/log.txt.gz 2026-03-13 00:50:28,803 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 00:50:28,993 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 00:50:28,996 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-dio-69b35ed0974c91e64560e9d4/lava-logs.yaml 2026-03-13 00:50:29,005 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 00:50:29,184 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 00:50:29,186 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-dio-69b35ed0974c91e64560e9d4/lava_callback.json.gz 2026-03-13 00:50:29,230 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:50:29,244 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35ed0974c91e64560e9d4 HTTP/1.1" 200 None 2026-03-13 00:50:29,246 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:50:29,268 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35934974c91e64560e49d HTTP/1.1" 200 None 2026-03-13 00:50:29,269 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:50:29,293 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b35ed0974c91e64560e9d4 HTTP/1.1" 200 None 2026-03-13 00:50:29,295 - lava_callback - INFO - Completed processing callback for node 69b35ed0974c91e64560e9d4 INFO: 172.20.0.1:50702 - "GET / HTTP/1.0" 200 OK 2026-03-13 00:50:44,682 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:50:44,699 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:43380 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41970 - "GET / HTTP/1.0" 200 OK 2026-03-13 00:51:00,946 - lava_callback - INFO - Received callback for node 69b35ea6974c91e64560e94d INFO: 172.20.0.1:47798 - "POST /node/69b35ea6974c91e64560e94d HTTP/1.0" 202 Accepted 2026-03-13 00:51:00,952 - lava_callback - INFO - Starting async job submit for node 69b35ea6974c91e64560e94d 2026-03-13 00:51:00,972 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:51:00,987 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35ea6974c91e64560e94d HTTP/1.1" 200 None 2026-03-13 00:51:00,988 - lava_callback - DEBUG - Processing node 69b35ea6974c91e64560e94d: baseline-arm-clabbe 2026-03-13 00:51:01,218 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 00:51:01,384 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 00:51:01,386 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69b35ea6974c91e64560e94d/log.txt.gz 2026-03-13 00:51:01,512 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 00:51:01,707 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 00:51:01,709 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69b35ea6974c91e64560e94d/lava-logs.yaml 2026-03-13 00:51:01,714 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 00:51:01,885 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 00:51:01,886 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69b35ea6974c91e64560e94d/lava_callback.json.gz 2026-03-13 00:51:01,909 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:51:01,924 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35ea6974c91e64560e94d HTTP/1.1" 200 None 2026-03-13 00:51:01,926 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:51:01,940 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35934974c91e64560e49d HTTP/1.1" 200 None 2026-03-13 00:51:01,942 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:51:01,978 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b35ea6974c91e64560e94d HTTP/1.1" 200 None 2026-03-13 00:51:01,980 - lava_callback - INFO - Completed processing callback for node 69b35ea6974c91e64560e94d INFO: 172.20.0.1:47808 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47822 - "GET / HTTP/1.0" 200 OK 2026-03-13 00:51:14,701 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:51:14,718 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:58402 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58406 - "GET / HTTP/1.0" 200 OK 2026-03-13 00:51:24,236 - lava_callback - INFO - Received callback for node 69b0bc899d1917581825268d INFO: 172.20.0.1:42814 - "POST /node/69b0bc899d1917581825268d HTTP/1.0" 202 Accepted 2026-03-13 00:51:24,242 - lava_callback - INFO - Starting async job submit for node 69b0bc899d1917581825268d 2026-03-13 00:51:24,243 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:51:24,271 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0bc899d1917581825268d HTTP/1.1" 200 None 2026-03-13 00:51:24,272 - lava_callback - DEBUG - Processing node 69b0bc899d1917581825268d: baseline-arm-clabbe 2026-03-13 00:51:24,272 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0bc899d1917581825268d 2026-03-13 00:51:24,273 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:51:24,326 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0bc899d1917581825268d HTTP/1.1" 200 None 2026-03-13 00:51:24,328 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:51:24,340 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11668ef4424e487b2ea2 HTTP/1.1" 200 None 2026-03-13 00:51:24,342 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:51:24,365 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0bc899d1917581825268d HTTP/1.1" 200 None 2026-03-13 00:51:24,367 - lava_callback - ERROR - Error processing callback for node 69b0bc899d1917581825268d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:42824 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59698 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57630 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43424 - "GET / HTTP/1.0" 200 OK 2026-03-13 00:52:04,283 - lava_callback - INFO - Received callback for node 69b0bcaf9d191758182526a4 INFO: 172.20.0.1:40926 - "POST /node/69b0bcaf9d191758182526a4 HTTP/1.0" 202 Accepted 2026-03-13 00:52:04,288 - lava_callback - INFO - Starting async job submit for node 69b0bcaf9d191758182526a4 2026-03-13 00:52:04,289 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:52:04,303 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0bcaf9d191758182526a4 HTTP/1.1" 200 None 2026-03-13 00:52:04,304 - lava_callback - DEBUG - Processing node 69b0bcaf9d191758182526a4: baseline-arm-clabbe 2026-03-13 00:52:04,305 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0bcaf9d191758182526a4 2026-03-13 00:52:04,305 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:52:04,318 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0bcaf9d191758182526a4 HTTP/1.1" 200 None 2026-03-13 00:52:04,320 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:52:04,333 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11668ef4424e487b2ea2 HTTP/1.1" 200 None 2026-03-13 00:52:04,334 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:52:04,355 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0bcaf9d191758182526a4 HTTP/1.1" 200 None 2026-03-13 00:52:04,361 - lava_callback - ERROR - Error processing callback for node 69b0bcaf9d191758182526a4: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-13 00:52:05,634 - lava_callback - INFO - Received callback for node 69b35ec8974c91e64560e9ac INFO: 172.20.0.1:40930 - "POST /node/69b35ec8974c91e64560e9ac HTTP/1.0" 202 Accepted 2026-03-13 00:52:05,641 - lava_callback - INFO - Starting async job submit for node 69b35ec8974c91e64560e9ac 2026-03-13 00:52:05,665 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:52:05,682 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35ec8974c91e64560e9ac HTTP/1.1" 200 None 2026-03-13 00:52:05,683 - lava_callback - DEBUG - Processing node 69b35ec8974c91e64560e9ac: ltp-cap_bounds 2026-03-13 00:52:06,028 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 00:52:06,326 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 00:52:06,328 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-cap_bounds-69b35ec8974c91e64560e9ac/log.txt.gz INFO: 172.20.0.1:40940 - "GET / HTTP/1.0" 200 OK 2026-03-13 00:52:06,534 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 00:52:06,793 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 00:52:06,795 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-cap_bounds-69b35ec8974c91e64560e9ac/lava-logs.yaml 2026-03-13 00:52:06,803 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 00:52:07,003 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 00:52:07,005 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-cap_bounds-69b35ec8974c91e64560e9ac/lava_callback.json.gz 2026-03-13 00:52:07,031 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:52:07,045 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35ec8974c91e64560e9ac HTTP/1.1" 200 None 2026-03-13 00:52:07,047 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:52:07,060 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35934974c91e64560e49d HTTP/1.1" 200 None 2026-03-13 00:52:07,062 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:52:07,089 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b35ec8974c91e64560e9ac HTTP/1.1" 200 None 2026-03-13 00:52:07,095 - lava_callback - INFO - Completed processing callback for node 69b35ec8974c91e64560e9ac INFO: 172.20.0.1:40954 - "GET / HTTP/1.0" 200 OK 2026-03-13 00:52:10,691 - lava_callback - INFO - Received callback for node 69b35ec9974c91e64560e9af INFO: 172.20.0.1:49464 - "POST /node/69b35ec9974c91e64560e9af HTTP/1.0" 202 Accepted 2026-03-13 00:52:10,697 - lava_callback - INFO - Starting async job submit for node 69b35ec9974c91e64560e9af 2026-03-13 00:52:10,721 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:52:10,735 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35ec9974c91e64560e9af HTTP/1.1" 200 None 2026-03-13 00:52:10,736 - lava_callback - DEBUG - Processing node 69b35ec9974c91e64560e9af: ltp-containers 2026-03-13 00:52:11,142 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 00:52:11,305 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 00:52:11,307 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-containers-69b35ec9974c91e64560e9af/log.txt.gz 2026-03-13 00:52:11,607 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 00:52:11,913 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 00:52:11,916 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-containers-69b35ec9974c91e64560e9af/lava-logs.yaml 2026-03-13 00:52:11,933 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 00:52:12,149 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 00:52:12,150 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-containers-69b35ec9974c91e64560e9af/lava_callback.json.gz 2026-03-13 00:52:12,177 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:52:12,191 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35ec9974c91e64560e9af HTTP/1.1" 200 None 2026-03-13 00:52:12,192 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:52:12,214 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35934974c91e64560e49d HTTP/1.1" 200 None 2026-03-13 00:52:12,216 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:52:12,245 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b35ec9974c91e64560e9af HTTP/1.1" 200 None 2026-03-13 00:52:12,247 - lava_callback - INFO - Completed processing callback for node 69b35ec9974c91e64560e9af 2026-03-13 00:52:14,720 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:52:14,736 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:49476 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49490 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47594 - "GET / HTTP/1.0" 200 OK 2026-03-13 00:52:29,809 - lava_callback - INFO - Received callback for node 69b35edd974c91e64560e9f4 INFO: 172.20.0.1:45070 - "POST /node/69b35edd974c91e64560e9f4 HTTP/1.0" 202 Accepted 2026-03-13 00:52:29,816 - lava_callback - INFO - Starting async job submit for node 69b35edd974c91e64560e9f4 2026-03-13 00:52:29,841 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:52:29,858 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35edd974c91e64560e9f4 HTTP/1.1" 200 None 2026-03-13 00:52:29,859 - lava_callback - DEBUG - Processing node 69b35edd974c91e64560e9f4: ltp-syscalls 2026-03-13 00:52:30,565 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 00:52:30,725 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 00:52:30,727 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-syscalls-69b35edd974c91e64560e9f4/log.txt.gz 2026-03-13 00:52:31,128 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 00:52:31,341 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 00:52:31,344 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-syscalls-69b35edd974c91e64560e9f4/lava-logs.yaml 2026-03-13 00:52:31,361 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 00:52:31,504 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 00:52:31,506 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-syscalls-69b35edd974c91e64560e9f4/lava_callback.json.gz 2026-03-13 00:52:31,531 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:52:31,545 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35edd974c91e64560e9f4 HTTP/1.1" 200 None 2026-03-13 00:52:31,547 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:52:31,560 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35934974c91e64560e49d HTTP/1.1" 200 None 2026-03-13 00:52:31,561 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:52:31,587 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b35edd974c91e64560e9f4 HTTP/1.1" 200 None 2026-03-13 00:52:31,589 - lava_callback - INFO - Completed processing callback for node 69b35edd974c91e64560e9f4 2026-03-13 00:52:36,002 - lava_callback - INFO - Received callback for node 69b35eab974c91e64560e959 INFO: 172.20.0.1:45080 - "POST /node/69b35eab974c91e64560e959 HTTP/1.0" 202 Accepted 2026-03-13 00:52:36,008 - lava_callback - INFO - Starting async job submit for node 69b35eab974c91e64560e959 2026-03-13 00:52:36,028 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:52:36,047 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35eab974c91e64560e959 HTTP/1.1" 200 None 2026-03-13 00:52:36,048 - lava_callback - DEBUG - Processing node 69b35eab974c91e64560e959: baseline-arm-clabbe 2026-03-13 00:52:36,270 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 00:52:36,383 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 00:52:36,385 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69b35eab974c91e64560e959/log.txt.gz 2026-03-13 00:52:36,509 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:45086 - "GET / HTTP/1.0" 200 OK 2026-03-13 00:52:36,682 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 00:52:36,684 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69b35eab974c91e64560e959/lava-logs.yaml 2026-03-13 00:52:36,690 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 00:52:36,816 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 00:52:36,818 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69b35eab974c91e64560e959/lava_callback.json.gz 2026-03-13 00:52:36,840 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:52:36,857 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35eab974c91e64560e959 HTTP/1.1" 200 None 2026-03-13 00:52:36,859 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:52:36,872 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35934974c91e64560e49d HTTP/1.1" 200 None 2026-03-13 00:52:36,874 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:52:36,897 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b35eab974c91e64560e959 HTTP/1.1" 200 None 2026-03-13 00:52:36,900 - lava_callback - INFO - Completed processing callback for node 69b35eab974c91e64560e959 2026-03-13 00:52:44,389 - lava_callback - INFO - Received callback for node 69b0bcd79d191758182526ab INFO: 172.20.0.1:38216 - "POST /node/69b0bcd79d191758182526ab HTTP/1.0" 202 Accepted 2026-03-13 00:52:44,395 - lava_callback - INFO - Starting async job submit for node 69b0bcd79d191758182526ab 2026-03-13 00:52:44,396 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:52:44,410 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0bcd79d191758182526ab HTTP/1.1" 200 None 2026-03-13 00:52:44,410 - lava_callback - DEBUG - Processing node 69b0bcd79d191758182526ab: baseline-arm-clabbe 2026-03-13 00:52:44,411 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0bcd79d191758182526ab 2026-03-13 00:52:44,411 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:52:44,424 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0bcd79d191758182526ab HTTP/1.1" 200 None 2026-03-13 00:52:44,426 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:52:44,439 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11668ef4424e487b2ea2 HTTP/1.1" 200 None 2026-03-13 00:52:44,440 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:52:44,467 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0bcd79d191758182526ab HTTP/1.1" 200 None 2026-03-13 00:52:44,473 - lava_callback - ERROR - Error processing callback for node 69b0bcd79d191758182526ab: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-13 00:52:44,738 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:52:44,754 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:38230 - "GET / HTTP/1.0" 200 OK 2026-03-13 00:52:50,593 - lava_callback - INFO - Received callback for node 69b35ecb974c91e64560e9b3 INFO: 172.20.0.1:48808 - "POST /node/69b35ecb974c91e64560e9b3 HTTP/1.0" 202 Accepted 2026-03-13 00:52:50,599 - lava_callback - INFO - Starting async job submit for node 69b35ecb974c91e64560e9b3 2026-03-13 00:52:50,624 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:52:50,638 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35ecb974c91e64560e9b3 HTTP/1.1" 200 None 2026-03-13 00:52:50,639 - lava_callback - DEBUG - Processing node 69b35ecb974c91e64560e9b3: ltp-controllers 2026-03-13 00:52:51,095 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 00:52:51,244 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 00:52:51,246 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-controllers-69b35ecb974c91e64560e9b3/log.txt.gz 2026-03-13 00:52:51,606 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 00:52:51,788 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 00:52:51,790 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-controllers-69b35ecb974c91e64560e9b3/lava-logs.yaml 2026-03-13 00:52:51,801 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 00:52:51,945 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 00:52:51,947 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-controllers-69b35ecb974c91e64560e9b3/lava_callback.json.gz 2026-03-13 00:52:51,984 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:52:52,000 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35ecb974c91e64560e9b3 HTTP/1.1" 200 None 2026-03-13 00:52:52,002 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:52:52,017 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35934974c91e64560e49d HTTP/1.1" 200 None 2026-03-13 00:52:52,019 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:52:52,042 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b35ecb974c91e64560e9b3 HTTP/1.1" 200 None 2026-03-13 00:52:52,044 - lava_callback - INFO - Completed processing callback for node 69b35ecb974c91e64560e9b3 INFO: 172.20.0.1:48822 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38018 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38034 - "GET / HTTP/1.0" 200 OK 2026-03-13 00:53:14,756 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:53:14,797 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:59284 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59288 - "GET / HTTP/1.0" 200 OK 2026-03-13 00:53:24,358 - lava_callback - INFO - Received callback for node 69b0bcfe9d191758182526c1 INFO: 172.20.0.1:35348 - "POST /node/69b0bcfe9d191758182526c1 HTTP/1.0" 202 Accepted 2026-03-13 00:53:24,363 - lava_callback - INFO - Starting async job submit for node 69b0bcfe9d191758182526c1 2026-03-13 00:53:24,364 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:53:24,378 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0bcfe9d191758182526c1 HTTP/1.1" 200 None 2026-03-13 00:53:24,378 - lava_callback - DEBUG - Processing node 69b0bcfe9d191758182526c1: baseline-arm-clabbe 2026-03-13 00:53:24,379 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0bcfe9d191758182526c1 2026-03-13 00:53:24,379 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:53:24,393 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0bcfe9d191758182526c1 HTTP/1.1" 200 None 2026-03-13 00:53:24,394 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:53:24,407 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11668ef4424e487b2ea2 HTTP/1.1" 200 None 2026-03-13 00:53:24,408 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:53:24,432 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0bcfe9d191758182526c1 HTTP/1.1" 200 None 2026-03-13 00:53:24,433 - lava_callback - ERROR - Error processing callback for node 69b0bcfe9d191758182526c1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:35354 - "GET / HTTP/1.0" 200 OK 2026-03-13 00:53:29,429 - lava_callback - INFO - Received callback for node 69b35ece974c91e64560e9b6 INFO: 172.20.0.1:35366 - "POST /node/69b35ece974c91e64560e9b6 HTTP/1.0" 202 Accepted 2026-03-13 00:53:29,435 - lava_callback - INFO - Starting async job submit for node 69b35ece974c91e64560e9b6 2026-03-13 00:53:29,460 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:53:29,474 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35ece974c91e64560e9b6 HTTP/1.1" 200 None 2026-03-13 00:53:29,475 - lava_callback - DEBUG - Processing node 69b35ece974c91e64560e9b6: ltp-cve 2026-03-13 00:53:29,834 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 00:53:29,997 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 00:53:29,999 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-cve-69b35ece974c91e64560e9b6/log.txt.gz 2026-03-13 00:53:30,217 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 00:53:30,445 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 00:53:30,447 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-cve-69b35ece974c91e64560e9b6/lava-logs.yaml 2026-03-13 00:53:30,456 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 00:53:30,599 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 00:53:30,601 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-cve-69b35ece974c91e64560e9b6/lava_callback.json.gz 2026-03-13 00:53:30,628 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:53:30,641 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35ece974c91e64560e9b6 HTTP/1.1" 200 None 2026-03-13 00:53:30,643 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:53:30,656 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35934974c91e64560e49d HTTP/1.1" 200 None 2026-03-13 00:53:30,657 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:53:30,682 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b35ece974c91e64560e9b6 HTTP/1.1" 200 None 2026-03-13 00:53:30,687 - lava_callback - INFO - Completed processing callback for node 69b35ece974c91e64560e9b6 INFO: 172.20.0.1:45870 - "GET / HTTP/1.0" 200 OK 2026-03-13 00:53:44,798 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:53:44,814 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:48862 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32992 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38550 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38560 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49236 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49240 - "GET / HTTP/1.0" 200 OK 2026-03-13 00:54:19,038 - lava_callback - INFO - Received callback for node 69b35eac974c91e64560e95c INFO: 172.20.0.1:49244 - "POST /node/69b35eac974c91e64560e95c HTTP/1.0" 202 Accepted 2026-03-13 00:54:19,044 - lava_callback - INFO - Starting async job submit for node 69b35eac974c91e64560e95c 2026-03-13 00:54:19,066 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:54:19,080 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35eac974c91e64560e95c HTTP/1.1" 200 None 2026-03-13 00:54:19,081 - lava_callback - DEBUG - Processing node 69b35eac974c91e64560e95c: baseline-arm-clabbe 2026-03-13 00:54:19,299 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 00:54:19,418 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 00:54:19,420 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69b35eac974c91e64560e95c/log.txt.gz 2026-03-13 00:54:19,566 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 00:54:19,747 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 00:54:19,749 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69b35eac974c91e64560e95c/lava-logs.yaml 2026-03-13 00:54:19,755 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 00:54:19,875 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 00:54:19,877 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69b35eac974c91e64560e95c/lava_callback.json.gz 2026-03-13 00:54:19,900 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:54:19,915 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35eac974c91e64560e95c HTTP/1.1" 200 None 2026-03-13 00:54:19,916 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:54:19,933 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35934974c91e64560e49d HTTP/1.1" 200 None 2026-03-13 00:54:19,935 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:54:19,958 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b35eac974c91e64560e95c HTTP/1.1" 200 None 2026-03-13 00:54:19,968 - lava_callback - INFO - Completed processing callback for node 69b35eac974c91e64560e95c INFO: 172.20.0.1:57140 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46856 - "GET / HTTP/1.0" 200 OK 2026-03-13 00:54:44,432 - lava_callback - INFO - Received callback for node 69b0bd4a9d191758182526eb INFO: 172.20.0.1:56136 - "POST /node/69b0bd4a9d191758182526eb HTTP/1.0" 202 Accepted 2026-03-13 00:54:44,438 - lava_callback - INFO - Starting async job submit for node 69b0bd4a9d191758182526eb 2026-03-13 00:54:44,439 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:54:44,453 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0bd4a9d191758182526eb HTTP/1.1" 200 None 2026-03-13 00:54:44,454 - lava_callback - DEBUG - Processing node 69b0bd4a9d191758182526eb: baseline-arm-clabbe 2026-03-13 00:54:44,454 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0bd4a9d191758182526eb 2026-03-13 00:54:44,455 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:54:44,489 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0bd4a9d191758182526eb HTTP/1.1" 200 None 2026-03-13 00:54:44,490 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:54:44,505 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11668ef4424e487b2ea2 HTTP/1.1" 200 None 2026-03-13 00:54:44,506 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:54:44,527 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0bd4a9d191758182526eb HTTP/1.1" 200 None 2026-03-13 00:54:44,528 - lava_callback - ERROR - Error processing callback for node 69b0bd4a9d191758182526eb: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-13 00:54:44,816 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:54:44,836 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:56146 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53970 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40210 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40218 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40942 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40952 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42762 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50932 - "GET / HTTP/1.0" 200 OK 2026-03-13 00:55:38,842 - lava_callback - INFO - Received callback for node 69b3606c974c91e64560eb26 INFO: 172.20.0.1:50940 - "POST /node/69b3606c974c91e64560eb26 HTTP/1.0" 202 Accepted 2026-03-13 00:55:38,848 - lava_callback - INFO - Starting async job submit for node 69b3606c974c91e64560eb26 2026-03-13 00:55:38,851 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:55:38,865 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3606c974c91e64560eb26 HTTP/1.1" 200 None 2026-03-13 00:55:38,866 - lava_callback - DEBUG - Processing node 69b3606c974c91e64560eb26: baseline-arm 2026-03-13 00:55:38,871 - lava_callback - WARNING - Failed to upload log 2026-03-13 00:55:38,875 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 00:55:39,038 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 00:55:39,040 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69b3606c974c91e64560eb26/lava-logs.yaml 2026-03-13 00:55:39,042 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 00:55:39,183 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 00:55:39,185 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69b3606c974c91e64560eb26/lava_callback.json.gz 2026-03-13 00:55:39,188 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:55:39,202 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3606c974c91e64560eb26 HTTP/1.1" 200 None 2026-03-13 00:55:39,204 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:55:39,217 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35934974c91e64560e49d HTTP/1.1" 200 None 2026-03-13 00:55:39,219 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:55:39,243 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b3606c974c91e64560eb26 HTTP/1.1" 200 None 2026-03-13 00:55:39,257 - lava_callback - INFO - Completed processing callback for node 69b3606c974c91e64560eb26 2026-03-13 00:55:44,838 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:55:44,863 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:38396 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56126 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33520 - "GET / HTTP/1.0" 200 OK 2026-03-13 00:56:08,211 - lava_callback - INFO - Received callback for node 69b35ed4974c91e64560e9da INFO: 172.20.0.1:33526 - "POST /node/69b35ed4974c91e64560e9da HTTP/1.0" 202 Accepted 2026-03-13 00:56:08,221 - lava_callback - INFO - Starting async job submit for node 69b35ed4974c91e64560e9da 2026-03-13 00:56:08,271 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:56:08,297 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35ed4974c91e64560e9da HTTP/1.1" 200 None 2026-03-13 00:56:08,298 - lava_callback - DEBUG - Processing node 69b35ed4974c91e64560e9da: ltp-fsx INFO: 172.20.0.1:33534 - "GET / HTTP/1.0" 200 OK 2026-03-13 00:56:08,912 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 00:56:09,239 - lava_callback - INFO - Received callback for node 69b35ed2974c91e64560e9d7 INFO: 172.20.0.1:33550 - "POST /node/69b35ed2974c91e64560e9d7 HTTP/1.0" 202 Accepted 2026-03-13 00:56:09,248 - lava_callback - INFO - Starting async job submit for node 69b35ed2974c91e64560e9d7 2026-03-13 00:56:09,287 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 00:56:09,290 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-fsx-69b35ed4974c91e64560e9da/log.txt.gz 2026-03-13 00:56:09,325 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:56:09,384 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35ed2974c91e64560e9d7 HTTP/1.1" 200 None 2026-03-13 00:56:09,385 - lava_callback - DEBUG - Processing node 69b35ed2974c91e64560e9d7: ltp-fs_bind 2026-03-13 00:56:09,998 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 00:56:10,391 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 00:56:10,757 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 00:56:10,760 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-fsx-69b35ed4974c91e64560e9da/lava-logs.yaml 2026-03-13 00:56:10,770 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 00:56:10,896 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 00:56:10,898 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-fs_bind-69b35ed2974c91e64560e9d7/log.txt.gz 2026-03-13 00:56:11,281 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 00:56:11,295 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 00:56:11,297 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-fsx-69b35ed4974c91e64560e9da/lava_callback.json.gz 2026-03-13 00:56:11,350 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:56:11,382 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35ed4974c91e64560e9da HTTP/1.1" 200 None 2026-03-13 00:56:11,385 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:56:11,407 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35934974c91e64560e49d HTTP/1.1" 200 None 2026-03-13 00:56:11,410 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:56:11,444 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b35ed4974c91e64560e9da HTTP/1.1" 200 None 2026-03-13 00:56:11,445 - lava_callback - INFO - Completed processing callback for node 69b35ed4974c91e64560e9da 2026-03-13 00:56:12,080 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 00:56:12,082 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-fs_bind-69b35ed2974c91e64560e9d7/lava-logs.yaml 2026-03-13 00:56:12,096 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 00:56:12,530 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 00:56:12,532 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-fs_bind-69b35ed2974c91e64560e9d7/lava_callback.json.gz 2026-03-13 00:56:12,575 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:56:12,602 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35ed2974c91e64560e9d7 HTTP/1.1" 200 None 2026-03-13 00:56:12,605 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:56:12,634 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35934974c91e64560e49d HTTP/1.1" 200 None 2026-03-13 00:56:12,637 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:56:12,683 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b35ed2974c91e64560e9d7 HTTP/1.1" 200 None 2026-03-13 00:56:12,684 - lava_callback - INFO - Completed processing callback for node 69b35ed2974c91e64560e9d7 2026-03-13 00:56:14,865 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:56:14,890 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:56536 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56542 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58248 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43852 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41066 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49930 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34128 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34138 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50800 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50802 - "GET / HTTP/1.0" 200 OK 2026-03-13 00:57:24,686 - lava_callback - INFO - Received callback for node 69b35c56974c91e64560e4e9 INFO: 172.20.0.1:53854 - "POST /node/69b35c56974c91e64560e4e9 HTTP/1.0" 202 Accepted 2026-03-13 00:57:24,693 - lava_callback - INFO - Starting async job submit for node 69b35c56974c91e64560e4e9 2026-03-13 00:57:24,749 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:57:24,766 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35c56974c91e64560e4e9 HTTP/1.1" 200 None 2026-03-13 00:57:24,767 - lava_callback - DEBUG - Processing node 69b35c56974c91e64560e4e9: ltp-smoke 2026-03-13 00:57:25,232 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 00:57:25,372 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 00:57:25,374 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoke-69b35c56974c91e64560e4e9/log.txt.gz 2026-03-13 00:57:25,678 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 00:57:25,881 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 00:57:25,883 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-smoke-69b35c56974c91e64560e4e9/lava-logs.yaml 2026-03-13 00:57:25,895 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 00:57:26,073 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 00:57:26,075 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoke-69b35c56974c91e64560e4e9/lava_callback.json.gz 2026-03-13 00:57:26,195 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:57:26,210 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35c56974c91e64560e4e9 HTTP/1.1" 200 None 2026-03-13 00:57:26,211 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:57:26,225 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3593e974c91e64560e4a3 HTTP/1.1" 200 None 2026-03-13 00:57:26,226 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:57:26,263 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b35c56974c91e64560e4e9 HTTP/1.1" 200 None 2026-03-13 00:57:26,393 - lava_callback - INFO - Completed processing callback for node 69b35c56974c91e64560e4e9 INFO: 172.20.0.1:53868 - "GET / HTTP/1.0" 200 OK 2026-03-13 00:57:29,317 - lava_callback - INFO - Received callback for node 69b35ed5974c91e64560e9df INFO: 172.20.0.1:53880 - "POST /node/69b35ed5974c91e64560e9df HTTP/1.0" 202 Accepted 2026-03-13 00:57:29,324 - lava_callback - INFO - Starting async job submit for node 69b35ed5974c91e64560e9df 2026-03-13 00:57:29,352 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:57:29,367 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35ed5974c91e64560e9df HTTP/1.1" 200 None 2026-03-13 00:57:29,367 - lava_callback - DEBUG - Processing node 69b35ed5974c91e64560e9df: ltp-input 2026-03-13 00:57:29,733 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 00:57:29,875 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 00:57:29,877 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-input-69b35ed5974c91e64560e9df/log.txt.gz 2026-03-13 00:57:30,114 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 00:57:30,306 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 00:57:30,308 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-input-69b35ed5974c91e64560e9df/lava-logs.yaml 2026-03-13 00:57:30,317 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 00:57:30,503 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 00:57:30,505 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-input-69b35ed5974c91e64560e9df/lava_callback.json.gz 2026-03-13 00:57:30,531 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:57:30,546 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35ed5974c91e64560e9df HTTP/1.1" 200 None 2026-03-13 00:57:30,547 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:57:30,561 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35934974c91e64560e49d HTTP/1.1" 200 None 2026-03-13 00:57:30,562 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:57:30,596 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b35ed5974c91e64560e9df HTTP/1.1" 200 None 2026-03-13 00:57:30,597 - lava_callback - INFO - Completed processing callback for node 69b35ed5974c91e64560e9df 2026-03-13 00:57:30,838 - lava_callback - INFO - Received callback for node 69b35ed7974c91e64560e9e2 INFO: 172.20.0.1:47636 - "POST /node/69b35ed7974c91e64560e9e2 HTTP/1.0" 202 Accepted 2026-03-13 00:57:30,845 - lava_callback - INFO - Starting async job submit for node 69b35ed7974c91e64560e9e2 2026-03-13 00:57:30,873 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:57:30,887 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35ed7974c91e64560e9e2 HTTP/1.1" 200 None 2026-03-13 00:57:30,888 - lava_callback - DEBUG - Processing node 69b35ed7974c91e64560e9e2: ltp-ipc 2026-03-13 00:57:31,242 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 00:57:31,354 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 00:57:31,356 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-ipc-69b35ed7974c91e64560e9e2/log.txt.gz 2026-03-13 00:57:31,561 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 00:57:31,756 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 00:57:31,759 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-ipc-69b35ed7974c91e64560e9e2/lava-logs.yaml 2026-03-13 00:57:31,767 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 00:57:31,912 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 00:57:31,914 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-ipc-69b35ed7974c91e64560e9e2/lava_callback.json.gz 2026-03-13 00:57:31,940 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:57:31,957 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35ed7974c91e64560e9e2 HTTP/1.1" 200 None 2026-03-13 00:57:31,958 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:57:31,972 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35934974c91e64560e49d HTTP/1.1" 200 None 2026-03-13 00:57:31,973 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:57:32,006 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b35ed7974c91e64560e9e2 HTTP/1.1" 200 None 2026-03-13 00:57:32,007 - lava_callback - INFO - Completed processing callback for node 69b35ed7974c91e64560e9e2 INFO: 172.20.0.1:47644 - "GET / HTTP/1.0" 200 OK 2026-03-13 00:57:44,893 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:57:44,932 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:60020 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36390 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38768 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38776 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49258 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49274 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40670 - "GET / HTTP/1.0" 200 OK 2026-03-13 00:58:26,560 - lava_callback - INFO - Received callback for node 69b0be2a9d191758182527e0 INFO: 172.20.0.1:40676 - "POST /node/69b0be2a9d191758182527e0 HTTP/1.0" 202 Accepted 2026-03-13 00:58:26,584 - lava_callback - INFO - Starting async job submit for node 69b0be2a9d191758182527e0 2026-03-13 00:58:26,586 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:58:26,619 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0be2a9d191758182527e0 HTTP/1.1" 200 None 2026-03-13 00:58:26,622 - lava_callback - DEBUG - Processing node 69b0be2a9d191758182527e0: baseline-arm64-baylibre 2026-03-13 00:58:26,622 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0be2a9d191758182527e0 2026-03-13 00:58:26,623 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:58:26,654 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0be2a9d191758182527e0 HTTP/1.1" 200 None 2026-03-13 00:58:26,657 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:58:26,682 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b5149d19175818251feb HTTP/1.1" 200 None 2026-03-13 00:58:26,684 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:58:26,723 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0be2a9d191758182527e0 HTTP/1.1" 200 None 2026-03-13 00:58:26,725 - lava_callback - ERROR - Error processing callback for node 69b0be2a9d191758182527e0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-13 00:58:27,232 - lava_callback - INFO - Received callback for node 69b0be2b9d191758182527e3 INFO: 172.20.0.1:40678 - "POST /node/69b0be2b9d191758182527e3 HTTP/1.0" 202 Accepted 2026-03-13 00:58:27,241 - lava_callback - INFO - Starting async job submit for node 69b0be2b9d191758182527e3 2026-03-13 00:58:27,243 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:58:27,269 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0be2b9d191758182527e3 HTTP/1.1" 200 None 2026-03-13 00:58:27,270 - lava_callback - DEBUG - Processing node 69b0be2b9d191758182527e3: baseline-arm64-baylibre 2026-03-13 00:58:27,270 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0be2b9d191758182527e3 2026-03-13 00:58:27,271 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:58:27,297 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0be2b9d191758182527e3 HTTP/1.1" 200 None 2026-03-13 00:58:27,300 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:58:27,328 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b5149d19175818251feb HTTP/1.1" 200 None 2026-03-13 00:58:27,331 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:58:27,361 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0be2b9d191758182527e3 HTTP/1.1" 200 None 2026-03-13 00:58:27,363 - lava_callback - ERROR - Error processing callback for node 69b0be2b9d191758182527e3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-13 00:58:27,902 - lava_callback - INFO - Received callback for node 69b0be2f9d191758182527f0 INFO: 172.20.0.1:40680 - "POST /node/69b0be2f9d191758182527f0 HTTP/1.0" 202 Accepted 2026-03-13 00:58:27,910 - lava_callback - INFO - Starting async job submit for node 69b0be2f9d191758182527f0 2026-03-13 00:58:27,917 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:58:27,940 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0be2f9d191758182527f0 HTTP/1.1" 200 None 2026-03-13 00:58:27,942 - lava_callback - DEBUG - Processing node 69b0be2f9d191758182527f0: baseline-arm64-baylibre 2026-03-13 00:58:27,942 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0be2f9d191758182527f0 2026-03-13 00:58:27,943 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:58:27,980 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0be2f9d191758182527f0 HTTP/1.1" 200 None 2026-03-13 00:58:27,982 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:58:28,011 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b5149d19175818251feb HTTP/1.1" 200 None 2026-03-13 00:58:28,014 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:58:28,062 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0be2f9d191758182527f0 HTTP/1.1" 200 None 2026-03-13 00:58:28,064 - lava_callback - ERROR - Error processing callback for node 69b0be2f9d191758182527f0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-13 00:58:28,577 - lava_callback - INFO - Received callback for node 69b0be2d9d191758182527e9 INFO: 172.20.0.1:40696 - "POST /node/69b0be2d9d191758182527e9 HTTP/1.0" 202 Accepted 2026-03-13 00:58:28,588 - lava_callback - INFO - Starting async job submit for node 69b0be2d9d191758182527e9 2026-03-13 00:58:28,590 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:58:28,625 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0be2d9d191758182527e9 HTTP/1.1" 200 None 2026-03-13 00:58:28,627 - lava_callback - DEBUG - Processing node 69b0be2d9d191758182527e9: baseline-arm64-baylibre 2026-03-13 00:58:28,627 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0be2d9d191758182527e9 2026-03-13 00:58:28,628 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:58:28,668 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0be2d9d191758182527e9 HTTP/1.1" 200 None 2026-03-13 00:58:28,670 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:58:28,694 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b5149d19175818251feb HTTP/1.1" 200 None 2026-03-13 00:58:28,697 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:58:28,735 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0be2d9d191758182527e9 HTTP/1.1" 200 None 2026-03-13 00:58:28,737 - lava_callback - ERROR - Error processing callback for node 69b0be2d9d191758182527e9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:44996 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44872 - "GET / HTTP/1.0" 200 OK 2026-03-13 00:58:46,554 - lava_callback - INFO - Received callback for node 69b0be329d191758182527f4 INFO: 172.20.0.1:44882 - "POST /node/69b0be329d191758182527f4 HTTP/1.0" 202 Accepted 2026-03-13 00:58:46,562 - lava_callback - INFO - Starting async job submit for node 69b0be329d191758182527f4 2026-03-13 00:58:46,568 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:58:46,602 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0be329d191758182527f4 HTTP/1.1" 200 None 2026-03-13 00:58:46,603 - lava_callback - DEBUG - Processing node 69b0be329d191758182527f4: baseline-arm64-baylibre 2026-03-13 00:58:46,604 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0be329d191758182527f4 2026-03-13 00:58:46,604 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:58:46,626 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0be329d191758182527f4 HTTP/1.1" 200 None 2026-03-13 00:58:46,628 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:58:46,654 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b5149d19175818251feb HTTP/1.1" 200 None 2026-03-13 00:58:46,656 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:58:46,707 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0be329d191758182527f4 HTTP/1.1" 200 None 2026-03-13 00:58:46,709 - lava_callback - ERROR - Error processing callback for node 69b0be329d191758182527f4: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:50076 - "GET / HTTP/1.0" 200 OK 2026-03-13 00:59:04,608 - lava_callback - INFO - Received callback for node 69b0be439d19175818252842 INFO: 172.20.0.1:45246 - "POST /node/69b0be439d19175818252842 HTTP/1.0" 202 Accepted 2026-03-13 00:59:04,614 - lava_callback - INFO - Starting async job submit for node 69b0be439d19175818252842 2026-03-13 00:59:04,614 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:59:04,628 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0be439d19175818252842 HTTP/1.1" 200 None 2026-03-13 00:59:04,629 - lava_callback - DEBUG - Processing node 69b0be439d19175818252842: baseline-arm64-clabbe 2026-03-13 00:59:04,629 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0be439d19175818252842 2026-03-13 00:59:04,630 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:59:04,652 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0be439d19175818252842 HTTP/1.1" 200 None 2026-03-13 00:59:04,653 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:59:04,667 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b5149d19175818251feb HTTP/1.1" 200 None 2026-03-13 00:59:04,668 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:59:04,688 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0be439d19175818252842 HTTP/1.1" 200 None 2026-03-13 00:59:04,694 - lava_callback - ERROR - Error processing callback for node 69b0be439d19175818252842: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-13 00:59:05,270 - lava_callback - INFO - Received callback for node 69b0be449d19175818252845 INFO: 172.20.0.1:45252 - "POST /node/69b0be449d19175818252845 HTTP/1.0" 202 Accepted 2026-03-13 00:59:05,275 - lava_callback - INFO - Starting async job submit for node 69b0be449d19175818252845 2026-03-13 00:59:05,276 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:59:05,290 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0be449d19175818252845 HTTP/1.1" 200 None 2026-03-13 00:59:05,291 - lava_callback - DEBUG - Processing node 69b0be449d19175818252845: baseline-arm64-clabbe 2026-03-13 00:59:05,291 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0be449d19175818252845 2026-03-13 00:59:05,292 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:59:05,305 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0be449d19175818252845 HTTP/1.1" 200 None 2026-03-13 00:59:05,307 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:59:05,320 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b5149d19175818251feb HTTP/1.1" 200 None 2026-03-13 00:59:05,321 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:59:05,347 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0be449d19175818252845 HTTP/1.1" 200 None 2026-03-13 00:59:05,349 - lava_callback - ERROR - Error processing callback for node 69b0be449d19175818252845: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-13 00:59:05,923 - lava_callback - INFO - Received callback for node 69b0be459d19175818252848 INFO: 172.20.0.1:45262 - "POST /node/69b0be459d19175818252848 HTTP/1.0" 202 Accepted 2026-03-13 00:59:05,928 - lava_callback - INFO - Starting async job submit for node 69b0be459d19175818252848 2026-03-13 00:59:05,929 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:59:05,946 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0be459d19175818252848 HTTP/1.1" 200 None 2026-03-13 00:59:05,947 - lava_callback - DEBUG - Processing node 69b0be459d19175818252848: baseline-arm64-clabbe 2026-03-13 00:59:05,948 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0be459d19175818252848 2026-03-13 00:59:05,948 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:59:05,969 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0be459d19175818252848 HTTP/1.1" 200 None 2026-03-13 00:59:05,971 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:59:05,985 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b5149d19175818251feb HTTP/1.1" 200 None 2026-03-13 00:59:05,986 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:59:06,006 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0be459d19175818252848 HTTP/1.1" 200 None 2026-03-13 00:59:06,012 - lava_callback - ERROR - Error processing callback for node 69b0be459d19175818252848: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:45266 - "GET / HTTP/1.0" 200 OK 2026-03-13 00:59:06,594 - lava_callback - INFO - Received callback for node 69b0be469d1917581825284b INFO: 172.20.0.1:45272 - "POST /node/69b0be469d1917581825284b HTTP/1.0" 202 Accepted 2026-03-13 00:59:06,600 - lava_callback - INFO - Starting async job submit for node 69b0be469d1917581825284b 2026-03-13 00:59:06,601 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:59:06,620 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0be469d1917581825284b HTTP/1.1" 200 None 2026-03-13 00:59:06,621 - lava_callback - DEBUG - Processing node 69b0be469d1917581825284b: baseline-arm64-clabbe 2026-03-13 00:59:06,622 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0be469d1917581825284b 2026-03-13 00:59:06,622 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:59:06,643 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0be469d1917581825284b HTTP/1.1" 200 None 2026-03-13 00:59:06,644 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:59:06,658 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b5149d19175818251feb HTTP/1.1" 200 None 2026-03-13 00:59:06,659 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:59:06,680 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0be469d1917581825284b HTTP/1.1" 200 None 2026-03-13 00:59:06,686 - lava_callback - ERROR - Error processing callback for node 69b0be469d1917581825284b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-13 00:59:07,258 - lava_callback - INFO - Received callback for node 69b0be499d19175818252854 INFO: 172.20.0.1:45278 - "POST /node/69b0be499d19175818252854 HTTP/1.0" 202 Accepted 2026-03-13 00:59:07,263 - lava_callback - INFO - Starting async job submit for node 69b0be499d19175818252854 2026-03-13 00:59:07,264 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:59:07,279 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0be499d19175818252854 HTTP/1.1" 200 None 2026-03-13 00:59:07,280 - lava_callback - DEBUG - Processing node 69b0be499d19175818252854: baseline-arm64-clabbe 2026-03-13 00:59:07,280 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0be499d19175818252854 2026-03-13 00:59:07,281 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:59:07,294 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0be499d19175818252854 HTTP/1.1" 200 None 2026-03-13 00:59:07,295 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:59:07,312 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b5149d19175818251feb HTTP/1.1" 200 None 2026-03-13 00:59:07,313 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:59:07,333 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0be499d19175818252854 HTTP/1.1" 200 None 2026-03-13 00:59:07,338 - lava_callback - ERROR - Error processing callback for node 69b0be499d19175818252854: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-13 00:59:07,927 - lava_callback - INFO - Received callback for node 69b0be489d19175818252851 INFO: 172.20.0.1:45280 - "POST /node/69b0be489d19175818252851 HTTP/1.0" 202 Accepted 2026-03-13 00:59:07,932 - lava_callback - INFO - Starting async job submit for node 69b0be489d19175818252851 2026-03-13 00:59:07,933 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:59:07,947 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0be489d19175818252851 HTTP/1.1" 200 None 2026-03-13 00:59:07,948 - lava_callback - DEBUG - Processing node 69b0be489d19175818252851: baseline-arm64-clabbe 2026-03-13 00:59:07,948 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0be489d19175818252851 2026-03-13 00:59:07,949 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:59:07,962 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0be489d19175818252851 HTTP/1.1" 200 None 2026-03-13 00:59:07,964 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:59:07,978 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b5149d19175818251feb HTTP/1.1" 200 None 2026-03-13 00:59:07,980 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:59:08,001 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0be489d19175818252851 HTTP/1.1" 200 None 2026-03-13 00:59:08,007 - lava_callback - ERROR - Error processing callback for node 69b0be489d19175818252851: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:45282 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55860 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55864 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51444 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49094 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46744 - "GET / HTTP/1.0" 200 OK 2026-03-13 00:59:46,663 - lava_callback - INFO - Received callback for node 69b0be6f9d191758182528b3 INFO: 172.20.0.1:46758 - "POST /node/69b0be6f9d191758182528b3 HTTP/1.0" 202 Accepted 2026-03-13 00:59:46,669 - lava_callback - INFO - Starting async job submit for node 69b0be6f9d191758182528b3 2026-03-13 00:59:46,669 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:59:46,684 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0be6f9d191758182528b3 HTTP/1.1" 200 None 2026-03-13 00:59:46,685 - lava_callback - DEBUG - Processing node 69b0be6f9d191758182528b3: baseline-arm-baylibre 2026-03-13 00:59:46,685 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0be6f9d191758182528b3 2026-03-13 00:59:46,685 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:59:46,698 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0be6f9d191758182528b3 HTTP/1.1" 200 None 2026-03-13 00:59:46,700 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:59:46,714 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e34efa461f55e59cdb7 HTTP/1.1" 200 None 2026-03-13 00:59:46,716 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 00:59:46,742 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0be6f9d191758182528b3 HTTP/1.1" 200 None 2026-03-13 00:59:46,748 - lava_callback - ERROR - Error processing callback for node 69b0be6f9d191758182528b3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:40984 - "GET / HTTP/1.0" 200 OK 2026-03-13 01:00:04,400 - lava_callback - INFO - Received callback for node 69b35ed8974c91e64560e9e5 INFO: 172.20.0.1:37618 - "POST /node/69b35ed8974c91e64560e9e5 HTTP/1.0" 202 Accepted 2026-03-13 01:00:04,406 - lava_callback - INFO - Starting async job submit for node 69b35ed8974c91e64560e9e5 2026-03-13 01:00:04,431 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:00:04,445 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35ed8974c91e64560e9e5 HTTP/1.1" 200 None 2026-03-13 01:00:04,446 - lava_callback - DEBUG - Processing node 69b35ed8974c91e64560e9e5: ltp-pty 2026-03-13 01:00:04,821 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:00:04,972 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:00:04,974 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-pty-69b35ed8974c91e64560e9e5/log.txt.gz 2026-03-13 01:00:05,176 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:00:05,386 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:00:05,389 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-pty-69b35ed8974c91e64560e9e5/lava-logs.yaml 2026-03-13 01:00:05,400 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:00:05,586 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:00:05,588 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-pty-69b35ed8974c91e64560e9e5/lava_callback.json.gz 2026-03-13 01:00:05,613 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:00:05,628 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35ed8974c91e64560e9e5 HTTP/1.1" 200 None 2026-03-13 01:00:05,629 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:00:05,642 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35934974c91e64560e49d HTTP/1.1" 200 None 2026-03-13 01:00:05,644 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:00:05,673 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b35ed8974c91e64560e9e5 HTTP/1.1" 200 None 2026-03-13 01:00:05,676 - lava_callback - INFO - Completed processing callback for node 69b35ed8974c91e64560e9e5 INFO: 172.20.0.1:37630 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37636 - "GET / HTTP/1.0" 200 OK 2026-03-13 01:00:09,867 - lava_callback - INFO - Received callback for node 69b35eda974c91e64560e9e8 INFO: 172.20.0.1:46288 - "POST /node/69b35eda974c91e64560e9e8 HTTP/1.0" 202 Accepted 2026-03-13 01:00:09,873 - lava_callback - INFO - Starting async job submit for node 69b35eda974c91e64560e9e8 2026-03-13 01:00:09,906 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:00:09,921 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35eda974c91e64560e9e8 HTTP/1.1" 200 None 2026-03-13 01:00:09,922 - lava_callback - DEBUG - Processing node 69b35eda974c91e64560e9e8: ltp-sched 2026-03-13 01:00:10,266 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:00:10,405 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:00:10,406 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-sched-69b35eda974c91e64560e9e8/log.txt.gz 2026-03-13 01:00:10,623 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:00:10,804 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:00:10,806 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-sched-69b35eda974c91e64560e9e8/lava-logs.yaml 2026-03-13 01:00:10,815 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:00:10,979 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:00:10,981 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-sched-69b35eda974c91e64560e9e8/lava_callback.json.gz 2026-03-13 01:00:11,007 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:00:11,025 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35eda974c91e64560e9e8 HTTP/1.1" 200 None 2026-03-13 01:00:11,026 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:00:11,040 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35934974c91e64560e49d HTTP/1.1" 200 None 2026-03-13 01:00:11,042 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:00:11,067 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b35eda974c91e64560e9e8 HTTP/1.1" 200 None 2026-03-13 01:00:11,073 - lava_callback - INFO - Completed processing callback for node 69b35eda974c91e64560e9e8 2026-03-13 01:00:14,935 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:00:14,952 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:46298 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46300 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51192 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41264 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37788 - "GET / HTTP/1.0" 200 OK 2026-03-13 01:00:46,686 - lava_callback - INFO - Received callback for node 69b0beb49d19175818252960 INFO: 172.20.0.1:37800 - "POST /node/69b0beb49d19175818252960 HTTP/1.0" 202 Accepted 2026-03-13 01:00:46,692 - lava_callback - INFO - Starting async job submit for node 69b0beb49d19175818252960 2026-03-13 01:00:46,693 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:00:46,707 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0beb49d19175818252960 HTTP/1.1" 200 None 2026-03-13 01:00:46,708 - lava_callback - DEBUG - Processing node 69b0beb49d19175818252960: baseline-arm-baylibre 2026-03-13 01:00:46,708 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0beb49d19175818252960 2026-03-13 01:00:46,708 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:00:46,722 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0beb49d19175818252960 HTTP/1.1" 200 None 2026-03-13 01:00:46,723 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:00:46,742 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e34efa461f55e59cdb7 HTTP/1.1" 200 None 2026-03-13 01:00:46,744 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:00:46,792 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0beb49d19175818252960 HTTP/1.1" 200 None 2026-03-13 01:00:46,794 - lava_callback - ERROR - Error processing callback for node 69b0beb49d19175818252960: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:58674 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48308 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48312 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50652 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50668 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39672 - "GET / HTTP/1.0" 200 OK 2026-03-13 01:01:29,614 - lava_callback - INFO - Received callback for node 69b35edc974c91e64560e9ed INFO: 172.20.0.1:39688 - "POST /node/69b35edc974c91e64560e9ed HTTP/1.0" 202 Accepted 2026-03-13 01:01:29,620 - lava_callback - INFO - Starting async job submit for node 69b35edc974c91e64560e9ed 2026-03-13 01:01:29,645 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:01:29,658 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35edc974c91e64560e9ed HTTP/1.1" 200 None 2026-03-13 01:01:29,660 - lava_callback - DEBUG - Processing node 69b35edc974c91e64560e9ed: ltp-smoke 2026-03-13 01:01:30,052 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:01:31,213 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:01:31,215 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoke-69b35edc974c91e64560e9ed/log.txt.gz 2026-03-13 01:01:31,418 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:01:31,642 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:01:31,644 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-smoke-69b35edc974c91e64560e9ed/lava-logs.yaml 2026-03-13 01:01:31,653 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:01:31,824 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:01:31,826 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoke-69b35edc974c91e64560e9ed/lava_callback.json.gz 2026-03-13 01:01:31,852 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:01:31,873 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35edc974c91e64560e9ed HTTP/1.1" 200 None 2026-03-13 01:01:31,875 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:01:31,889 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35934974c91e64560e49d HTTP/1.1" 200 None 2026-03-13 01:01:31,891 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:01:31,926 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b35edc974c91e64560e9ed HTTP/1.1" 200 None 2026-03-13 01:01:31,928 - lava_callback - INFO - Completed processing callback for node 69b35edc974c91e64560e9ed INFO: 172.20.0.1:55668 - "GET / HTTP/1.0" 200 OK 2026-03-13 01:01:44,954 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:01:44,971 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:58220 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45308 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41410 - "GET / HTTP/1.0" 200 OK 2026-03-13 01:02:06,846 - lava_callback - INFO - Received callback for node 69b0befc9d191758182529a4 INFO: 172.20.0.1:41424 - "POST /node/69b0befc9d191758182529a4 HTTP/1.0" 202 Accepted 2026-03-13 01:02:06,851 - lava_callback - INFO - Starting async job submit for node 69b0befc9d191758182529a4 2026-03-13 01:02:06,853 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:02:06,872 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0befc9d191758182529a4 HTTP/1.1" 200 None 2026-03-13 01:02:06,873 - lava_callback - DEBUG - Processing node 69b0befc9d191758182529a4: baseline-arm-baylibre 2026-03-13 01:02:06,874 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0befc9d191758182529a4 2026-03-13 01:02:06,874 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:02:06,889 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0befc9d191758182529a4 HTTP/1.1" 200 None 2026-03-13 01:02:06,890 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:02:06,906 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e34efa461f55e59cdb7 HTTP/1.1" 200 None 2026-03-13 01:02:06,909 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:02:06,939 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0befc9d191758182529a4 HTTP/1.1" 200 None 2026-03-13 01:02:06,945 - lava_callback - ERROR - Error processing callback for node 69b0befc9d191758182529a4: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:41436 - "GET / HTTP/1.0" 200 OK 2026-03-13 01:02:09,751 - lava_callback - INFO - Received callback for node 69b35edf974c91e64560e9f7 INFO: 172.20.0.1:58328 - "POST /node/69b35edf974c91e64560e9f7 HTTP/1.0" 202 Accepted 2026-03-13 01:02:09,764 - lava_callback - INFO - Starting async job submit for node 69b35edf974c91e64560e9f7 2026-03-13 01:02:09,814 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:02:09,838 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35edf974c91e64560e9f7 HTTP/1.1" 200 None 2026-03-13 01:02:09,839 - lava_callback - DEBUG - Processing node 69b35edf974c91e64560e9f7: ltp-syscalls-ipc 2026-03-13 01:02:10,557 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:02:10,907 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:02:10,909 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-syscalls-ipc-69b35edf974c91e64560e9f7/log.txt.gz 2026-03-13 01:02:11,322 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:02:11,844 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:02:11,846 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-syscalls-ipc-69b35edf974c91e64560e9f7/lava-logs.yaml 2026-03-13 01:02:11,859 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:02:12,288 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:02:12,290 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-syscalls-ipc-69b35edf974c91e64560e9f7/lava_callback.json.gz 2026-03-13 01:02:12,342 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:02:12,368 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35edf974c91e64560e9f7 HTTP/1.1" 200 None 2026-03-13 01:02:12,371 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:02:12,394 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35934974c91e64560e49d HTTP/1.1" 200 None 2026-03-13 01:02:12,396 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:02:12,449 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b35edf974c91e64560e9f7 HTTP/1.1" 200 None 2026-03-13 01:02:12,451 - lava_callback - INFO - Completed processing callback for node 69b35edf974c91e64560e9f7 2026-03-13 01:02:14,973 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:02:15,007 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:58330 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58346 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60054 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41688 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41440 - "GET / HTTP/1.0" 200 OK 2026-03-13 01:02:46,784 - lava_callback - INFO - Received callback for node 69b0bf229d191758182529b9 INFO: 172.20.0.1:41450 - "POST /node/69b0bf229d191758182529b9 HTTP/1.0" 202 Accepted 2026-03-13 01:02:46,793 - lava_callback - INFO - Starting async job submit for node 69b0bf229d191758182529b9 2026-03-13 01:02:46,795 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:02:46,827 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0bf229d191758182529b9 HTTP/1.1" 200 None 2026-03-13 01:02:46,828 - lava_callback - DEBUG - Processing node 69b0bf229d191758182529b9: baseline-arm-baylibre 2026-03-13 01:02:46,828 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0bf229d191758182529b9 2026-03-13 01:02:46,829 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:02:46,854 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0bf229d191758182529b9 HTTP/1.1" 200 None 2026-03-13 01:02:46,865 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:02:46,897 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e34efa461f55e59cdb7 HTTP/1.1" 200 None 2026-03-13 01:02:46,899 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:02:46,936 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0bf229d191758182529b9 HTTP/1.1" 200 None 2026-03-13 01:02:46,937 - lava_callback - ERROR - Error processing callback for node 69b0bf229d191758182529b9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:59462 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53830 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53842 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48604 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48618 - "GET / HTTP/1.0" 200 OK 2026-03-13 01:03:17,845 - lava_callback - INFO - Received callback for node 69b35e90974c91e64560e90a INFO: 172.20.0.1:48622 - "POST /node/69b35e90974c91e64560e90a HTTP/1.0" 202 Accepted 2026-03-13 01:03:17,855 - lava_callback - INFO - Starting async job submit for node 69b35e90974c91e64560e90a 2026-03-13 01:03:17,963 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:03:17,995 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35e90974c91e64560e90a HTTP/1.1" 200 None 2026-03-13 01:03:17,996 - lava_callback - DEBUG - Processing node 69b35e90974c91e64560e90a: baseline-arm 2026-03-13 01:03:19,010 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:03:20,604 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:03:20,607 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69b35e90974c91e64560e90a/log.txt.gz 2026-03-13 01:03:21,250 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:03:24,789 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:03:24,792 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69b35e90974c91e64560e90a/lava-logs.yaml 2026-03-13 01:03:24,813 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:48554 - "GET / HTTP/1.0" 200 OK 2026-03-13 01:03:27,247 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:03:27,249 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69b35e90974c91e64560e90a/lava_callback.json.gz DEBUG: Setup failed for 69b35e90974c91e64560e90a. Transitting job node result: incomplete -> fail 2026-03-13 01:03:27,456 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:03:27,482 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35e90974c91e64560e90a HTTP/1.1" 200 None 2026-03-13 01:03:27,484 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:03:27,505 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35934974c91e64560e49d HTTP/1.1" 200 None 2026-03-13 01:03:27,507 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:03:27,551 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b35e90974c91e64560e90a HTTP/1.1" 200 None 2026-03-13 01:03:27,731 - lava_callback - INFO - Completed processing callback for node 69b35e90974c91e64560e90a INFO: 172.20.0.1:57228 - "GET / HTTP/1.0" 200 OK 2026-03-13 01:03:45,013 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:03:45,053 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:42984 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52016 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40778 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40794 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38542 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38544 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46002 - "GET / HTTP/1.0" 200 OK 2026-03-13 01:04:29,313 - lava_callback - INFO - Received callback for node 69b36283974c91e64560ec59 INFO: 172.20.0.1:46004 - "POST /node/69b36283974c91e64560ec59 HTTP/1.0" 202 Accepted 2026-03-13 01:04:29,322 - lava_callback - INFO - Starting async job submit for node 69b36283974c91e64560ec59 2026-03-13 01:04:29,332 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:04:29,365 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b36283974c91e64560ec59 HTTP/1.1" 200 None 2026-03-13 01:04:29,366 - lava_callback - DEBUG - Processing node 69b36283974c91e64560ec59: baseline-arm 2026-03-13 01:04:29,375 - lava_callback - WARNING - Failed to upload log 2026-03-13 01:04:29,381 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:04:30,608 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:04:30,610 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69b36283974c91e64560ec59/lava-logs.yaml 2026-03-13 01:04:30,612 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:04:31,769 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:04:31,772 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69b36283974c91e64560ec59/lava_callback.json.gz 2026-03-13 01:04:31,776 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:04:31,801 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b36283974c91e64560ec59 HTTP/1.1" 200 None 2026-03-13 01:04:31,803 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:04:31,843 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35934974c91e64560e49d HTTP/1.1" 200 None 2026-03-13 01:04:31,846 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:04:31,883 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b36283974c91e64560ec59 HTTP/1.1" 200 None 2026-03-13 01:04:31,889 - lava_callback - INFO - Completed processing callback for node 69b36283974c91e64560ec59 INFO: 172.20.0.1:40940 - "GET / HTTP/1.0" 200 OK 2026-03-13 01:04:45,056 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:04:45,088 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:46008 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59336 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34572 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34584 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42976 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42980 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52368 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43980 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43840 - "GET / HTTP/1.0" 200 OK 2026-03-13 01:05:47,038 - lava_callback - INFO - Received callback for node 69b0bfe59d19175818252a0b INFO: 172.20.0.1:43842 - "POST /node/69b0bfe59d19175818252a0b HTTP/1.0" 202 Accepted 2026-03-13 01:05:47,046 - lava_callback - INFO - Starting async job submit for node 69b0bfe59d19175818252a0b 2026-03-13 01:05:47,048 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:05:47,071 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0bfe59d19175818252a0b HTTP/1.1" 200 None 2026-03-13 01:05:47,072 - lava_callback - DEBUG - Processing node 69b0bfe59d19175818252a0b: baseline-arm-baylibre 2026-03-13 01:05:47,072 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0bfe59d19175818252a0b 2026-03-13 01:05:47,073 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:05:47,096 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0bfe59d19175818252a0b HTTP/1.1" 200 None 2026-03-13 01:05:47,099 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:05:47,136 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e34efa461f55e59cdb7 HTTP/1.1" 200 None 2026-03-13 01:05:47,138 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:05:47,180 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0bfe59d19175818252a0b HTTP/1.1" 200 None 2026-03-13 01:05:47,182 - lava_callback - ERROR - Error processing callback for node 69b0bfe59d19175818252a0b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:60598 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36586 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36594 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57504 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57520 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51842 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38966 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51154 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54840 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58054 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58070 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42698 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42704 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51342 - "GET / HTTP/1.0" 200 OK 2026-03-13 01:07:29,502 - lava_callback - INFO - Received callback for node 69b35eb5974c91e64560e996 INFO: 172.20.0.1:51358 - "POST /node/69b35eb5974c91e64560e996 HTTP/1.0" 202 Accepted 2026-03-13 01:07:29,553 - lava_callback - INFO - Starting async job submit for node 69b35eb5974c91e64560e996 2026-03-13 01:07:29,594 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:07:29,620 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35eb5974c91e64560e996 HTTP/1.1" 200 None 2026-03-13 01:07:29,622 - lava_callback - DEBUG - Processing node 69b35eb5974c91e64560e996: baseline-arm-pengutronix 2026-03-13 01:07:29,827 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:07:31,194 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:07:31,196 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69b35eb5974c91e64560e996/log.txt.gz 2026-03-13 01:07:31,310 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:07:33,225 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:07:33,228 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69b35eb5974c91e64560e996/lava-logs.yaml 2026-03-13 01:07:33,231 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:07:34,681 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:07:34,683 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69b35eb5974c91e64560e996/lava_callback.json.gz DEBUG: Setup failed for 69b35eb5974c91e64560e996. Transitting job node result: incomplete -> fail 2026-03-13 01:07:34,779 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:07:34,808 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35eb5974c91e64560e996 HTTP/1.1" 200 None 2026-03-13 01:07:34,810 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:07:34,834 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35934974c91e64560e49d HTTP/1.1" 200 None 2026-03-13 01:07:34,836 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:07:34,896 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b35eb5974c91e64560e996 HTTP/1.1" 200 None 2026-03-13 01:07:35,011 - lava_callback - INFO - Completed processing callback for node 69b35eb5974c91e64560e996 INFO: 172.20.0.1:48050 - "GET / HTTP/1.0" 200 OK 2026-03-13 01:07:45,091 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:07:45,118 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:56938 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41180 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45122 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45134 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55268 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55272 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33348 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50192 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47968 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51982 - "GET / HTTP/1.0" 200 OK 2026-03-13 01:09:05,139 - lava_callback - INFO - Received callback for node 69b0c0a99d19175818252a45 INFO: 172.20.0.1:43672 - "POST /node/69b0c0a99d19175818252a45 HTTP/1.0" 202 Accepted 2026-03-13 01:09:05,145 - lava_callback - INFO - Starting async job submit for node 69b0c0a99d19175818252a45 2026-03-13 01:09:05,146 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:09:05,161 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0c0a99d19175818252a45 HTTP/1.1" 200 None 2026-03-13 01:09:05,162 - lava_callback - DEBUG - Processing node 69b0c0a99d19175818252a45: baseline-arm-clabbe 2026-03-13 01:09:05,163 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0c0a99d19175818252a45 2026-03-13 01:09:05,163 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:09:05,177 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0c0a99d19175818252a45 HTTP/1.1" 200 None 2026-03-13 01:09:05,178 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:09:05,200 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e34efa461f55e59cdb7 HTTP/1.1" 200 None 2026-03-13 01:09:05,202 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:09:05,225 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0c0a99d19175818252a45 HTTP/1.1" 200 None 2026-03-13 01:09:05,232 - lava_callback - ERROR - Error processing callback for node 69b0c0a99d19175818252a45: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:43684 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43690 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56902 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56916 - "GET / HTTP/1.0" 200 OK 2026-03-13 01:09:17,185 - lava_callback - INFO - Received callback for node 69b36399974c91e64560ecc3 INFO: 172.20.0.1:56930 - "POST /node/69b36399974c91e64560ecc3 HTTP/1.0" 202 Accepted 2026-03-13 01:09:17,191 - lava_callback - INFO - Starting async job submit for node 69b36399974c91e64560ecc3 2026-03-13 01:09:17,210 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:09:17,224 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b36399974c91e64560ecc3 HTTP/1.1" 200 None 2026-03-13 01:09:17,225 - lava_callback - DEBUG - Processing node 69b36399974c91e64560ecc3: baseline-arm64 2026-03-13 01:09:17,378 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:09:17,604 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:09:17,607 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69b36399974c91e64560ecc3/log.txt.gz 2026-03-13 01:09:17,701 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:09:17,878 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:09:17,881 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69b36399974c91e64560ecc3/lava-logs.yaml 2026-03-13 01:09:17,887 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:09:18,025 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:09:18,027 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69b36399974c91e64560ecc3/lava_callback.json.gz 2026-03-13 01:09:18,046 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:09:18,061 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b36399974c91e64560ecc3 HTTP/1.1" 200 None 2026-03-13 01:09:18,063 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:09:18,076 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3594d974c91e64560e4b2 HTTP/1.1" 200 None 2026-03-13 01:09:18,078 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:09:18,108 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b36399974c91e64560ecc3 HTTP/1.1" 200 None 2026-03-13 01:09:18,110 - lava_callback - INFO - Completed processing callback for node 69b36399974c91e64560ecc3 INFO: 172.20.0.1:59406 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48396 - "GET / HTTP/1.0" 200 OK 2026-03-13 01:09:40,489 - lava_callback - INFO - Received callback for node 69b363b6974c91e64560ed18 INFO: 172.20.0.1:43148 - "POST /node/69b363b6974c91e64560ed18 HTTP/1.0" 202 Accepted 2026-03-13 01:09:40,495 - lava_callback - INFO - Starting async job submit for node 69b363b6974c91e64560ed18 2026-03-13 01:09:40,497 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:09:40,555 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b363b6974c91e64560ed18 HTTP/1.1" 200 None 2026-03-13 01:09:40,557 - lava_callback - DEBUG - Processing node 69b363b6974c91e64560ed18: baseline-nfs-arm64 2026-03-13 01:09:40,560 - lava_callback - WARNING - Failed to upload log 2026-03-13 01:09:40,562 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:09:40,768 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:09:40,770 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b363b6974c91e64560ed18/lava-logs.yaml 2026-03-13 01:09:40,773 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:09:40,943 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:09:40,945 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b363b6974c91e64560ed18/lava_callback.json.gz 2026-03-13 01:09:40,947 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:09:40,961 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b363b6974c91e64560ed18 HTTP/1.1" 200 None 2026-03-13 01:09:40,962 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:09:40,975 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3594d974c91e64560e4b2 HTTP/1.1" 200 None 2026-03-13 01:09:40,977 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:09:41,000 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b363b6974c91e64560ed18 HTTP/1.1" 200 None 2026-03-13 01:09:41,005 - lava_callback - INFO - Completed processing callback for node 69b363b6974c91e64560ed18 2026-03-13 01:09:41,918 - lava_callback - INFO - Received callback for node 69b363a5974c91e64560ecef INFO: 172.20.0.1:43164 - "POST /node/69b363a5974c91e64560ecef HTTP/1.0" 202 Accepted 2026-03-13 01:09:41,928 - lava_callback - INFO - Starting async job submit for node 69b363a5974c91e64560ecef 2026-03-13 01:09:41,950 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:09:41,970 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b363a5974c91e64560ecef HTTP/1.1" 200 None 2026-03-13 01:09:41,971 - lava_callback - DEBUG - Processing node 69b363a5974c91e64560ecef: baseline-arm64 2026-03-13 01:09:42,125 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:09:42,241 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:09:42,243 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69b363a5974c91e64560ecef/log.txt.gz 2026-03-13 01:09:42,333 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:09:42,503 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:09:42,504 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69b363a5974c91e64560ecef/lava-logs.yaml 2026-03-13 01:09:42,509 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:09:42,656 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:09:42,658 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69b363a5974c91e64560ecef/lava_callback.json.gz 2026-03-13 01:09:42,677 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:09:42,695 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b363a5974c91e64560ecef HTTP/1.1" 200 None 2026-03-13 01:09:42,697 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:09:42,715 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3594d974c91e64560e4b2 HTTP/1.1" 200 None 2026-03-13 01:09:42,717 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:09:42,754 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b363a5974c91e64560ecef HTTP/1.1" 200 None 2026-03-13 01:09:42,756 - lava_callback - INFO - Completed processing callback for node 69b363a5974c91e64560ecef 2026-03-13 01:09:45,120 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:09:45,141 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-13 01:09:46,491 - lava_callback - INFO - Received callback for node 69b3639d974c91e64560eccf INFO: 172.20.0.1:43170 - "POST /node/69b3639d974c91e64560eccf HTTP/1.0" 202 Accepted 2026-03-13 01:09:46,517 - lava_callback - INFO - Starting async job submit for node 69b3639d974c91e64560eccf INFO: 172.20.0.1:43178 - "GET / HTTP/1.0" 200 OK 2026-03-13 01:09:46,552 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:09:46,568 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3639d974c91e64560eccf HTTP/1.1" 200 None 2026-03-13 01:09:46,569 - lava_callback - DEBUG - Processing node 69b3639d974c91e64560eccf: baseline-arm64 2026-03-13 01:09:46,806 - lava_callback - WARNING - Failed to upload log 2026-03-13 01:09:46,904 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:09:47,100 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:09:47,101 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69b3639d974c91e64560eccf/lava-logs.yaml 2026-03-13 01:09:47,105 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:09:47,222 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:09:47,224 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69b3639d974c91e64560eccf/lava_callback.json.gz 2026-03-13 01:09:47,255 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:09:47,269 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3639d974c91e64560eccf HTTP/1.1" 200 None 2026-03-13 01:09:47,271 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:09:47,284 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3594d974c91e64560e4b2 HTTP/1.1" 200 None 2026-03-13 01:09:47,286 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:09:47,306 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b3639d974c91e64560eccf HTTP/1.1" 200 None 2026-03-13 01:09:47,364 - lava_callback - INFO - Completed processing callback for node 69b3639d974c91e64560eccf 2026-03-13 01:09:47,796 - lava_callback - INFO - Received callback for node 69b3639e974c91e64560ecdd INFO: 172.20.0.1:43184 - "POST /node/69b3639e974c91e64560ecdd HTTP/1.0" 202 Accepted 2026-03-13 01:09:47,803 - lava_callback - INFO - Starting async job submit for node 69b3639e974c91e64560ecdd 2026-03-13 01:09:47,834 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:09:47,853 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3639e974c91e64560ecdd HTTP/1.1" 200 None 2026-03-13 01:09:47,853 - lava_callback - DEBUG - Processing node 69b3639e974c91e64560ecdd: baseline-arm64 2026-03-13 01:09:48,003 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:09:48,116 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:09:48,118 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69b3639e974c91e64560ecdd/log.txt.gz 2026-03-13 01:09:48,191 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:09:48,457 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:09:48,459 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69b3639e974c91e64560ecdd/lava-logs.yaml 2026-03-13 01:09:48,464 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:09:48,627 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:09:48,629 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69b3639e974c91e64560ecdd/lava_callback.json.gz 2026-03-13 01:09:48,670 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:09:48,689 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3639e974c91e64560ecdd HTTP/1.1" 200 None 2026-03-13 01:09:48,690 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:09:48,705 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3594d974c91e64560e4b2 HTTP/1.1" 200 None 2026-03-13 01:09:48,706 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:09:48,730 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b3639e974c91e64560ecdd HTTP/1.1" 200 None 2026-03-13 01:09:48,793 - lava_callback - INFO - Completed processing callback for node 69b3639e974c91e64560ecdd INFO: 172.20.0.1:33278 - "GET / HTTP/1.0" 200 OK 2026-03-13 01:09:57,095 - lava_callback - INFO - Received callback for node 69b363a7974c91e64560ecf2 INFO: 172.20.0.1:33286 - "POST /node/69b363a7974c91e64560ecf2 HTTP/1.0" 202 Accepted 2026-03-13 01:09:57,103 - lava_callback - INFO - Starting async job submit for node 69b363a7974c91e64560ecf2 2026-03-13 01:09:57,134 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:09:57,149 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b363a7974c91e64560ecf2 HTTP/1.1" 200 None 2026-03-13 01:09:57,150 - lava_callback - DEBUG - Processing node 69b363a7974c91e64560ecf2: baseline-arm64 2026-03-13 01:09:57,424 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:09:57,541 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:09:57,543 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69b363a7974c91e64560ecf2/log.txt.gz 2026-03-13 01:09:57,693 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:09:57,878 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:09:57,880 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69b363a7974c91e64560ecf2/lava-logs.yaml 2026-03-13 01:09:57,888 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:09:58,052 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:09:58,054 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69b363a7974c91e64560ecf2/lava_callback.json.gz 2026-03-13 01:09:58,077 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:09:58,094 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b363a7974c91e64560ecf2 HTTP/1.1" 200 None 2026-03-13 01:09:58,096 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:09:58,113 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3594d974c91e64560e4b2 HTTP/1.1" 200 None 2026-03-13 01:09:58,115 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:09:58,163 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b363a7974c91e64560ecf2 HTTP/1.1" 200 None 2026-03-13 01:09:58,166 - lava_callback - INFO - Completed processing callback for node 69b363a7974c91e64560ecf2 2026-03-13 01:10:05,807 - lava_callback - INFO - Received callback for node 69b3639a974c91e64560ecc9 INFO: 172.20.0.1:34308 - "POST /node/69b3639a974c91e64560ecc9 HTTP/1.0" 202 Accepted 2026-03-13 01:10:05,814 - lava_callback - INFO - Starting async job submit for node 69b3639a974c91e64560ecc9 2026-03-13 01:10:05,857 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:10:05,874 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3639a974c91e64560ecc9 HTTP/1.1" 200 None 2026-03-13 01:10:05,875 - lava_callback - DEBUG - Processing node 69b3639a974c91e64560ecc9: baseline-arm64 2026-03-13 01:10:06,167 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:10:06,289 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:10:06,291 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69b3639a974c91e64560ecc9/log.txt.gz 2026-03-13 01:10:06,489 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:34316 - "GET / HTTP/1.0" 200 OK 2026-03-13 01:10:06,743 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:10:06,745 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69b3639a974c91e64560ecc9/lava-logs.yaml 2026-03-13 01:10:06,752 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:10:06,900 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:10:06,902 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69b3639a974c91e64560ecc9/lava_callback.json.gz 2026-03-13 01:10:06,934 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:10:06,947 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3639a974c91e64560ecc9 HTTP/1.1" 200 None 2026-03-13 01:10:06,949 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:10:06,963 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3594d974c91e64560e4b2 HTTP/1.1" 200 None 2026-03-13 01:10:06,965 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:10:06,992 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b3639a974c91e64560ecc9 HTTP/1.1" 200 None 2026-03-13 01:10:06,997 - lava_callback - INFO - Completed processing callback for node 69b3639a974c91e64560ecc9 INFO: 172.20.0.1:34320 - "GET / HTTP/1.0" 200 OK 2026-03-13 01:10:11,606 - lava_callback - INFO - Received callback for node 69b363a9974c91e64560ecf8 INFO: 172.20.0.1:47834 - "POST /node/69b363a9974c91e64560ecf8 HTTP/1.0" 202 Accepted 2026-03-13 01:10:11,613 - lava_callback - INFO - Starting async job submit for node 69b363a9974c91e64560ecf8 2026-03-13 01:10:11,634 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:10:11,649 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b363a9974c91e64560ecf8 HTTP/1.1" 200 None 2026-03-13 01:10:11,650 - lava_callback - DEBUG - Processing node 69b363a9974c91e64560ecf8: baseline-arm64 2026-03-13 01:10:11,923 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:10:12,044 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:10:12,046 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69b363a9974c91e64560ecf8/log.txt.gz 2026-03-13 01:10:12,210 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:10:12,399 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:10:12,401 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69b363a9974c91e64560ecf8/lava-logs.yaml 2026-03-13 01:10:12,408 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:10:12,558 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:10:12,560 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69b363a9974c91e64560ecf8/lava_callback.json.gz 2026-03-13 01:10:12,583 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:10:12,600 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b363a9974c91e64560ecf8 HTTP/1.1" 200 None 2026-03-13 01:10:12,602 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:10:12,626 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3594d974c91e64560e4b2 HTTP/1.1" 200 None 2026-03-13 01:10:12,628 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:10:12,659 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b363a9974c91e64560ecf8 HTTP/1.1" 200 None 2026-03-13 01:10:12,661 - lava_callback - INFO - Completed processing callback for node 69b363a9974c91e64560ecf8 2026-03-13 01:10:15,143 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:10:15,160 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 INFO: 172.20.0.1:47848 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47860 - "GET / HTTP/1.0" 200 OK 2026-03-13 01:10:17,253 - lava_callback - INFO - Received callback for node 69b3639b974c91e64560eccc INFO: 172.20.0.1:47870 - "POST /node/69b3639b974c91e64560eccc HTTP/1.0" 202 Accepted 2026-03-13 01:10:17,259 - lava_callback - INFO - Starting async job submit for node 69b3639b974c91e64560eccc 2026-03-13 01:10:17,282 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:10:17,297 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3639b974c91e64560eccc HTTP/1.1" 200 None 2026-03-13 01:10:17,298 - lava_callback - DEBUG - Processing node 69b3639b974c91e64560eccc: baseline-arm64 2026-03-13 01:10:17,566 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:10:17,693 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:10:17,695 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69b3639b974c91e64560eccc/log.txt.gz 2026-03-13 01:10:17,844 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:10:18,034 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:10:18,036 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69b3639b974c91e64560eccc/lava-logs.yaml 2026-03-13 01:10:18,042 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:10:18,168 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:10:18,169 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69b3639b974c91e64560eccc/lava_callback.json.gz 2026-03-13 01:10:18,200 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:10:18,215 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3639b974c91e64560eccc HTTP/1.1" 200 None 2026-03-13 01:10:18,216 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:10:18,230 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3594d974c91e64560e4b2 HTTP/1.1" 200 None 2026-03-13 01:10:18,232 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:10:18,257 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b3639b974c91e64560eccc HTTP/1.1" 200 None 2026-03-13 01:10:18,259 - lava_callback - INFO - Completed processing callback for node 69b3639b974c91e64560eccc 2026-03-13 01:10:19,561 - lava_callback - INFO - Received callback for node 69b3639a974c91e64560ecc6 INFO: 172.20.0.1:47872 - "POST /node/69b3639a974c91e64560ecc6 HTTP/1.0" 202 Accepted 2026-03-13 01:10:19,567 - lava_callback - INFO - Starting async job submit for node 69b3639a974c91e64560ecc6 2026-03-13 01:10:19,589 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:10:19,610 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3639a974c91e64560ecc6 HTTP/1.1" 200 None 2026-03-13 01:10:19,611 - lava_callback - DEBUG - Processing node 69b3639a974c91e64560ecc6: baseline-arm64 2026-03-13 01:10:19,834 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:10:19,973 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:10:19,975 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69b3639a974c91e64560ecc6/log.txt.gz 2026-03-13 01:10:20,113 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:10:20,278 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:10:20,280 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69b3639a974c91e64560ecc6/lava-logs.yaml 2026-03-13 01:10:20,286 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:10:20,408 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:10:20,410 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69b3639a974c91e64560ecc6/lava_callback.json.gz 2026-03-13 01:10:20,432 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:10:20,446 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3639a974c91e64560ecc6 HTTP/1.1" 200 None 2026-03-13 01:10:20,448 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:10:20,462 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3594d974c91e64560e4b2 HTTP/1.1" 200 None 2026-03-13 01:10:20,464 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:10:20,491 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b3639a974c91e64560ecc6 HTTP/1.1" 200 None 2026-03-13 01:10:20,499 - lava_callback - INFO - Completed processing callback for node 69b3639a974c91e64560ecc6 2026-03-13 01:10:22,443 - lava_callback - INFO - Received callback for node 69b363aa974c91e64560ecfb INFO: 172.20.0.1:55134 - "POST /node/69b363aa974c91e64560ecfb HTTP/1.0" 202 Accepted 2026-03-13 01:10:22,449 - lava_callback - INFO - Starting async job submit for node 69b363aa974c91e64560ecfb 2026-03-13 01:10:22,470 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:10:22,485 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b363aa974c91e64560ecfb HTTP/1.1" 200 None 2026-03-13 01:10:22,486 - lava_callback - DEBUG - Processing node 69b363aa974c91e64560ecfb: baseline-arm64 2026-03-13 01:10:22,763 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:10:22,891 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:10:22,893 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69b363aa974c91e64560ecfb/log.txt.gz 2026-03-13 01:10:23,047 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:10:23,220 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:10:23,222 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69b363aa974c91e64560ecfb/lava-logs.yaml 2026-03-13 01:10:23,229 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:10:23,411 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:10:23,412 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69b363aa974c91e64560ecfb/lava_callback.json.gz 2026-03-13 01:10:23,435 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:10:23,449 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b363aa974c91e64560ecfb HTTP/1.1" 200 None 2026-03-13 01:10:23,451 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:10:23,466 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3594d974c91e64560e4b2 HTTP/1.1" 200 None 2026-03-13 01:10:23,467 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:10:23,501 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b363aa974c91e64560ecfb HTTP/1.1" 200 None 2026-03-13 01:10:23,503 - lava_callback - INFO - Completed processing callback for node 69b363aa974c91e64560ecfb 2026-03-13 01:10:25,064 - lava_callback - INFO - Received callback for node 69b0c0f19d19175818252a5c INFO: 172.20.0.1:55146 - "POST /node/69b0c0f19d19175818252a5c HTTP/1.0" 202 Accepted 2026-03-13 01:10:25,070 - lava_callback - INFO - Starting async job submit for node 69b0c0f19d19175818252a5c 2026-03-13 01:10:25,071 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:10:25,085 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0c0f19d19175818252a5c HTTP/1.1" 200 None 2026-03-13 01:10:25,086 - lava_callback - DEBUG - Processing node 69b0c0f19d19175818252a5c: baseline-arm-clabbe 2026-03-13 01:10:25,086 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0c0f19d19175818252a5c 2026-03-13 01:10:25,087 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:10:25,102 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0c0f19d19175818252a5c HTTP/1.1" 200 None 2026-03-13 01:10:25,104 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:10:25,119 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e34efa461f55e59cdb7 HTTP/1.1" 200 None 2026-03-13 01:10:25,121 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:10:25,141 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0c0f19d19175818252a5c HTTP/1.1" 200 None 2026-03-13 01:10:25,149 - lava_callback - ERROR - Error processing callback for node 69b0c0f19d19175818252a5c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-13 01:10:26,490 - lava_callback - INFO - Received callback for node 69b363a0974c91e64560ece3 INFO: 172.20.0.1:55148 - "POST /node/69b363a0974c91e64560ece3 HTTP/1.0" 202 Accepted 2026-03-13 01:10:26,504 - lava_callback - INFO - Starting async job submit for node 69b363a0974c91e64560ece3 INFO: 172.20.0.1:55160 - "GET / HTTP/1.0" 200 OK 2026-03-13 01:10:26,544 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:10:26,595 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b363a0974c91e64560ece3 HTTP/1.1" 200 None 2026-03-13 01:10:26,596 - lava_callback - DEBUG - Processing node 69b363a0974c91e64560ece3: baseline-arm64 2026-03-13 01:10:27,061 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:10:27,366 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:10:27,368 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69b363a0974c91e64560ece3/log.txt.gz 2026-03-13 01:10:27,628 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:10:28,176 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:10:28,179 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69b363a0974c91e64560ece3/lava-logs.yaml 2026-03-13 01:10:28,192 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:10:28,599 - lava_callback - INFO - Received callback for node 69b363b1974c91e64560ed0c 2026-03-13 01:10:28,606 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:10:28,611 - lava_callback - INFO - Starting async job submit for node 69b363b1974c91e64560ed0c INFO: 172.20.0.1:55174 - "POST /node/69b363b1974c91e64560ed0c HTTP/1.0" 202 Accepted 2026-03-13 01:10:28,635 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69b363a0974c91e64560ece3/lava_callback.json.gz 2026-03-13 01:10:28,696 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:10:28,719 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:10:28,729 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b363b1974c91e64560ed0c HTTP/1.1" 200 None 2026-03-13 01:10:28,730 - lava_callback - DEBUG - Processing node 69b363b1974c91e64560ed0c: baseline-nfs-arm64 2026-03-13 01:10:28,773 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b363a0974c91e64560ece3 HTTP/1.1" 200 None 2026-03-13 01:10:28,776 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:10:28,842 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3594d974c91e64560e4b2 HTTP/1.1" 200 None 2026-03-13 01:10:28,845 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:10:28,925 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b363a0974c91e64560ece3 HTTP/1.1" 200 None 2026-03-13 01:10:28,926 - lava_callback - INFO - Completed processing callback for node 69b363a0974c91e64560ece3 2026-03-13 01:10:29,690 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:10:30,090 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:10:30,093 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b363b1974c91e64560ed0c/log.txt.gz 2026-03-13 01:10:30,452 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:10:30,960 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:10:30,963 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b363b1974c91e64560ed0c/lava-logs.yaml 2026-03-13 01:10:30,977 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:10:31,267 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:10:31,269 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b363b1974c91e64560ed0c/lava_callback.json.gz 2026-03-13 01:10:31,319 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:10:31,355 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b363b1974c91e64560ed0c HTTP/1.1" 200 None 2026-03-13 01:10:31,358 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:10:31,384 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3594d974c91e64560e4b2 HTTP/1.1" 200 None 2026-03-13 01:10:31,387 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:10:31,440 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b363b1974c91e64560ed0c HTTP/1.1" 200 None 2026-03-13 01:10:31,443 - lava_callback - INFO - Completed processing callback for node 69b363b1974c91e64560ed0c INFO: 172.20.0.1:57798 - "GET / HTTP/1.0" 200 OK 2026-03-13 01:10:42,796 - lava_callback - INFO - Received callback for node 69b363ac974c91e64560ecfe INFO: 172.20.0.1:44218 - "POST /node/69b363ac974c91e64560ecfe HTTP/1.0" 202 Accepted 2026-03-13 01:10:42,809 - lava_callback - INFO - Starting async job submit for node 69b363ac974c91e64560ecfe 2026-03-13 01:10:42,867 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:10:42,894 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b363ac974c91e64560ecfe HTTP/1.1" 200 None 2026-03-13 01:10:42,895 - lava_callback - DEBUG - Processing node 69b363ac974c91e64560ecfe: baseline-arm64 2026-03-13 01:10:43,439 - lava_callback - INFO - Received callback for node 69b363bc974c91e64560ed44 INFO: 172.20.0.1:44226 - "POST /node/69b363bc974c91e64560ed44 HTTP/1.0" 202 Accepted 2026-03-13 01:10:43,467 - lava_callback - INFO - Starting async job submit for node 69b363bc974c91e64560ed44 2026-03-13 01:10:43,522 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:10:43,539 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:10:43,575 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b363bc974c91e64560ed44 HTTP/1.1" 200 None 2026-03-13 01:10:43,577 - lava_callback - DEBUG - Processing node 69b363bc974c91e64560ed44: baseline-nfs-arm64 2026-03-13 01:10:43,966 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:10:43,968 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69b363ac974c91e64560ecfe/log.txt.gz 2026-03-13 01:10:44,732 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:10:44,766 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:10:45,079 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:10:45,081 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b363bc974c91e64560ed44/log.txt.gz 2026-03-13 01:10:45,165 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:10:45,249 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 2026-03-13 01:10:45,335 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:10:45,338 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69b363ac974c91e64560ecfe/lava-logs.yaml 2026-03-13 01:10:45,348 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:10:45,585 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:10:45,814 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:10:45,816 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69b363ac974c91e64560ecfe/lava_callback.json.gz 2026-03-13 01:10:45,894 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:10:45,918 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b363ac974c91e64560ecfe HTTP/1.1" 200 None 2026-03-13 01:10:45,921 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:10:45,942 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3594d974c91e64560e4b2 HTTP/1.1" 200 None 2026-03-13 01:10:45,944 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:10:45,990 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b363ac974c91e64560ecfe HTTP/1.1" 200 None 2026-03-13 01:10:45,992 - lava_callback - INFO - Completed processing callback for node 69b363ac974c91e64560ecfe 2026-03-13 01:10:46,241 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:10:46,245 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b363bc974c91e64560ed44/lava-logs.yaml 2026-03-13 01:10:46,269 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:44238 - "GET / HTTP/1.0" 200 OK 2026-03-13 01:10:46,814 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:10:46,816 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b363bc974c91e64560ed44/lava_callback.json.gz 2026-03-13 01:10:46,862 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:10:46,889 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b363bc974c91e64560ed44 HTTP/1.1" 200 None 2026-03-13 01:10:46,891 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:10:46,921 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3594d974c91e64560e4b2 HTTP/1.1" 200 None 2026-03-13 01:10:46,923 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:10:46,968 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b363bc974c91e64560ed44 HTTP/1.1" 200 None 2026-03-13 01:10:46,970 - lava_callback - INFO - Completed processing callback for node 69b363bc974c91e64560ed44 INFO: 172.20.0.1:58466 - "GET / HTTP/1.0" 200 OK 2026-03-13 01:11:05,086 - lava_callback - INFO - Received callback for node 69b0c11a9d19175818252a68 INFO: 172.20.0.1:40600 - "POST /node/69b0c11a9d19175818252a68 HTTP/1.0" 202 Accepted 2026-03-13 01:11:05,096 - lava_callback - INFO - Starting async job submit for node 69b0c11a9d19175818252a68 2026-03-13 01:11:05,098 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:11:05,123 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0c11a9d19175818252a68 HTTP/1.1" 200 None 2026-03-13 01:11:05,124 - lava_callback - DEBUG - Processing node 69b0c11a9d19175818252a68: baseline-arm-clabbe 2026-03-13 01:11:05,124 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0c11a9d19175818252a68 2026-03-13 01:11:05,125 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:11:05,150 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0c11a9d19175818252a68 HTTP/1.1" 200 None 2026-03-13 01:11:05,152 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:11:05,174 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e34efa461f55e59cdb7 HTTP/1.1" 200 None 2026-03-13 01:11:05,177 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:11:05,215 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0c11a9d19175818252a68 HTTP/1.1" 200 None 2026-03-13 01:11:05,216 - lava_callback - ERROR - Error processing callback for node 69b0c11a9d19175818252a68: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:40608 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40622 - "GET / HTTP/1.0" 200 OK 2026-03-13 01:11:12,186 - lava_callback - INFO - Received callback for node 69b363dd974c91e64560edb9 INFO: 172.20.0.1:40792 - "POST /node/69b363dd974c91e64560edb9 HTTP/1.0" 202 Accepted 2026-03-13 01:11:12,196 - lava_callback - INFO - Starting async job submit for node 69b363dd974c91e64560edb9 2026-03-13 01:11:12,229 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:11:12,256 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b363dd974c91e64560edb9 HTTP/1.1" 200 None 2026-03-13 01:11:12,257 - lava_callback - DEBUG - Processing node 69b363dd974c91e64560edb9: baseline-arm64-pengutronix 2026-03-13 01:11:12,577 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:11:12,960 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:11:12,962 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69b363dd974c91e64560edb9/log.txt.gz 2026-03-13 01:11:13,148 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:11:13,769 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:11:13,771 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69b363dd974c91e64560edb9/lava-logs.yaml 2026-03-13 01:11:13,783 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:11:14,354 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:11:14,356 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69b363dd974c91e64560edb9/lava_callback.json.gz 2026-03-13 01:11:14,392 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:11:14,433 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b363dd974c91e64560edb9 HTTP/1.1" 200 None 2026-03-13 01:11:14,436 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:11:14,464 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3594d974c91e64560e4b2 HTTP/1.1" 200 None 2026-03-13 01:11:14,467 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:11:14,521 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b363dd974c91e64560edb9 HTTP/1.1" 200 None 2026-03-13 01:11:14,523 - lava_callback - INFO - Completed processing callback for node 69b363dd974c91e64560edb9 INFO: 172.20.0.1:40802 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-13 01:11:15,252 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:11:15,279 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:40812 - "GET / HTTP/1.0" 200 OK 2026-03-13 01:11:19,406 - lava_callback - INFO - Received callback for node 69b363df974c91e64560edbc INFO: 172.20.0.1:40818 - "POST /node/69b363df974c91e64560edbc HTTP/1.0" 202 Accepted 2026-03-13 01:11:19,415 - lava_callback - INFO - Starting async job submit for node 69b363df974c91e64560edbc 2026-03-13 01:11:19,453 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:11:19,477 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b363df974c91e64560edbc HTTP/1.1" 200 None 2026-03-13 01:11:19,478 - lava_callback - DEBUG - Processing node 69b363df974c91e64560edbc: baseline-arm64-pengutronix 2026-03-13 01:11:19,879 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:11:20,194 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:11:20,196 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69b363df974c91e64560edbc/log.txt.gz 2026-03-13 01:11:20,399 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:11:20,871 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:11:20,874 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69b363df974c91e64560edbc/lava-logs.yaml 2026-03-13 01:11:20,881 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:11:21,277 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:11:21,279 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69b363df974c91e64560edbc/lava_callback.json.gz 2026-03-13 01:11:21,315 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:11:21,339 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b363df974c91e64560edbc HTTP/1.1" 200 None 2026-03-13 01:11:21,342 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:11:21,372 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3594d974c91e64560e4b2 HTTP/1.1" 200 None 2026-03-13 01:11:21,375 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:11:21,423 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b363df974c91e64560edbc HTTP/1.1" 200 None 2026-03-13 01:11:21,425 - lava_callback - INFO - Completed processing callback for node 69b363df974c91e64560edbc INFO: 172.20.0.1:45372 - "GET / HTTP/1.0" 200 OK 2026-03-13 01:11:26,850 - lava_callback - INFO - Received callback for node 69b363ba974c91e64560ed22 INFO: 172.20.0.1:45376 - "POST /node/69b363ba974c91e64560ed22 HTTP/1.0" 202 Accepted 2026-03-13 01:11:26,860 - lava_callback - INFO - Starting async job submit for node 69b363ba974c91e64560ed22 2026-03-13 01:11:26,911 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:11:26,942 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b363ba974c91e64560ed22 HTTP/1.1" 200 None 2026-03-13 01:11:26,944 - lava_callback - DEBUG - Processing node 69b363ba974c91e64560ed22: baseline-nfs-arm64 2026-03-13 01:11:27,545 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:11:27,905 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:11:27,908 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b363ba974c91e64560ed22/log.txt.gz 2026-03-13 01:11:28,278 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:11:28,783 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:11:28,786 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b363ba974c91e64560ed22/lava-logs.yaml 2026-03-13 01:11:28,801 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:11:29,409 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:11:29,411 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b363ba974c91e64560ed22/lava_callback.json.gz 2026-03-13 01:11:29,452 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:11:29,485 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b363ba974c91e64560ed22 HTTP/1.1" 200 None 2026-03-13 01:11:29,488 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:11:29,509 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3594d974c91e64560e4b2 HTTP/1.1" 200 None 2026-03-13 01:11:29,511 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:11:29,564 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b363ba974c91e64560ed22 HTTP/1.1" 200 None 2026-03-13 01:11:29,566 - lava_callback - INFO - Completed processing callback for node 69b363ba974c91e64560ed22 2026-03-13 01:11:30,312 - lava_callback - INFO - Received callback for node 69b363e0974c91e64560edbf INFO: 172.20.0.1:53658 - "POST /node/69b363e0974c91e64560edbf HTTP/1.0" 202 Accepted 2026-03-13 01:11:30,322 - lava_callback - INFO - Starting async job submit for node 69b363e0974c91e64560edbf 2026-03-13 01:11:30,384 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:11:30,412 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b363e0974c91e64560edbf HTTP/1.1" 200 None 2026-03-13 01:11:30,413 - lava_callback - DEBUG - Processing node 69b363e0974c91e64560edbf: baseline-arm64-pengutronix 2026-03-13 01:11:30,784 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:11:31,141 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:11:31,144 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69b363e0974c91e64560edbf/log.txt.gz 2026-03-13 01:11:31,356 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:11:32,025 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:11:32,027 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69b363e0974c91e64560edbf/lava-logs.yaml 2026-03-13 01:11:32,035 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:11:32,414 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:11:32,416 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69b363e0974c91e64560edbf/lava_callback.json.gz 2026-03-13 01:11:32,484 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:11:32,510 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b363e0974c91e64560edbf HTTP/1.1" 200 None 2026-03-13 01:11:32,512 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:11:32,534 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3594d974c91e64560e4b2 HTTP/1.1" 200 None 2026-03-13 01:11:32,536 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:11:32,584 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b363e0974c91e64560edbf HTTP/1.1" 200 None 2026-03-13 01:11:32,586 - lava_callback - INFO - Completed processing callback for node 69b363e0974c91e64560edbf INFO: 172.20.0.1:53664 - "GET / HTTP/1.0" 200 OK 2026-03-13 01:11:39,435 - lava_callback - INFO - Received callback for node 69b363b7974c91e64560ed1b INFO: 172.20.0.1:53670 - "POST /node/69b363b7974c91e64560ed1b HTTP/1.0" 202 Accepted 2026-03-13 01:11:39,447 - lava_callback - INFO - Starting async job submit for node 69b363b7974c91e64560ed1b 2026-03-13 01:11:39,490 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:11:39,533 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b363b7974c91e64560ed1b HTTP/1.1" 200 None 2026-03-13 01:11:39,534 - lava_callback - DEBUG - Processing node 69b363b7974c91e64560ed1b: baseline-nfs-arm64 2026-03-13 01:11:40,105 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:11:40,366 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:11:40,369 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b363b7974c91e64560ed1b/log.txt.gz 2026-03-13 01:11:40,702 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:11:41,300 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:11:41,302 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b363b7974c91e64560ed1b/lava-logs.yaml 2026-03-13 01:11:41,311 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:11:41,680 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:11:41,682 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b363b7974c91e64560ed1b/lava_callback.json.gz 2026-03-13 01:11:41,723 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:11:41,751 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b363b7974c91e64560ed1b HTTP/1.1" 200 None 2026-03-13 01:11:41,753 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:11:41,775 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3594d974c91e64560e4b2 HTTP/1.1" 200 None 2026-03-13 01:11:41,777 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:11:41,833 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b363b7974c91e64560ed1b HTTP/1.1" 200 None 2026-03-13 01:11:41,834 - lava_callback - INFO - Completed processing callback for node 69b363b7974c91e64560ed1b 2026-03-13 01:11:44,884 - lava_callback - INFO - Received callback for node 69b363a3974c91e64560ece9 INFO: 172.20.0.1:41244 - "POST /node/69b363a3974c91e64560ece9 HTTP/1.0" 202 Accepted 2026-03-13 01:11:44,895 - lava_callback - INFO - Starting async job submit for node 69b363a3974c91e64560ece9 2026-03-13 01:11:44,941 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:11:44,974 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b363a3974c91e64560ece9 HTTP/1.1" 200 None 2026-03-13 01:11:44,979 - lava_callback - DEBUG - Processing node 69b363a3974c91e64560ece9: baseline-arm64 2026-03-13 01:11:45,287 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:11:45,327 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 2026-03-13 01:11:45,816 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:11:46,219 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:11:46,221 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69b363a3974c91e64560ece9/log.txt.gz INFO: 172.20.0.1:41246 - "GET / HTTP/1.0" 200 OK 2026-03-13 01:11:46,672 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:11:47,350 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:11:47,352 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69b363a3974c91e64560ece9/lava-logs.yaml 2026-03-13 01:11:47,364 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:11:47,549 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:11:47,551 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69b363a3974c91e64560ece9/lava_callback.json.gz 2026-03-13 01:11:47,597 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:11:47,612 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b363a3974c91e64560ece9 HTTP/1.1" 200 None 2026-03-13 01:11:47,614 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:11:47,629 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3594d974c91e64560e4b2 HTTP/1.1" 200 None 2026-03-13 01:11:47,631 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:11:47,658 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b363a3974c91e64560ece9 HTTP/1.1" 200 None 2026-03-13 01:11:47,720 - lava_callback - INFO - Completed processing callback for node 69b363a3974c91e64560ece9 2026-03-13 01:11:47,862 - lava_callback - INFO - Received callback for node 69b363e1974c91e64560edc2 INFO: 172.20.0.1:41248 - "POST /node/69b363e1974c91e64560edc2 HTTP/1.0" 202 Accepted 2026-03-13 01:11:47,873 - lava_callback - INFO - Starting async job submit for node 69b363e1974c91e64560edc2 2026-03-13 01:11:47,915 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:11:47,940 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b363e1974c91e64560edc2 HTTP/1.1" 200 None 2026-03-13 01:11:47,942 - lava_callback - DEBUG - Processing node 69b363e1974c91e64560edc2: baseline-arm64-foundriesio 2026-03-13 01:11:48,182 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:11:48,298 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:11:48,299 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69b363e1974c91e64560edc2/log.txt.gz 2026-03-13 01:11:48,419 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:11:48,593 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:11:48,595 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69b363e1974c91e64560edc2/lava-logs.yaml 2026-03-13 01:11:48,601 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:11:48,739 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:11:48,741 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69b363e1974c91e64560edc2/lava_callback.json.gz 2026-03-13 01:11:48,785 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:11:48,799 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b363e1974c91e64560edc2 HTTP/1.1" 200 None 2026-03-13 01:11:48,801 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:11:48,821 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3594d974c91e64560e4b2 HTTP/1.1" 200 None 2026-03-13 01:11:48,823 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:11:48,848 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b363e1974c91e64560edc2 HTTP/1.1" 200 None 2026-03-13 01:11:48,894 - lava_callback - INFO - Completed processing callback for node 69b363e1974c91e64560edc2 2026-03-13 01:11:49,460 - lava_callback - INFO - Received callback for node 69b363ae974c91e64560ed04 INFO: 172.20.0.1:41256 - "POST /node/69b363ae974c91e64560ed04 HTTP/1.0" 202 Accepted 2026-03-13 01:11:49,467 - lava_callback - INFO - Starting async job submit for node 69b363ae974c91e64560ed04 2026-03-13 01:11:49,491 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:11:49,505 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b363ae974c91e64560ed04 HTTP/1.1" 200 None 2026-03-13 01:11:49,506 - lava_callback - DEBUG - Processing node 69b363ae974c91e64560ed04: baseline-nfs-arm64 2026-03-13 01:11:49,845 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:11:49,956 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:11:49,958 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b363ae974c91e64560ed04/log.txt.gz 2026-03-13 01:11:50,143 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:11:50,357 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:11:50,359 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b363ae974c91e64560ed04/lava-logs.yaml 2026-03-13 01:11:50,367 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:11:50,496 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:11:50,498 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b363ae974c91e64560ed04/lava_callback.json.gz 2026-03-13 01:11:50,521 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:11:50,538 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b363ae974c91e64560ed04 HTTP/1.1" 200 None 2026-03-13 01:11:50,540 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:11:50,553 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3594d974c91e64560e4b2 HTTP/1.1" 200 None 2026-03-13 01:11:50,555 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:11:50,583 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b363ae974c91e64560ed04 HTTP/1.1" 200 None 2026-03-13 01:11:50,585 - lava_callback - INFO - Completed processing callback for node 69b363ae974c91e64560ed04 2026-03-13 01:11:55,830 - lava_callback - INFO - Received callback for node 69b363bb974c91e64560ed25 INFO: 172.20.0.1:58492 - "POST /node/69b363bb974c91e64560ed25 HTTP/1.0" 202 Accepted 2026-03-13 01:11:55,837 - lava_callback - INFO - Starting async job submit for node 69b363bb974c91e64560ed25 2026-03-13 01:11:55,859 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:11:55,873 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b363bb974c91e64560ed25 HTTP/1.1" 200 None 2026-03-13 01:11:55,874 - lava_callback - DEBUG - Processing node 69b363bb974c91e64560ed25: baseline-nfs-arm64 2026-03-13 01:11:56,200 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:11:56,374 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:11:56,376 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b363bb974c91e64560ed25/log.txt.gz INFO: 172.20.0.1:58496 - "GET / HTTP/1.0" 200 OK 2026-03-13 01:11:56,572 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:11:56,789 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:11:56,791 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b363bb974c91e64560ed25/lava-logs.yaml 2026-03-13 01:11:56,799 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:11:56,935 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:11:56,936 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b363bb974c91e64560ed25/lava_callback.json.gz 2026-03-13 01:11:56,960 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:11:56,974 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b363bb974c91e64560ed25 HTTP/1.1" 200 None 2026-03-13 01:11:56,976 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:11:56,990 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3594d974c91e64560e4b2 HTTP/1.1" 200 None 2026-03-13 01:11:56,992 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:11:57,019 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b363bb974c91e64560ed25 HTTP/1.1" 200 None 2026-03-13 01:11:57,021 - lava_callback - INFO - Completed processing callback for node 69b363bb974c91e64560ed25 INFO: 172.20.0.1:40242 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40246 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35552 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-13 01:12:15,335 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:12:15,354 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 INFO: 172.20.0.1:35558 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55992 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36586 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34812 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50276 - "GET / HTTP/1.0" 200 OK 2026-03-13 01:13:01,216 - lava_callback - INFO - Received callback for node 69b363e2974c91e64560edc5 INFO: 172.20.0.1:44062 - "POST /node/69b363e2974c91e64560edc5 HTTP/1.0" 202 Accepted 2026-03-13 01:13:01,224 - lava_callback - INFO - Starting async job submit for node 69b363e2974c91e64560edc5 2026-03-13 01:13:01,262 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:13:01,277 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b363e2974c91e64560edc5 HTTP/1.1" 200 None 2026-03-13 01:13:01,278 - lava_callback - DEBUG - Processing node 69b363e2974c91e64560edc5: baseline-arm64-foundriesio 2026-03-13 01:13:02,032 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:13:02,224 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:13:02,226 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69b363e2974c91e64560edc5/log.txt.gz 2026-03-13 01:13:02,700 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:13:02,931 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:13:02,933 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69b363e2974c91e64560edc5/lava-logs.yaml 2026-03-13 01:13:02,952 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:13:03,145 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:13:03,147 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69b363e2974c91e64560edc5/lava_callback.json.gz DEBUG: 69b363e2974c91e64560edc5 Transitting job node result: pass -> incomplete 2026-03-13 01:13:03,187 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:13:03,201 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b363e2974c91e64560edc5 HTTP/1.1" 200 None 2026-03-13 01:13:03,202 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:13:03,216 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3594d974c91e64560e4b2 HTTP/1.1" 200 None 2026-03-13 01:13:03,218 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:13:03,244 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b363e2974c91e64560edc5 HTTP/1.1" 200 None 2026-03-13 01:13:03,245 - lava_callback - INFO - Completed processing callback for node 69b363e2974c91e64560edc5 2026-03-13 01:13:04,074 - lava_callback - INFO - Received callback for node 69b363b3974c91e64560ed12 INFO: 172.20.0.1:44076 - "POST /node/69b363b3974c91e64560ed12 HTTP/1.0" 202 Accepted 2026-03-13 01:13:04,081 - lava_callback - INFO - Starting async job submit for node 69b363b3974c91e64560ed12 2026-03-13 01:13:04,104 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:13:04,119 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b363b3974c91e64560ed12 HTTP/1.1" 200 None 2026-03-13 01:13:04,120 - lava_callback - DEBUG - Processing node 69b363b3974c91e64560ed12: baseline-nfs-arm64 2026-03-13 01:13:04,611 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:13:04,773 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:13:04,775 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b363b3974c91e64560ed12/log.txt.gz 2026-03-13 01:13:05,083 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:13:05,265 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:13:05,267 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b363b3974c91e64560ed12/lava-logs.yaml 2026-03-13 01:13:05,278 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:13:05,424 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:13:05,426 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b363b3974c91e64560ed12/lava_callback.json.gz 2026-03-13 01:13:05,450 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:13:05,464 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b363b3974c91e64560ed12 HTTP/1.1" 200 None 2026-03-13 01:13:05,466 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:13:05,480 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3594d974c91e64560e4b2 HTTP/1.1" 200 None 2026-03-13 01:13:05,482 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:13:05,507 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b363b3974c91e64560ed12 HTTP/1.1" 200 None 2026-03-13 01:13:05,509 - lava_callback - INFO - Completed processing callback for node 69b363b3974c91e64560ed12 INFO: 172.20.0.1:44090 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44102 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40430 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-13 01:13:15,357 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:13:15,374 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:40444 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48064 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58234 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35336 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59134 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45610 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45616 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47860 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47868 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47138 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38896 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45028 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38958 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40260 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40276 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42518 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42522 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52784 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37584 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43974 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53904 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33952 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33962 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47166 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47178 - "GET / HTTP/1.0" 200 OK 2026-03-13 01:16:17,519 - lava_callback - INFO - Received callback for node 69b363a4974c91e64560ecec INFO: 172.20.0.1:47192 - "POST /node/69b363a4974c91e64560ecec HTTP/1.0" 202 Accepted 2026-03-13 01:16:17,525 - lava_callback - INFO - Starting async job submit for node 69b363a4974c91e64560ecec 2026-03-13 01:16:17,548 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:16:17,568 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b363a4974c91e64560ecec HTTP/1.1" 200 None 2026-03-13 01:16:17,569 - lava_callback - DEBUG - Processing node 69b363a4974c91e64560ecec: baseline-arm64 2026-03-13 01:16:18,036 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:16:18,203 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:16:18,205 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69b363a4974c91e64560ecec/log.txt.gz 2026-03-13 01:16:18,463 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:16:18,679 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:16:18,682 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69b363a4974c91e64560ecec/lava-logs.yaml 2026-03-13 01:16:18,694 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:16:18,913 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:16:18,916 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69b363a4974c91e64560ecec/lava_callback.json.gz 2026-03-13 01:16:18,939 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:16:18,953 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b363a4974c91e64560ecec HTTP/1.1" 200 None 2026-03-13 01:16:18,955 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:16:18,968 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3594d974c91e64560e4b2 HTTP/1.1" 200 None 2026-03-13 01:16:18,969 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:16:18,994 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b363a4974c91e64560ecec HTTP/1.1" 200 None 2026-03-13 01:16:18,995 - lava_callback - INFO - Completed processing callback for node 69b363a4974c91e64560ecec INFO: 172.20.0.1:38764 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46794 - "GET / HTTP/1.0" 200 OK 2026-03-13 01:16:45,380 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:16:45,397 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:35672 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44236 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53044 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53046 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50284 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50286 - "GET / HTTP/1.0" 200 OK 2026-03-13 01:17:19,734 - lava_callback - INFO - Received callback for node 69b363ad974c91e64560ed01 INFO: 172.20.0.1:52204 - "POST /node/69b363ad974c91e64560ed01 HTTP/1.0" 202 Accepted 2026-03-13 01:17:19,741 - lava_callback - INFO - Starting async job submit for node 69b363ad974c91e64560ed01 2026-03-13 01:17:19,776 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:17:19,791 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b363ad974c91e64560ed01 HTTP/1.1" 200 None 2026-03-13 01:17:19,792 - lava_callback - DEBUG - Processing node 69b363ad974c91e64560ed01: baseline-arm64 2026-03-13 01:17:20,189 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:17:20,341 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:17:20,343 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69b363ad974c91e64560ed01/log.txt.gz 2026-03-13 01:17:20,551 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:17:20,785 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:17:20,788 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69b363ad974c91e64560ed01/lava-logs.yaml 2026-03-13 01:17:20,799 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:17:20,979 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:17:20,981 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69b363ad974c91e64560ed01/lava_callback.json.gz DEBUG: Setup failed for 69b363ad974c91e64560ed01. Transitting job node result: incomplete -> fail 2026-03-13 01:17:21,054 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:17:21,070 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b363ad974c91e64560ed01 HTTP/1.1" 200 None 2026-03-13 01:17:21,072 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:17:21,089 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3594d974c91e64560e4b2 HTTP/1.1" 200 None 2026-03-13 01:17:21,091 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:17:21,115 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b363ad974c91e64560ed01 HTTP/1.1" 200 None 2026-03-13 01:17:21,211 - lava_callback - INFO - Completed processing callback for node 69b363ad974c91e64560ed01 INFO: 172.20.0.1:52212 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44408 - "GET / HTTP/1.0" 200 OK 2026-03-13 01:17:45,399 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:17:45,415 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-13 01:17:45,665 - lava_callback - INFO - Received callback for node 69b363b5974c91e64560ed15 INFO: 172.20.0.1:38198 - "POST /node/69b363b5974c91e64560ed15 HTTP/1.0" 202 Accepted 2026-03-13 01:17:45,672 - lava_callback - INFO - Starting async job submit for node 69b363b5974c91e64560ed15 2026-03-13 01:17:45,695 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:17:45,710 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b363b5974c91e64560ed15 HTTP/1.1" 200 None 2026-03-13 01:17:45,711 - lava_callback - DEBUG - Processing node 69b363b5974c91e64560ed15: baseline-nfs-arm64 2026-03-13 01:17:46,308 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:17:46,483 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:17:46,485 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b363b5974c91e64560ed15/log.txt.gz INFO: 172.20.0.1:38204 - "GET / HTTP/1.0" 200 OK 2026-03-13 01:17:46,836 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:17:47,117 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:17:47,120 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b363b5974c91e64560ed15/lava-logs.yaml 2026-03-13 01:17:47,132 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:17:47,291 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:17:47,293 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b363b5974c91e64560ed15/lava_callback.json.gz 2026-03-13 01:17:47,316 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:17:47,337 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b363b5974c91e64560ed15 HTTP/1.1" 200 None 2026-03-13 01:17:47,338 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:17:47,354 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3594d974c91e64560e4b2 HTTP/1.1" 200 None 2026-03-13 01:17:47,355 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:17:47,396 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b363b5974c91e64560ed15 HTTP/1.1" 200 None 2026-03-13 01:17:47,398 - lava_callback - INFO - Completed processing callback for node 69b363b5974c91e64560ed15 INFO: 172.20.0.1:37040 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36436 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36446 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36284 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-13 01:18:15,417 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:18:15,459 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:36292 - "GET / HTTP/1.0" 200 OK 2026-03-13 01:18:21,754 - lava_callback - INFO - Received callback for node 69b363d9974c91e64560ed8d INFO: 172.20.0.1:56258 - "POST /node/69b363d9974c91e64560ed8d HTTP/1.0" 202 Accepted 2026-03-13 01:18:21,764 - lava_callback - INFO - Starting async job submit for node 69b363d9974c91e64560ed8d 2026-03-13 01:18:21,796 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:18:21,810 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b363d9974c91e64560ed8d HTTP/1.1" 200 None 2026-03-13 01:18:21,811 - lava_callback - DEBUG - Processing node 69b363d9974c91e64560ed8d: baseline-arm64-clabbe 2026-03-13 01:18:22,201 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:18:22,339 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:18:22,341 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-clabbe-69b363d9974c91e64560ed8d/log.txt.gz 2026-03-13 01:18:22,599 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:18:22,840 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:18:22,842 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-clabbe-69b363d9974c91e64560ed8d/lava-logs.yaml 2026-03-13 01:18:22,851 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:18:23,018 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:18:23,020 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-clabbe-69b363d9974c91e64560ed8d/lava_callback.json.gz 2026-03-13 01:18:23,051 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:18:23,067 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b363d9974c91e64560ed8d HTTP/1.1" 200 None 2026-03-13 01:18:23,069 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:18:23,083 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3594d974c91e64560e4b2 HTTP/1.1" 200 None 2026-03-13 01:18:23,085 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:18:23,111 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b363d9974c91e64560ed8d HTTP/1.1" 200 None 2026-03-13 01:18:23,113 - lava_callback - INFO - Completed processing callback for node 69b363d9974c91e64560ed8d INFO: 172.20.0.1:56264 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37572 - "GET / HTTP/1.0" 200 OK 2026-03-13 01:18:45,461 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:18:45,478 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:41744 - "GET / HTTP/1.0" 200 OK 2026-03-13 01:18:55,159 - lava_callback - INFO - Received callback for node 69b363a8974c91e64560ecf5 INFO: 172.20.0.1:47900 - "POST /node/69b363a8974c91e64560ecf5 HTTP/1.0" 202 Accepted 2026-03-13 01:18:55,169 - lava_callback - INFO - Starting async job submit for node 69b363a8974c91e64560ecf5 2026-03-13 01:18:55,189 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:18:55,208 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b363a8974c91e64560ecf5 HTTP/1.1" 200 None 2026-03-13 01:18:55,209 - lava_callback - DEBUG - Processing node 69b363a8974c91e64560ecf5: baseline-arm64 INFO: 172.20.0.1:47904 - "GET / HTTP/1.0" 200 OK 2026-03-13 01:18:56,544 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:18:56,703 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:18:56,705 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69b363a8974c91e64560ecf5/log.txt.gz 2026-03-13 01:18:57,564 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:18:57,811 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:18:57,813 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69b363a8974c91e64560ecf5/lava-logs.yaml 2026-03-13 01:18:57,837 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:18:58,019 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:18:58,021 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69b363a8974c91e64560ecf5/lava_callback.json.gz 2026-03-13 01:18:58,043 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:18:58,058 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b363a8974c91e64560ecf5 HTTP/1.1" 200 None 2026-03-13 01:18:58,060 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:18:58,074 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3594d974c91e64560e4b2 HTTP/1.1" 200 None 2026-03-13 01:18:58,076 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:18:58,101 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b363a8974c91e64560ecf5 HTTP/1.1" 200 None 2026-03-13 01:18:58,102 - lava_callback - INFO - Completed processing callback for node 69b363a8974c91e64560ecf5 INFO: 172.20.0.1:40030 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40036 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36798 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-13 01:19:15,480 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:19:15,495 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:36812 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44238 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60388 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48070 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56890 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51346 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51362 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46282 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46286 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52034 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50620 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52424 - "GET / HTTP/1.0" 200 OK 2026-03-13 01:20:50,056 - lava_callback - INFO - Received callback for node 69b363b8974c91e64560ed1e INFO: 172.20.0.1:41100 - "POST /node/69b363b8974c91e64560ed1e HTTP/1.0" 202 Accepted 2026-03-13 01:20:50,063 - lava_callback - INFO - Starting async job submit for node 69b363b8974c91e64560ed1e 2026-03-13 01:20:50,084 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:20:50,101 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b363b8974c91e64560ed1e HTTP/1.1" 200 None 2026-03-13 01:20:50,102 - lava_callback - DEBUG - Processing node 69b363b8974c91e64560ed1e: baseline-nfs-arm64 2026-03-13 01:20:50,531 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:20:50,710 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:20:50,712 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b363b8974c91e64560ed1e/log.txt.gz 2026-03-13 01:20:50,969 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:20:51,258 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:20:51,260 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b363b8974c91e64560ed1e/lava-logs.yaml 2026-03-13 01:20:51,271 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:20:51,435 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:20:51,437 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b363b8974c91e64560ed1e/lava_callback.json.gz 2026-03-13 01:20:51,459 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:20:51,474 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b363b8974c91e64560ed1e HTTP/1.1" 200 None 2026-03-13 01:20:51,475 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:20:51,489 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3594d974c91e64560e4b2 HTTP/1.1" 200 None 2026-03-13 01:20:51,491 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:20:51,520 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b363b8974c91e64560ed1e HTTP/1.1" 200 None 2026-03-13 01:20:51,522 - lava_callback - INFO - Completed processing callback for node 69b363b8974c91e64560ed1e INFO: 172.20.0.1:41108 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42836 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42852 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41170 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-13 01:21:15,497 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:21:15,515 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:41174 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40300 - "GET / HTTP/1.0" 200 OK 2026-03-13 01:21:27,896 - lava_callback - INFO - Received callback for node 69b0c3909d19175818252b11 INFO: 172.20.0.1:40302 - "POST /node/69b0c3909d19175818252b11 HTTP/1.0" 202 Accepted 2026-03-13 01:21:27,901 - lava_callback - INFO - Starting async job submit for node 69b0c3909d19175818252b11 2026-03-13 01:21:27,902 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:21:27,916 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0c3909d19175818252b11 HTTP/1.1" 200 None 2026-03-13 01:21:27,917 - lava_callback - DEBUG - Processing node 69b0c3909d19175818252b11: baseline-arm64-baylibre 2026-03-13 01:21:27,918 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0c3909d19175818252b11 2026-03-13 01:21:27,918 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:21:27,943 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0c3909d19175818252b11 HTTP/1.1" 200 None 2026-03-13 01:21:27,944 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:21:27,960 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a8cbcf911a9cbe7f77ccbd HTTP/1.1" 200 None 2026-03-13 01:21:27,962 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:21:27,983 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0c3909d19175818252b11 HTTP/1.1" 200 None 2026-03-13 01:21:27,993 - lava_callback - ERROR - Error processing callback for node 69b0c3909d19175818252b11: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:60558 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57110 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46232 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36938 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36944 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56488 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56500 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50048 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34646 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59406 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41168 - "GET / HTTP/1.0" 200 OK 2026-03-13 01:23:05,755 - lava_callback - INFO - Received callback for node 69b0c3e49d19175818252b1b INFO: 172.20.0.1:48006 - "POST /node/69b0c3e49d19175818252b1b HTTP/1.0" 202 Accepted 2026-03-13 01:23:05,761 - lava_callback - INFO - Starting async job submit for node 69b0c3e49d19175818252b1b 2026-03-13 01:23:05,762 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:23:05,787 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0c3e49d19175818252b1b HTTP/1.1" 200 None 2026-03-13 01:23:05,788 - lava_callback - DEBUG - Processing node 69b0c3e49d19175818252b1b: baseline-arm64-clabbe 2026-03-13 01:23:05,789 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0c3e49d19175818252b1b 2026-03-13 01:23:05,789 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:23:05,802 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0c3e49d19175818252b1b HTTP/1.1" 200 None 2026-03-13 01:23:05,804 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:23:05,821 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a8cbcf911a9cbe7f77ccbd HTTP/1.1" 200 None 2026-03-13 01:23:05,822 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:23:05,843 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0c3e49d19175818252b1b HTTP/1.1" 200 None 2026-03-13 01:23:05,844 - lava_callback - ERROR - Error processing callback for node 69b0c3e49d19175818252b1b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:48014 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48024 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36774 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36780 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49944 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35098 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56032 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32872 - "GET / HTTP/1.0" 200 OK 2026-03-13 01:24:05,809 - lava_callback - INFO - Received callback for node 69b0c4289d19175818252b20 INFO: 172.20.0.1:38540 - "POST /node/69b0c4289d19175818252b20 HTTP/1.0" 202 Accepted 2026-03-13 01:24:05,815 - lava_callback - INFO - Starting async job submit for node 69b0c4289d19175818252b20 2026-03-13 01:24:05,816 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:24:05,841 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0c4289d19175818252b20 HTTP/1.1" 200 None 2026-03-13 01:24:05,842 - lava_callback - DEBUG - Processing node 69b0c4289d19175818252b20: baseline-arm-clabbe 2026-03-13 01:24:05,842 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0c4289d19175818252b20 2026-03-13 01:24:05,843 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:24:05,857 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0c4289d19175818252b20 HTTP/1.1" 200 None 2026-03-13 01:24:05,859 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:24:05,881 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a8cbbb911a9cbe7f77ccab HTTP/1.1" 200 None 2026-03-13 01:24:05,882 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:24:05,903 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0c4289d19175818252b20 HTTP/1.1" 200 None 2026-03-13 01:24:05,904 - lava_callback - ERROR - Error processing callback for node 69b0c4289d19175818252b20: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:38548 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38554 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59164 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59178 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35852 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57892 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52610 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45396 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55772 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55786 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54722 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54734 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55004 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46768 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55236 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50732 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48048 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48052 - "GET / HTTP/1.0" 200 OK 2026-03-13 01:26:10,508 - lava_callback - INFO - Received callback for node 69b363a2974c91e64560ece6 INFO: 172.20.0.1:41526 - "POST /node/69b363a2974c91e64560ece6 HTTP/1.0" 202 Accepted 2026-03-13 01:26:10,515 - lava_callback - INFO - Starting async job submit for node 69b363a2974c91e64560ece6 2026-03-13 01:26:10,535 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:26:10,550 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b363a2974c91e64560ece6 HTTP/1.1" 200 None 2026-03-13 01:26:10,550 - lava_callback - DEBUG - Processing node 69b363a2974c91e64560ece6: baseline-arm64 2026-03-13 01:26:10,966 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:26:11,156 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:26:11,158 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69b363a2974c91e64560ece6/log.txt.gz 2026-03-13 01:26:11,363 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:26:11,589 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:26:11,592 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69b363a2974c91e64560ece6/lava-logs.yaml 2026-03-13 01:26:11,600 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:26:11,916 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:26:11,918 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69b363a2974c91e64560ece6/lava_callback.json.gz 2026-03-13 01:26:11,940 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:26:11,954 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b363a2974c91e64560ece6 HTTP/1.1" 200 None 2026-03-13 01:26:11,956 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:26:11,969 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3594d974c91e64560e4b2 HTTP/1.1" 200 None 2026-03-13 01:26:11,971 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:26:11,994 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b363a2974c91e64560ece6 HTTP/1.1" 200 None 2026-03-13 01:26:11,996 - lava_callback - INFO - Completed processing callback for node 69b363a2974c91e64560ece6 INFO: 172.20.0.1:41542 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-13 01:26:15,518 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:26:15,534 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:41554 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35188 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52466 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38340 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42394 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51128 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51140 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37936 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37952 - "GET / HTTP/1.0" 200 OK 2026-03-13 01:27:25,961 - lava_callback - INFO - Received callback for node 69b0c4ed9d19175818252b47 INFO: 172.20.0.1:37138 - "POST /node/69b0c4ed9d19175818252b47 HTTP/1.0" 202 Accepted 2026-03-13 01:27:25,966 - lava_callback - INFO - Starting async job submit for node 69b0c4ed9d19175818252b47 2026-03-13 01:27:25,967 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:27:25,982 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0c4ed9d19175818252b47 HTTP/1.1" 200 None 2026-03-13 01:27:25,982 - lava_callback - DEBUG - Processing node 69b0c4ed9d19175818252b47: baseline-arm-clabbe 2026-03-13 01:27:25,983 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0c4ed9d19175818252b47 2026-03-13 01:27:25,983 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:27:26,001 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0c4ed9d19175818252b47 HTTP/1.1" 200 None 2026-03-13 01:27:26,003 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:27:26,016 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a8cbbb911a9cbe7f77ccab HTTP/1.1" 200 None 2026-03-13 01:27:26,017 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:27:26,037 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0c4ed9d19175818252b47 HTTP/1.1" 200 None 2026-03-13 01:27:26,043 - lava_callback - ERROR - Error processing callback for node 69b0c4ed9d19175818252b47: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:37154 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50480 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37564 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44288 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57684 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57696 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45966 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45980 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57712 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37302 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39546 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56236 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44912 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44914 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45452 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45462 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58498 - "GET / HTTP/1.0" 200 OK 2026-03-13 01:29:28,248 - lava_callback - INFO - Received callback for node 69b0c5669d19175818252b71 INFO: 172.20.0.1:58504 - "POST /node/69b0c5669d19175818252b71 HTTP/1.0" 202 Accepted 2026-03-13 01:29:28,254 - lava_callback - INFO - Starting async job submit for node 69b0c5669d19175818252b71 2026-03-13 01:29:28,255 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:29:28,271 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0c5669d19175818252b71 HTTP/1.1" 200 None 2026-03-13 01:29:28,272 - lava_callback - DEBUG - Processing node 69b0c5669d19175818252b71: baseline-arm64-baylibre 2026-03-13 01:29:28,272 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0c5669d19175818252b71 2026-03-13 01:29:28,273 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:29:28,295 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0c5669d19175818252b71 HTTP/1.1" 200 None 2026-03-13 01:29:28,296 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:29:28,310 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11758ef4424e487b2eb1 HTTP/1.1" 200 None 2026-03-13 01:29:28,313 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:29:28,334 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0c5669d19175818252b71 HTTP/1.1" 200 None 2026-03-13 01:29:28,340 - lava_callback - ERROR - Error processing callback for node 69b0c5669d19175818252b71: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:46468 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52154 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42100 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33390 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33394 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48704 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48718 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43132 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46946 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47980 - "GET / HTTP/1.0" 200 OK 2026-03-13 01:30:48,369 - lava_callback - INFO - Received callback for node 69b0c5b69d19175818252b7f INFO: 172.20.0.1:47996 - "POST /node/69b0c5b69d19175818252b7f HTTP/1.0" 202 Accepted 2026-03-13 01:30:48,377 - lava_callback - INFO - Starting async job submit for node 69b0c5b69d19175818252b7f 2026-03-13 01:30:48,378 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:30:48,401 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0c5b69d19175818252b7f HTTP/1.1" 200 None 2026-03-13 01:30:48,401 - lava_callback - DEBUG - Processing node 69b0c5b69d19175818252b7f: baseline-arm64-baylibre 2026-03-13 01:30:48,402 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0c5b69d19175818252b7f 2026-03-13 01:30:48,402 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:30:48,418 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0c5b69d19175818252b7f HTTP/1.1" 200 None 2026-03-13 01:30:48,420 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:30:48,435 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11758ef4424e487b2eb1 HTTP/1.1" 200 None 2026-03-13 01:30:48,437 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:30:48,457 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0c5b69d19175818252b7f HTTP/1.1" 200 None 2026-03-13 01:30:48,458 - lava_callback - ERROR - Error processing callback for node 69b0c5b69d19175818252b7f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:60494 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38904 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38910 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52782 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52790 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46498 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52194 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58606 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42414 - "GET / HTTP/1.0" 200 OK 2026-03-13 01:32:03,855 - lava_callback - INFO - Received callback for node 69b368f9974c91e64560f129 INFO: 172.20.0.1:54928 - "POST /node/69b368f9974c91e64560f129 HTTP/1.0" 202 Accepted 2026-03-13 01:32:03,861 - lava_callback - INFO - Starting async job submit for node 69b368f9974c91e64560f129 2026-03-13 01:32:03,863 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:32:03,877 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b368f9974c91e64560f129 HTTP/1.1" 200 None 2026-03-13 01:32:03,877 - lava_callback - DEBUG - Processing node 69b368f9974c91e64560f129: baseline-nfs-arm64 2026-03-13 01:32:03,879 - lava_callback - WARNING - Failed to upload log 2026-03-13 01:32:03,880 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:32:04,030 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:32:04,032 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b368f9974c91e64560f129/lava-logs.yaml 2026-03-13 01:32:04,033 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:32:04,150 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:32:04,152 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b368f9974c91e64560f129/lava_callback.json.gz 2026-03-13 01:32:04,155 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:32:04,180 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b368f9974c91e64560f129 HTTP/1.1" 200 None 2026-03-13 01:32:04,182 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:32:04,196 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3594d974c91e64560e4b2 HTTP/1.1" 200 None 2026-03-13 01:32:04,198 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:32:04,218 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b368f9974c91e64560f129 HTTP/1.1" 200 None 2026-03-13 01:32:04,225 - lava_callback - INFO - Completed processing callback for node 69b368f9974c91e64560f129 INFO: 172.20.0.1:54930 - "GET / HTTP/1.0" 200 OK 2026-03-13 01:32:08,444 - lava_callback - INFO - Received callback for node 69b0c6099d19175818252b85 INFO: 172.20.0.1:54940 - "POST /node/69b0c6099d19175818252b85 HTTP/1.0" 202 Accepted 2026-03-13 01:32:08,450 - lava_callback - INFO - Starting async job submit for node 69b0c6099d19175818252b85 2026-03-13 01:32:08,451 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:32:08,473 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0c6099d19175818252b85 HTTP/1.1" 200 None 2026-03-13 01:32:08,474 - lava_callback - DEBUG - Processing node 69b0c6099d19175818252b85: baseline-arm64-baylibre 2026-03-13 01:32:08,474 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0c6099d19175818252b85 2026-03-13 01:32:08,475 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:32:08,488 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0c6099d19175818252b85 HTTP/1.1" 200 None 2026-03-13 01:32:08,489 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:32:08,503 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11758ef4424e487b2eb1 HTTP/1.1" 200 None 2026-03-13 01:32:08,504 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:32:08,531 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0c6099d19175818252b85 HTTP/1.1" 200 None 2026-03-13 01:32:08,534 - lava_callback - ERROR - Error processing callback for node 69b0c6099d19175818252b85: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:54950 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38408 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-13 01:32:15,536 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:32:15,553 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:38422 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59022 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35152 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35062 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60510 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40200 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40208 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48588 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48590 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54596 - "GET / HTTP/1.0" 200 OK 2026-03-13 01:33:28,457 - lava_callback - INFO - Received callback for node 69b0c6599d19175818252b8e INFO: 172.20.0.1:54604 - "POST /node/69b0c6599d19175818252b8e HTTP/1.0" 202 Accepted 2026-03-13 01:33:28,463 - lava_callback - INFO - Starting async job submit for node 69b0c6599d19175818252b8e 2026-03-13 01:33:28,464 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:33:28,477 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0c6599d19175818252b8e HTTP/1.1" 200 None 2026-03-13 01:33:28,478 - lava_callback - DEBUG - Processing node 69b0c6599d19175818252b8e: baseline-arm64-baylibre 2026-03-13 01:33:28,478 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0c6599d19175818252b8e 2026-03-13 01:33:28,479 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:33:28,492 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0c6599d19175818252b8e HTTP/1.1" 200 None 2026-03-13 01:33:28,493 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:33:28,534 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11758ef4424e487b2eb1 HTTP/1.1" 200 None 2026-03-13 01:33:28,535 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:33:28,555 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0c6599d19175818252b8e HTTP/1.1" 200 None 2026-03-13 01:33:28,557 - lava_callback - ERROR - Error processing callback for node 69b0c6599d19175818252b8e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:45388 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48544 - "GET / HTTP/1.0" 200 OK 2026-03-13 01:33:51,824 - lava_callback - INFO - Received callback for node 69b36946974c91e64560f166 INFO: 172.20.0.1:53324 - "POST /node/69b36946974c91e64560f166 HTTP/1.0" 202 Accepted 2026-03-13 01:33:51,831 - lava_callback - INFO - Starting async job submit for node 69b36946974c91e64560f166 2026-03-13 01:33:51,860 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:33:51,876 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b36946974c91e64560f166 HTTP/1.1" 200 None 2026-03-13 01:33:51,877 - lava_callback - DEBUG - Processing node 69b36946974c91e64560f166: baseline-arm64 2026-03-13 01:33:52,011 - lava_callback - WARNING - Failed to upload log 2026-03-13 01:33:52,080 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:33:52,287 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:33:52,289 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69b36946974c91e64560f166/lava-logs.yaml 2026-03-13 01:33:52,294 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:33:52,432 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:33:52,434 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69b36946974c91e64560f166/lava_callback.json.gz 2026-03-13 01:33:52,464 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:33:52,490 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b36946974c91e64560f166 HTTP/1.1" 200 None 2026-03-13 01:33:52,491 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:33:52,505 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3594d974c91e64560e4b2 HTTP/1.1" 200 None 2026-03-13 01:33:52,507 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:33:52,527 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b36946974c91e64560f166 HTTP/1.1" 200 None 2026-03-13 01:33:52,617 - lava_callback - INFO - Completed processing callback for node 69b36946974c91e64560f166 INFO: 172.20.0.1:53330 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60856 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60866 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35618 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-13 01:34:15,555 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:34:15,582 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:35630 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54876 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52636 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36268 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51006 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49670 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49672 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54542 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54556 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34396 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46642 - "GET / HTTP/1.0" 200 OK 2026-03-13 01:35:38,241 - lava_callback - INFO - Received callback for node 69b369a0974c91e64560f19c INFO: 172.20.0.1:46654 - "POST /node/69b369a0974c91e64560f19c HTTP/1.0" 202 Accepted 2026-03-13 01:35:38,248 - lava_callback - INFO - Starting async job submit for node 69b369a0974c91e64560f19c 2026-03-13 01:35:38,277 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:35:38,292 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b369a0974c91e64560f19c HTTP/1.1" 200 None 2026-03-13 01:35:38,293 - lava_callback - DEBUG - Processing node 69b369a0974c91e64560f19c: baseline-arm64 2026-03-13 01:35:38,442 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:35:38,605 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:35:38,607 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69b369a0974c91e64560f19c/log.txt.gz 2026-03-13 01:35:38,680 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:35:38,882 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:35:38,883 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69b369a0974c91e64560f19c/lava-logs.yaml 2026-03-13 01:35:38,887 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:35:39,005 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:35:39,007 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69b369a0974c91e64560f19c/lava_callback.json.gz 2026-03-13 01:35:39,037 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:35:39,054 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b369a0974c91e64560f19c HTTP/1.1" 200 None 2026-03-13 01:35:39,055 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:35:39,069 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3594d974c91e64560e4b2 HTTP/1.1" 200 None 2026-03-13 01:35:39,071 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:35:39,093 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b369a0974c91e64560f19c HTTP/1.1" 200 None 2026-03-13 01:35:39,162 - lava_callback - INFO - Completed processing callback for node 69b369a0974c91e64560f19c 2026-03-13 01:35:45,584 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:35:45,601 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:44610 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49520 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47776 - "GET / HTTP/1.0" 200 OK 2026-03-13 01:36:08,929 - lava_callback - INFO - Received callback for node 69b0c6f69d19175818252ba6 INFO: 172.20.0.1:47790 - "POST /node/69b0c6f69d19175818252ba6 HTTP/1.0" 202 Accepted 2026-03-13 01:36:08,934 - lava_callback - INFO - Starting async job submit for node 69b0c6f69d19175818252ba6 2026-03-13 01:36:08,935 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:36:08,949 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0c6f69d19175818252ba6 HTTP/1.1" 200 None 2026-03-13 01:36:08,950 - lava_callback - DEBUG - Processing node 69b0c6f69d19175818252ba6: baseline-arm64-baylibre 2026-03-13 01:36:08,950 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0c6f69d19175818252ba6 2026-03-13 01:36:08,951 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:36:08,964 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0c6f69d19175818252ba6 HTTP/1.1" 200 None 2026-03-13 01:36:08,965 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.20.0.1:47798 - "GET / HTTP/1.0" 200 OK 2026-03-13 01:36:08,979 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11758ef4424e487b2eb1 HTTP/1.1" 200 None 2026-03-13 01:36:08,980 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:36:09,001 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0c6f69d19175818252ba6 HTTP/1.1" 200 None 2026-03-13 01:36:09,002 - lava_callback - ERROR - Error processing callback for node 69b0c6f69d19175818252ba6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:56788 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56792 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53210 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39736 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46408 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54316 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52482 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52498 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60266 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60276 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49150 - "GET / HTTP/1.0" 200 OK 2026-03-13 01:37:28,722 - lava_callback - INFO - Received callback for node 69b0c7459d19175818252bb0 INFO: 172.20.0.1:49156 - "POST /node/69b0c7459d19175818252bb0 HTTP/1.0" 202 Accepted 2026-03-13 01:37:28,728 - lava_callback - INFO - Starting async job submit for node 69b0c7459d19175818252bb0 2026-03-13 01:37:28,729 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:37:28,763 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0c7459d19175818252bb0 HTTP/1.1" 200 None 2026-03-13 01:37:28,763 - lava_callback - DEBUG - Processing node 69b0c7459d19175818252bb0: baseline-arm64-baylibre 2026-03-13 01:37:28,764 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0c7459d19175818252bb0 2026-03-13 01:37:28,764 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:37:28,782 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0c7459d19175818252bb0 HTTP/1.1" 200 None 2026-03-13 01:37:28,783 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:37:28,796 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11758ef4424e487b2eb1 HTTP/1.1" 200 None 2026-03-13 01:37:28,797 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:37:28,822 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0c7459d19175818252bb0 HTTP/1.1" 200 None 2026-03-13 01:37:28,824 - lava_callback - ERROR - Error processing callback for node 69b0c7459d19175818252bb0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:40244 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49186 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58112 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44998 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45014 - "GET / HTTP/1.0" 200 OK 2026-03-13 01:38:09,152 - lava_callback - INFO - Received callback for node 69b36a53974c91e64560f1f1 INFO: 172.20.0.1:45018 - "POST /node/69b36a53974c91e64560f1f1 HTTP/1.0" 202 Accepted 2026-03-13 01:38:09,158 - lava_callback - INFO - Starting async job submit for node 69b36a53974c91e64560f1f1 2026-03-13 01:38:09,179 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:38:09,194 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b36a53974c91e64560f1f1 HTTP/1.1" 200 None 2026-03-13 01:38:09,195 - lava_callback - DEBUG - Processing node 69b36a53974c91e64560f1f1: baseline-arm64 2026-03-13 01:38:09,558 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:38:09,725 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:38:09,727 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69b36a53974c91e64560f1f1/log.txt.gz 2026-03-13 01:38:09,953 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:38:10,184 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:38:10,186 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69b36a53974c91e64560f1f1/lava-logs.yaml 2026-03-13 01:38:10,194 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:38:10,378 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:38:10,380 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69b36a53974c91e64560f1f1/lava_callback.json.gz 2026-03-13 01:38:10,402 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:38:10,416 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b36a53974c91e64560f1f1 HTTP/1.1" 200 None 2026-03-13 01:38:10,418 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:38:10,446 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3594d974c91e64560e4b2 HTTP/1.1" 200 None 2026-03-13 01:38:10,447 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:38:10,485 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b36a53974c91e64560f1f1 HTTP/1.1" 200 None 2026-03-13 01:38:10,487 - lava_callback - INFO - Completed processing callback for node 69b36a53974c91e64560f1f1 INFO: 172.20.0.1:52176 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-13 01:38:15,604 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:38:15,619 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:52190 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43568 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58984 - "GET / HTTP/1.0" 200 OK 2026-03-13 01:38:46,417 - lava_callback - INFO - Received callback for node 69b0c79b9d19175818252bb9 INFO: 172.20.0.1:43056 - "POST /node/69b0c79b9d19175818252bb9 HTTP/1.0" 202 Accepted 2026-03-13 01:38:46,423 - lava_callback - INFO - Starting async job submit for node 69b0c79b9d19175818252bb9 2026-03-13 01:38:46,424 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:38:46,438 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0c79b9d19175818252bb9 HTTP/1.1" 200 None 2026-03-13 01:38:46,439 - lava_callback - DEBUG - Processing node 69b0c79b9d19175818252bb9: baseline-arm64-clabbe 2026-03-13 01:38:46,440 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0c79b9d19175818252bb9 2026-03-13 01:38:46,440 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:38:46,461 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0c79b9d19175818252bb9 HTTP/1.1" 200 None 2026-03-13 01:38:46,462 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:38:46,476 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11758ef4424e487b2eb1 HTTP/1.1" 200 None 2026-03-13 01:38:46,477 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:38:46,498 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0c79b9d19175818252bb9 HTTP/1.1" 200 None 2026-03-13 01:38:46,505 - lava_callback - ERROR - Error processing callback for node 69b0c79b9d19175818252bb9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:43072 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52564 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50774 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50782 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39428 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39440 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32776 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58456 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36386 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57822 - "GET / HTTP/1.0" 200 OK 2026-03-13 01:40:01,500 - lava_callback - INFO - Received callback for node 69b36abf974c91e64560f232 INFO: 172.20.0.1:33344 - "POST /node/69b36abf974c91e64560f232 HTTP/1.0" 202 Accepted 2026-03-13 01:40:01,506 - lava_callback - INFO - Starting async job submit for node 69b36abf974c91e64560f232 2026-03-13 01:40:01,528 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:40:01,542 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b36abf974c91e64560f232 HTTP/1.1" 200 None 2026-03-13 01:40:01,543 - lava_callback - DEBUG - Processing node 69b36abf974c91e64560f232: baseline-arm64-foundriesio 2026-03-13 01:40:01,766 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:40:01,937 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:40:01,939 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69b36abf974c91e64560f232/log.txt.gz 2026-03-13 01:40:02,063 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:40:02,290 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:40:02,292 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69b36abf974c91e64560f232/lava-logs.yaml 2026-03-13 01:40:02,298 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:40:02,464 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:40:02,465 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69b36abf974c91e64560f232/lava_callback.json.gz 2026-03-13 01:40:02,487 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:40:02,502 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b36abf974c91e64560f232 HTTP/1.1" 200 None 2026-03-13 01:40:02,503 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:40:02,522 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3594d974c91e64560e4b2 HTTP/1.1" 200 None 2026-03-13 01:40:02,524 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:40:02,551 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b36abf974c91e64560f232 HTTP/1.1" 200 None 2026-03-13 01:40:02,566 - lava_callback - INFO - Completed processing callback for node 69b36abf974c91e64560f232 INFO: 172.20.0.1:33356 - "GET / HTTP/1.0" 200 OK 2026-03-13 01:40:06,548 - lava_callback - INFO - Received callback for node 69b0c7ec9d19175818252bbe INFO: 172.20.0.1:33360 - "POST /node/69b0c7ec9d19175818252bbe HTTP/1.0" 202 Accepted 2026-03-13 01:40:06,553 - lava_callback - INFO - Starting async job submit for node 69b0c7ec9d19175818252bbe 2026-03-13 01:40:06,554 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:40:06,568 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0c7ec9d19175818252bbe HTTP/1.1" 200 None 2026-03-13 01:40:06,569 - lava_callback - DEBUG - Processing node 69b0c7ec9d19175818252bbe: baseline-arm64-clabbe 2026-03-13 01:40:06,569 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0c7ec9d19175818252bbe 2026-03-13 01:40:06,570 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:40:06,583 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0c7ec9d19175818252bbe HTTP/1.1" 200 None 2026-03-13 01:40:06,585 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:40:06,601 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11758ef4424e487b2eb1 HTTP/1.1" 200 None 2026-03-13 01:40:06,603 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:40:06,628 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0c7ec9d19175818252bbe HTTP/1.1" 200 None 2026-03-13 01:40:06,630 - lava_callback - ERROR - Error processing callback for node 69b0c7ec9d19175818252bbe: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:33364 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37324 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-13 01:40:15,622 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:40:15,637 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:37328 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53298 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49046 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56222 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46244 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59576 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59588 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46748 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46764 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55954 - "GET / HTTP/1.0" 200 OK 2026-03-13 01:41:26,583 - lava_callback - INFO - Received callback for node 69b0c83e9d19175818252bc7 INFO: 172.20.0.1:55966 - "POST /node/69b0c83e9d19175818252bc7 HTTP/1.0" 202 Accepted 2026-03-13 01:41:26,589 - lava_callback - INFO - Starting async job submit for node 69b0c83e9d19175818252bc7 2026-03-13 01:41:26,590 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:41:26,604 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0c83e9d19175818252bc7 HTTP/1.1" 200 None 2026-03-13 01:41:26,605 - lava_callback - DEBUG - Processing node 69b0c83e9d19175818252bc7: baseline-arm64-clabbe 2026-03-13 01:41:26,605 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0c83e9d19175818252bc7 2026-03-13 01:41:26,606 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:41:26,619 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0c83e9d19175818252bc7 HTTP/1.1" 200 None 2026-03-13 01:41:26,620 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:41:26,641 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11758ef4424e487b2eb1 HTTP/1.1" 200 None 2026-03-13 01:41:26,643 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:41:26,663 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0c83e9d19175818252bc7 HTTP/1.1" 200 None 2026-03-13 01:41:26,664 - lava_callback - ERROR - Error processing callback for node 69b0c83e9d19175818252bc7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:36766 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55424 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49604 - "GET / HTTP/1.0" 200 OK 2026-03-13 01:41:56,727 - lava_callback - INFO - Received callback for node 69b36b17974c91e64560f268 INFO: 172.20.0.1:49614 - "POST /node/69b36b17974c91e64560f268 HTTP/1.0" 202 Accepted 2026-03-13 01:41:56,734 - lava_callback - INFO - Starting async job submit for node 69b36b17974c91e64560f268 2026-03-13 01:41:56,765 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:41:56,778 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b36b17974c91e64560f268 HTTP/1.1" 200 None 2026-03-13 01:41:56,779 - lava_callback - DEBUG - Processing node 69b36b17974c91e64560f268: baseline-arm64-foundriesio 2026-03-13 01:41:57,180 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:41:57,371 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:41:57,373 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69b36b17974c91e64560f268/log.txt.gz 2026-03-13 01:41:57,640 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:41:57,865 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:41:57,868 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69b36b17974c91e64560f268/lava-logs.yaml 2026-03-13 01:41:57,878 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:41:58,050 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:41:58,052 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69b36b17974c91e64560f268/lava_callback.json.gz 2026-03-13 01:41:58,082 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:41:58,095 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b36b17974c91e64560f268 HTTP/1.1" 200 None 2026-03-13 01:41:58,097 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:41:58,111 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3594d974c91e64560e4b2 HTTP/1.1" 200 None 2026-03-13 01:41:58,112 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:41:58,143 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b36b17974c91e64560f268 HTTP/1.1" 200 None 2026-03-13 01:41:58,149 - lava_callback - INFO - Completed processing callback for node 69b36b17974c91e64560f268 INFO: 172.20.0.1:40854 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40860 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57710 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-13 01:42:15,640 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:42:15,656 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:57712 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34796 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42368 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45618 - "GET / HTTP/1.0" 200 OK 2026-03-13 01:42:46,649 - lava_callback - INFO - Received callback for node 69b0c8919d19175818252bd0 INFO: 172.20.0.1:45630 - "POST /node/69b0c8919d19175818252bd0 HTTP/1.0" 202 Accepted 2026-03-13 01:42:46,655 - lava_callback - INFO - Starting async job submit for node 69b0c8919d19175818252bd0 2026-03-13 01:42:46,656 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:42:46,671 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0c8919d19175818252bd0 HTTP/1.1" 200 None 2026-03-13 01:42:46,672 - lava_callback - DEBUG - Processing node 69b0c8919d19175818252bd0: baseline-arm64-clabbe 2026-03-13 01:42:46,672 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0c8919d19175818252bd0 2026-03-13 01:42:46,673 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:42:46,686 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0c8919d19175818252bd0 HTTP/1.1" 200 None 2026-03-13 01:42:46,688 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:42:46,726 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11758ef4424e487b2eb1 HTTP/1.1" 200 None 2026-03-13 01:42:46,728 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:42:46,750 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0c8919d19175818252bd0 HTTP/1.1" 200 None 2026-03-13 01:42:46,751 - lava_callback - ERROR - Error processing callback for node 69b0c8919d19175818252bd0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:56308 - "GET /node/699aec79c1226ad11a5f4195%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:56322 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59784 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59792 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53582 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53594 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37836 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48500 - "GET / HTTP/1.0" 200 OK 2026-03-13 01:43:45,867 - lava_callback - INFO - Received callback for node 69b367e8974c91e64560f0be INFO: 172.20.0.1:50804 - "POST /node/69b367e8974c91e64560f0be HTTP/1.0" 202 Accepted 2026-03-13 01:43:45,874 - lava_callback - INFO - Starting async job submit for node 69b367e8974c91e64560f0be 2026-03-13 01:43:45,928 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:43:45,943 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b367e8974c91e64560f0be HTTP/1.1" 200 None 2026-03-13 01:43:45,944 - lava_callback - DEBUG - Processing node 69b367e8974c91e64560f0be: baseline-arm 2026-03-13 01:43:46,516 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:50808 - "GET / HTTP/1.0" 200 OK 2026-03-13 01:43:46,816 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:43:46,819 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69b367e8974c91e64560f0be/log.txt.gz 2026-03-13 01:43:47,176 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:43:47,451 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:43:47,453 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69b367e8974c91e64560f0be/lava-logs.yaml 2026-03-13 01:43:47,467 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:43:47,672 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:43:47,674 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69b367e8974c91e64560f0be/lava_callback.json.gz DEBUG: Setup failed for 69b367e8974c91e64560f0be. Transitting job node result: incomplete -> fail 2026-03-13 01:43:47,792 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:43:47,807 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b367e8974c91e64560f0be HTTP/1.1" 200 None 2026-03-13 01:43:47,809 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:43:47,823 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35934974c91e64560e49d HTTP/1.1" 200 None 2026-03-13 01:43:47,824 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:43:47,869 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b367e8974c91e64560f0be HTTP/1.1" 200 None 2026-03-13 01:43:47,987 - lava_callback - INFO - Completed processing callback for node 69b367e8974c91e64560f0be 2026-03-13 01:43:50,001 - lava_callback - INFO - Received callback for node 69b3639f974c91e64560ece0 INFO: 172.20.0.1:34020 - "POST /node/69b3639f974c91e64560ece0 HTTP/1.0" 202 Accepted 2026-03-13 01:43:50,007 - lava_callback - INFO - Starting async job submit for node 69b3639f974c91e64560ece0 2026-03-13 01:43:50,024 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:43:50,040 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3639f974c91e64560ece0 HTTP/1.1" 200 None 2026-03-13 01:43:50,041 - lava_callback - DEBUG - Processing node 69b3639f974c91e64560ece0: baseline-arm64 2026-03-13 01:43:50,355 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:43:50,515 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:43:50,517 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69b3639f974c91e64560ece0/log.txt.gz 2026-03-13 01:43:50,708 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:43:50,916 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:43:50,919 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69b3639f974c91e64560ece0/lava-logs.yaml 2026-03-13 01:43:50,928 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:43:51,057 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:43:51,058 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69b3639f974c91e64560ece0/lava_callback.json.gz 2026-03-13 01:43:51,076 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:43:51,093 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3639f974c91e64560ece0 HTTP/1.1" 200 None 2026-03-13 01:43:51,094 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:43:51,107 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3594d974c91e64560e4b2 HTTP/1.1" 200 None 2026-03-13 01:43:51,109 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:43:51,134 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b3639f974c91e64560ece0 HTTP/1.1" 200 None 2026-03-13 01:43:51,144 - lava_callback - INFO - Completed processing callback for node 69b3639f974c91e64560ece0 INFO: 172.20.0.1:34024 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54334 - "GET / HTTP/1.0" 200 OK 2026-03-13 01:44:06,679 - lava_callback - INFO - Received callback for node 69b0c8e29d19175818252bd9 INFO: 172.20.0.1:54346 - "POST /node/69b0c8e29d19175818252bd9 HTTP/1.0" 202 Accepted 2026-03-13 01:44:06,685 - lava_callback - INFO - Starting async job submit for node 69b0c8e29d19175818252bd9 2026-03-13 01:44:06,686 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:44:06,701 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0c8e29d19175818252bd9 HTTP/1.1" 200 None 2026-03-13 01:44:06,701 - lava_callback - DEBUG - Processing node 69b0c8e29d19175818252bd9: baseline-arm64-clabbe 2026-03-13 01:44:06,702 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0c8e29d19175818252bd9 2026-03-13 01:44:06,702 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:44:06,724 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0c8e29d19175818252bd9 HTTP/1.1" 200 None 2026-03-13 01:44:06,725 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:44:06,745 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11758ef4424e487b2eb1 HTTP/1.1" 200 None 2026-03-13 01:44:06,746 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:44:06,770 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0c8e29d19175818252bd9 HTTP/1.1" 200 None 2026-03-13 01:44:06,771 - lava_callback - ERROR - Error processing callback for node 69b0c8e29d19175818252bd9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:54358 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50180 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-13 01:44:15,659 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:44:15,675 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:50188 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51278 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59844 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52482 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34240 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60016 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60026 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52818 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52820 - "GET / HTTP/1.0" 200 OK 2026-03-13 01:45:23,875 - lava_callback - INFO - Received callback for node 69b363b2974c91e64560ed0f INFO: 172.20.0.1:58864 - "POST /node/69b363b2974c91e64560ed0f HTTP/1.0" 202 Accepted 2026-03-13 01:45:23,883 - lava_callback - INFO - Starting async job submit for node 69b363b2974c91e64560ed0f 2026-03-13 01:45:23,926 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:45:23,940 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b363b2974c91e64560ed0f HTTP/1.1" 200 None 2026-03-13 01:45:23,941 - lava_callback - DEBUG - Processing node 69b363b2974c91e64560ed0f: baseline-nfs-arm64 2026-03-13 01:45:25,062 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:45:25,267 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:45:25,269 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b363b2974c91e64560ed0f/log.txt.gz 2026-03-13 01:45:25,901 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:45:26,182 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:45:26,185 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b363b2974c91e64560ed0f/lava-logs.yaml 2026-03-13 01:45:26,206 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:45:26,384 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:45:26,386 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b363b2974c91e64560ed0f/lava_callback.json.gz DEBUG: Setup failed for 69b363b2974c91e64560ed0f. Transitting job node result: incomplete -> fail 2026-03-13 01:45:26,474 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:45:26,489 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b363b2974c91e64560ed0f HTTP/1.1" 200 None 2026-03-13 01:45:26,491 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:45:26,506 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3594d974c91e64560e4b2 HTTP/1.1" 200 None 2026-03-13 01:45:26,507 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:45:26,532 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b363b2974c91e64560ed0f HTTP/1.1" 200 None INFO: 172.20.0.1:58876 - "GET / HTTP/1.0" 200 OK 2026-03-13 01:45:26,640 - lava_callback - INFO - Completed processing callback for node 69b363b2974c91e64560ed0f 2026-03-13 01:45:26,746 - lava_callback - INFO - Received callback for node 69b0c9339d19175818252be4 INFO: 172.20.0.1:58882 - "POST /node/69b0c9339d19175818252be4 HTTP/1.0" 202 Accepted 2026-03-13 01:45:26,753 - lava_callback - INFO - Starting async job submit for node 69b0c9339d19175818252be4 2026-03-13 01:45:26,754 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:45:26,769 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0c9339d19175818252be4 HTTP/1.1" 200 None 2026-03-13 01:45:26,771 - lava_callback - DEBUG - Processing node 69b0c9339d19175818252be4: baseline-arm64-clabbe 2026-03-13 01:45:26,771 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0c9339d19175818252be4 2026-03-13 01:45:26,772 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:45:26,791 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0c9339d19175818252be4 HTTP/1.1" 200 None 2026-03-13 01:45:26,792 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:45:26,805 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11758ef4424e487b2eb1 HTTP/1.1" 200 None 2026-03-13 01:45:26,806 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:45:26,828 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0c9339d19175818252be4 HTTP/1.1" 200 None 2026-03-13 01:45:26,833 - lava_callback - ERROR - Error processing callback for node 69b0c9339d19175818252be4: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:51422 - "GET / HTTP/1.0" 200 OK 2026-03-13 01:45:45,677 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:45:45,694 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:33042 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47708 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42164 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42166 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36948 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36952 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52662 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46198 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42540 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38806 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50364 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50374 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39090 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39106 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38936 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58614 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52140 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57518 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44288 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44304 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46064 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46076 - "GET / HTTP/1.0" 200 OK 2026-03-13 01:48:24,456 - lava_callback - INFO - Received callback for node 69b36cc5974c91e64560f375 INFO: 172.20.0.1:38346 - "POST /node/69b36cc5974c91e64560f375 HTTP/1.0" 202 Accepted 2026-03-13 01:48:24,461 - lava_callback - INFO - Starting async job submit for node 69b36cc5974c91e64560f375 2026-03-13 01:48:24,464 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:48:24,486 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b36cc5974c91e64560f375 HTTP/1.1" 200 None 2026-03-13 01:48:24,486 - lava_callback - DEBUG - Processing node 69b36cc5974c91e64560f375: baseline-nfs-arm64 2026-03-13 01:48:24,488 - lava_callback - WARNING - Failed to upload log 2026-03-13 01:48:24,489 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:48:24,632 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:48:24,634 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b36cc5974c91e64560f375/lava-logs.yaml 2026-03-13 01:48:24,635 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:48:24,786 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:48:24,788 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b36cc5974c91e64560f375/lava_callback.json.gz 2026-03-13 01:48:24,790 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:48:24,808 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b36cc5974c91e64560f375 HTTP/1.1" 200 None 2026-03-13 01:48:24,809 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:48:24,823 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3594d974c91e64560e4b2 HTTP/1.1" 200 None 2026-03-13 01:48:24,824 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:48:24,846 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b36cc5974c91e64560f375 HTTP/1.1" 200 None 2026-03-13 01:48:24,857 - lava_callback - INFO - Completed processing callback for node 69b36cc5974c91e64560f375 INFO: 172.20.0.1:38348 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53750 - "GET / HTTP/1.0" 200 OK 2026-03-13 01:48:45,696 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:48:45,711 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:40804 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59114 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36480 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36488 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56442 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56444 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47280 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45584 - "GET / HTTP/1.0" 200 OK 2026-03-13 01:49:39,428 - lava_callback - INFO - Received callback for node 69b36b5b974c91e64560f282 INFO: 172.20.0.1:45598 - "POST /node/69b36b5b974c91e64560f282 HTTP/1.0" 202 Accepted 2026-03-13 01:49:39,435 - lava_callback - INFO - Starting async job submit for node 69b36b5b974c91e64560f282 2026-03-13 01:49:39,471 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:49:39,494 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b36b5b974c91e64560f282 HTTP/1.1" 200 None 2026-03-13 01:49:39,495 - lava_callback - DEBUG - Processing node 69b36b5b974c91e64560f282: baseline-arm64 2026-03-13 01:49:39,887 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:49:40,049 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:49:40,051 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69b36b5b974c91e64560f282/log.txt.gz 2026-03-13 01:49:40,270 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:49:40,496 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:49:40,499 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69b36b5b974c91e64560f282/lava-logs.yaml 2026-03-13 01:49:40,509 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:49:40,666 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:49:40,668 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69b36b5b974c91e64560f282/lava_callback.json.gz DEBUG: Setup failed for 69b36b5b974c91e64560f282. Transitting job node result: incomplete -> fail 2026-03-13 01:49:40,746 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:49:40,761 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b36b5b974c91e64560f282 HTTP/1.1" 200 None 2026-03-13 01:49:40,763 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:49:40,776 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3594d974c91e64560e4b2 HTTP/1.1" 200 None 2026-03-13 01:49:40,778 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:49:40,802 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b36b5b974c91e64560f282 HTTP/1.1" 200 None 2026-03-13 01:49:40,892 - lava_callback - INFO - Completed processing callback for node 69b36b5b974c91e64560f282 2026-03-13 01:49:45,713 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:49:45,737 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:54646 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47520 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58290 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58306 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41920 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41924 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56762 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42038 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60268 - "GET / HTTP/1.0" 200 OK 2026-03-13 01:50:49,638 - lava_callback - INFO - Received callback for node 69b0ca6b9d19175818252c12 INFO: 172.20.0.1:60280 - "POST /node/69b0ca6b9d19175818252c12 HTTP/1.0" 202 Accepted 2026-03-13 01:50:49,644 - lava_callback - INFO - Starting async job submit for node 69b0ca6b9d19175818252c12 2026-03-13 01:50:49,645 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:50:49,659 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0ca6b9d19175818252c12 HTTP/1.1" 200 None 2026-03-13 01:50:49,660 - lava_callback - DEBUG - Processing node 69b0ca6b9d19175818252c12: baseline-arm64-baylibre 2026-03-13 01:50:49,660 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0ca6b9d19175818252c12 2026-03-13 01:50:49,661 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:50:49,674 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0ca6b9d19175818252c12 HTTP/1.1" 200 None 2026-03-13 01:50:49,675 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:50:49,688 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e44efa461f55e59cdc8 HTTP/1.1" 200 None 2026-03-13 01:50:49,690 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:50:49,711 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0ca6b9d19175818252c12 HTTP/1.1" 200 None 2026-03-13 01:50:49,717 - lava_callback - ERROR - Error processing callback for node 69b0ca6b9d19175818252c12: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:41506 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59598 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59608 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51558 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51560 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53844 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53250 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58764 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49740 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39410 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39414 - "GET / HTTP/1.0" 200 OK 2026-03-13 01:52:09,461 - lava_callback - INFO - Received callback for node 69b0caba9d19175818252c18 INFO: 172.20.0.1:39424 - "POST /node/69b0caba9d19175818252c18 HTTP/1.0" 202 Accepted 2026-03-13 01:52:09,466 - lava_callback - INFO - Starting async job submit for node 69b0caba9d19175818252c18 2026-03-13 01:52:09,467 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:52:09,482 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0caba9d19175818252c18 HTTP/1.1" 200 None 2026-03-13 01:52:09,483 - lava_callback - DEBUG - Processing node 69b0caba9d19175818252c18: baseline-arm64-baylibre 2026-03-13 01:52:09,483 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0caba9d19175818252c18 2026-03-13 01:52:09,483 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:52:09,504 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0caba9d19175818252c18 HTTP/1.1" 200 None 2026-03-13 01:52:09,506 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:52:09,525 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e44efa461f55e59cdc8 HTTP/1.1" 200 None 2026-03-13 01:52:09,526 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:52:09,555 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0caba9d19175818252c18 HTTP/1.1" 200 None 2026-03-13 01:52:09,557 - lava_callback - ERROR - Error processing callback for node 69b0caba9d19175818252c18: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:52516 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52518 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57466 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38390 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40910 - "GET / HTTP/1.0" 200 OK 2026-03-13 01:52:52,657 - lava_callback - INFO - Received callback for node 69b36dc7974c91e64560f3fd INFO: 172.20.0.1:44460 - "POST /node/69b36dc7974c91e64560f3fd HTTP/1.0" 202 Accepted 2026-03-13 01:52:52,664 - lava_callback - INFO - Starting async job submit for node 69b36dc7974c91e64560f3fd 2026-03-13 01:52:52,693 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:52:52,709 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b36dc7974c91e64560f3fd HTTP/1.1" 200 None 2026-03-13 01:52:52,710 - lava_callback - DEBUG - Processing node 69b36dc7974c91e64560f3fd: baseline-arm64 2026-03-13 01:52:52,858 - lava_callback - WARNING - Failed to upload log 2026-03-13 01:52:52,928 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:52:53,156 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:52:53,159 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69b36dc7974c91e64560f3fd/lava-logs.yaml 2026-03-13 01:52:53,163 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:52:53,315 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:52:53,317 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69b36dc7974c91e64560f3fd/lava_callback.json.gz 2026-03-13 01:52:53,347 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:52:53,361 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b36dc7974c91e64560f3fd HTTP/1.1" 200 None 2026-03-13 01:52:53,363 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:52:53,377 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3594d974c91e64560e4b2 HTTP/1.1" 200 None 2026-03-13 01:52:53,379 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:52:53,406 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b36dc7974c91e64560f3fd HTTP/1.1" 200 None 2026-03-13 01:52:53,486 - lava_callback - INFO - Completed processing callback for node 69b36dc7974c91e64560f3fd INFO: 172.20.0.1:44474 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57152 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57160 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38838 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-13 01:53:15,739 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:53:15,754 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:38850 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40708 - "GET / HTTP/1.0" 200 OK 2026-03-13 01:53:29,592 - lava_callback - INFO - Received callback for node 69b0cb0b9d19175818252c21 INFO: 172.20.0.1:40720 - "POST /node/69b0cb0b9d19175818252c21 HTTP/1.0" 202 Accepted 2026-03-13 01:53:29,598 - lava_callback - INFO - Starting async job submit for node 69b0cb0b9d19175818252c21 2026-03-13 01:53:29,599 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:53:29,620 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0cb0b9d19175818252c21 HTTP/1.1" 200 None 2026-03-13 01:53:29,621 - lava_callback - DEBUG - Processing node 69b0cb0b9d19175818252c21: baseline-arm64-baylibre 2026-03-13 01:53:29,621 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0cb0b9d19175818252c21 2026-03-13 01:53:29,622 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:53:29,635 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0cb0b9d19175818252c21 HTTP/1.1" 200 None 2026-03-13 01:53:29,636 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:53:29,649 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e44efa461f55e59cdc8 HTTP/1.1" 200 None 2026-03-13 01:53:29,651 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:53:29,671 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0cb0b9d19175818252c21 HTTP/1.1" 200 None 2026-03-13 01:53:29,677 - lava_callback - ERROR - Error processing callback for node 69b0cb0b9d19175818252c21: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:41920 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37522 - "GET / HTTP/1.0" 200 OK 2026-03-13 01:53:49,219 - lava_callback - INFO - Received callback for node 69b363b0974c91e64560ed07 INFO: 172.20.0.1:37536 - "POST /node/69b363b0974c91e64560ed07 HTTP/1.0" 202 Accepted 2026-03-13 01:53:49,225 - lava_callback - INFO - Starting async job submit for node 69b363b0974c91e64560ed07 2026-03-13 01:53:49,247 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:53:49,287 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b363b0974c91e64560ed07 HTTP/1.1" 200 None 2026-03-13 01:53:49,288 - lava_callback - DEBUG - Processing node 69b363b0974c91e64560ed07: baseline-nfs-arm64 2026-03-13 01:53:49,746 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:53:49,911 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:53:49,913 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b363b0974c91e64560ed07/log.txt.gz 2026-03-13 01:53:50,176 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:53:50,388 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:53:50,390 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b363b0974c91e64560ed07/lava-logs.yaml 2026-03-13 01:53:50,401 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:53:50,677 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:53:50,678 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b363b0974c91e64560ed07/lava_callback.json.gz 2026-03-13 01:53:50,700 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:53:50,715 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b363b0974c91e64560ed07 HTTP/1.1" 200 None 2026-03-13 01:53:50,717 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:53:50,731 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3594d974c91e64560e4b2 HTTP/1.1" 200 None 2026-03-13 01:53:50,733 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:53:50,758 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b363b0974c91e64560ed07 HTTP/1.1" 200 None 2026-03-13 01:53:50,759 - lava_callback - INFO - Completed processing callback for node 69b363b0974c91e64560ed07 INFO: 172.20.0.1:37728 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48614 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48626 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37754 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-13 01:54:15,756 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:54:15,773 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:37762 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37748 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47270 - "GET / HTTP/1.0" 200 OK 2026-03-13 01:54:39,048 - lava_callback - INFO - Received callback for node 69b36e22974c91e64560f432 INFO: 172.20.0.1:47280 - "POST /node/69b36e22974c91e64560f432 HTTP/1.0" 202 Accepted 2026-03-13 01:54:39,055 - lava_callback - INFO - Starting async job submit for node 69b36e22974c91e64560f432 2026-03-13 01:54:39,084 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:54:39,101 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b36e22974c91e64560f432 HTTP/1.1" 200 None 2026-03-13 01:54:39,102 - lava_callback - DEBUG - Processing node 69b36e22974c91e64560f432: baseline-arm64 2026-03-13 01:54:39,243 - lava_callback - WARNING - Failed to upload log 2026-03-13 01:54:39,315 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:54:39,528 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:54:39,530 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69b36e22974c91e64560f432/lava-logs.yaml 2026-03-13 01:54:39,534 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 01:54:39,671 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 01:54:39,673 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69b36e22974c91e64560f432/lava_callback.json.gz 2026-03-13 01:54:39,704 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:54:39,718 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b36e22974c91e64560f432 HTTP/1.1" 200 None 2026-03-13 01:54:39,719 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:54:39,733 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3594d974c91e64560e4b2 HTTP/1.1" 200 None 2026-03-13 01:54:39,734 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:54:39,755 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b36e22974c91e64560f432 HTTP/1.1" 200 None 2026-03-13 01:54:39,828 - lava_callback - INFO - Completed processing callback for node 69b36e22974c91e64560f432 2026-03-13 01:54:45,775 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:54:45,792 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:39806 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35032 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45704 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45716 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41364 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41372 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38558 - "GET / HTTP/1.0" 200 OK 2026-03-13 01:55:29,672 - lava_callback - INFO - Received callback for node 69b0cb809d19175818252c35 INFO: 172.20.0.1:38562 - "POST /node/69b0cb809d19175818252c35 HTTP/1.0" 202 Accepted 2026-03-13 01:55:29,678 - lava_callback - INFO - Starting async job submit for node 69b0cb809d19175818252c35 2026-03-13 01:55:29,679 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:55:29,698 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0cb809d19175818252c35 HTTP/1.1" 200 None 2026-03-13 01:55:29,699 - lava_callback - DEBUG - Processing node 69b0cb809d19175818252c35: baseline-arm64-baylibre 2026-03-13 01:55:29,699 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0cb809d19175818252c35 2026-03-13 01:55:29,700 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:55:29,713 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0cb809d19175818252c35 HTTP/1.1" 200 None 2026-03-13 01:55:29,715 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:55:29,727 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e44efa461f55e59cdc8 HTTP/1.1" 200 None 2026-03-13 01:55:29,729 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:55:29,809 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0cb809d19175818252c35 HTTP/1.1" 200 None 2026-03-13 01:55:29,815 - lava_callback - ERROR - Error processing callback for node 69b0cb809d19175818252c35: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:33016 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39168 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37880 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60110 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60122 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57728 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57744 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60124 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50956 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33640 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59916 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40854 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40856 - "GET / HTTP/1.0" 200 OK 2026-03-13 01:57:09,753 - lava_callback - INFO - Received callback for node 69b0cbf29d19175818252c4f INFO: 172.20.0.1:50934 - "POST /node/69b0cbf29d19175818252c4f HTTP/1.0" 202 Accepted 2026-03-13 01:57:09,758 - lava_callback - INFO - Starting async job submit for node 69b0cbf29d19175818252c4f 2026-03-13 01:57:09,759 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:57:09,783 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0cbf29d19175818252c4f HTTP/1.1" 200 None 2026-03-13 01:57:09,785 - lava_callback - DEBUG - Processing node 69b0cbf29d19175818252c4f: baseline-arm64-baylibre 2026-03-13 01:57:09,785 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0cbf29d19175818252c4f 2026-03-13 01:57:09,785 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:57:09,802 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0cbf29d19175818252c4f HTTP/1.1" 200 None 2026-03-13 01:57:09,804 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:57:09,817 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e44efa461f55e59cdc8 HTTP/1.1" 200 None 2026-03-13 01:57:09,818 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:57:09,839 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0cbf29d19175818252c4f HTTP/1.1" 200 None 2026-03-13 01:57:09,840 - lava_callback - ERROR - Error processing callback for node 69b0cbf29d19175818252c4f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:50950 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50956 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49080 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58452 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58458 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47518 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41180 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41182 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35660 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35668 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48782 - "GET / HTTP/1.0" 200 OK 2026-03-13 01:58:29,876 - lava_callback - INFO - Received callback for node 69b0cc409d19175818252c56 INFO: 172.20.0.1:46368 - "POST /node/69b0cc409d19175818252c56 HTTP/1.0" 202 Accepted 2026-03-13 01:58:29,881 - lava_callback - INFO - Starting async job submit for node 69b0cc409d19175818252c56 2026-03-13 01:58:29,882 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:58:29,898 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0cc409d19175818252c56 HTTP/1.1" 200 None 2026-03-13 01:58:29,898 - lava_callback - DEBUG - Processing node 69b0cc409d19175818252c56: baseline-arm64-baylibre 2026-03-13 01:58:29,899 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0cc409d19175818252c56 2026-03-13 01:58:29,899 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:58:29,913 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0cc409d19175818252c56 HTTP/1.1" 200 None 2026-03-13 01:58:29,915 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:58:29,933 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e44efa461f55e59cdc8 HTTP/1.1" 200 None 2026-03-13 01:58:29,935 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 01:58:29,961 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0cc409d19175818252c56 HTTP/1.1" 200 None 2026-03-13 01:58:29,963 - lava_callback - ERROR - Error processing callback for node 69b0cc409d19175818252c56: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:46380 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58222 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41842 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36694 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36702 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41194 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41206 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60026 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54750 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58606 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41142 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38560 - "GET / HTTP/1.0" 200 OK 2026-03-13 02:00:07,693 - lava_callback - INFO - Received callback for node 69b0cc959d19175818252c60 INFO: 172.20.0.1:38562 - "POST /node/69b0cc959d19175818252c60 HTTP/1.0" 202 Accepted 2026-03-13 02:00:07,699 - lava_callback - INFO - Starting async job submit for node 69b0cc959d19175818252c60 2026-03-13 02:00:07,700 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 02:00:07,719 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0cc959d19175818252c60 HTTP/1.1" 200 None 2026-03-13 02:00:07,719 - lava_callback - DEBUG - Processing node 69b0cc959d19175818252c60: baseline-arm64-clabbe 2026-03-13 02:00:07,720 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0cc959d19175818252c60 2026-03-13 02:00:07,720 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 02:00:07,738 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0cc959d19175818252c60 HTTP/1.1" 200 None 2026-03-13 02:00:07,739 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 02:00:07,754 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e44efa461f55e59cdc8 HTTP/1.1" 200 None 2026-03-13 02:00:07,757 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 02:00:07,778 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0cc959d19175818252c60 HTTP/1.1" 200 None 2026-03-13 02:00:07,783 - lava_callback - ERROR - Error processing callback for node 69b0cc959d19175818252c60: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:38572 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53806 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53812 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49228 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50746 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44110 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39514 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45666 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45674 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51256 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51270 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50482 - "GET / HTTP/1.0" 200 OK 2026-03-13 02:01:27,444 - lava_callback - INFO - Received callback for node 69b0cce69d19175818252c6a INFO: 172.20.0.1:50488 - "POST /node/69b0cce69d19175818252c6a HTTP/1.0" 202 Accepted 2026-03-13 02:01:27,450 - lava_callback - INFO - Starting async job submit for node 69b0cce69d19175818252c6a 2026-03-13 02:01:27,451 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 02:01:27,465 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0cce69d19175818252c6a HTTP/1.1" 200 None 2026-03-13 02:01:27,465 - lava_callback - DEBUG - Processing node 69b0cce69d19175818252c6a: baseline-arm64-clabbe 2026-03-13 02:01:27,466 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0cce69d19175818252c6a 2026-03-13 02:01:27,466 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 02:01:27,483 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0cce69d19175818252c6a HTTP/1.1" 200 None 2026-03-13 02:01:27,484 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 02:01:27,499 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e44efa461f55e59cdc8 HTTP/1.1" 200 None 2026-03-13 02:01:27,500 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 02:01:27,544 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0cce69d19175818252c6a HTTP/1.1" 200 None 2026-03-13 02:01:27,545 - lava_callback - ERROR - Error processing callback for node 69b0cce69d19175818252c6a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:56302 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37520 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58258 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38710 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38726 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46506 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46508 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51614 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55508 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54952 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49660 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42294 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42304 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50288 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50302 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59676 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42240 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51096 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48322 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56124 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56130 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41336 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41350 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55486 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54294 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38356 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46124 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57812 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57816 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34860 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34864 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40210 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37014 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58440 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38860 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52640 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52652 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57930 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57946 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59656 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56490 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38650 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54864 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52868 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52870 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55120 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55132 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57532 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42182 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58094 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56638 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48128 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48130 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58798 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58806 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46944 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54770 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47062 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35216 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56768 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56778 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37984 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37992 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57894 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45612 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46436 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36494 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37776 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37792 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52514 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52522 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48916 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32850 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33542 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48692 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47806 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47816 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54370 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54378 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60554 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57982 - "GET / HTTP/1.0" 200 OK 2026-03-13 02:11:45,389 - lava_callback - INFO - Received callback for node 69b37242974c91e64560f5e2 INFO: 172.20.0.1:54316 - "POST /node/69b37242974c91e64560f5e2 HTTP/1.0" 202 Accepted 2026-03-13 02:11:45,395 - lava_callback - INFO - Starting async job submit for node 69b37242974c91e64560f5e2 2026-03-13 02:11:45,397 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 02:11:45,411 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b37242974c91e64560f5e2 HTTP/1.1" 200 None 2026-03-13 02:11:45,412 - lava_callback - DEBUG - Processing node 69b37242974c91e64560f5e2: baseline-nfs-arm64 2026-03-13 02:11:45,413 - lava_callback - WARNING - Failed to upload log 2026-03-13 02:11:45,414 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 02:11:45,581 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 02:11:45,583 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b37242974c91e64560f5e2/lava-logs.yaml 2026-03-13 02:11:45,585 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 02:11:45,732 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 02:11:45,735 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b37242974c91e64560f5e2/lava_callback.json.gz 2026-03-13 02:11:45,736 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 02:11:45,751 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b37242974c91e64560f5e2 HTTP/1.1" 200 None 2026-03-13 02:11:45,753 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 02:11:45,766 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3594d974c91e64560e4b2 HTTP/1.1" 200 None 2026-03-13 02:11:45,768 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 02:11:45,789 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b37242974c91e64560f5e2 HTTP/1.1" 200 None 2026-03-13 02:11:45,793 - lava_callback - INFO - Completed processing callback for node 69b37242974c91e64560f5e2 2026-03-13 02:11:45,797 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 02:11:45,874 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:54328 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35836 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35722 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35732 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57314 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57322 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58902 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47438 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33538 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57892 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47206 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47208 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46392 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46408 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34866 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38160 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38000 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51336 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52894 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52910 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55490 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55500 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59476 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40354 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49656 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52814 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38028 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38038 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57464 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57470 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41156 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38192 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37014 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52466 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59240 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59256 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38086 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38100 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55956 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41348 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49062 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40164 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36174 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36180 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39128 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39132 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53358 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57938 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49620 - "GET / HTTP/1.0" 200 OK 2026-03-13 02:17:52,262 - lava_callback - INFO - Received callback for node 69b3729e974c91e64560f5f9 INFO: 172.20.0.1:38982 - "POST /node/69b3729e974c91e64560f5f9 HTTP/1.0" 202 Accepted 2026-03-13 02:17:52,269 - lava_callback - INFO - Starting async job submit for node 69b3729e974c91e64560f5f9 2026-03-13 02:17:52,304 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 02:17:52,318 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3729e974c91e64560f5f9 HTTP/1.1" 200 None 2026-03-13 02:17:52,319 - lava_callback - DEBUG - Processing node 69b3729e974c91e64560f5f9: baseline-arm64 2026-03-13 02:17:52,705 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 02:17:52,856 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 02:17:52,857 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69b3729e974c91e64560f5f9/log.txt.gz 2026-03-13 02:17:53,075 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 02:17:53,312 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 02:17:53,314 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69b3729e974c91e64560f5f9/lava-logs.yaml 2026-03-13 02:17:53,323 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 02:17:53,535 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 02:17:53,537 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69b3729e974c91e64560f5f9/lava_callback.json.gz DEBUG: Setup failed for 69b3729e974c91e64560f5f9. Transitting job node result: incomplete -> fail 2026-03-13 02:17:53,611 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 02:17:53,633 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3729e974c91e64560f5f9 HTTP/1.1" 200 None 2026-03-13 02:17:53,635 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 02:17:53,648 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3594d974c91e64560e4b2 HTTP/1.1" 200 None 2026-03-13 02:17:53,650 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 02:17:53,680 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b3729e974c91e64560f5f9 HTTP/1.1" 200 None 2026-03-13 02:17:53,751 - lava_callback - INFO - Completed processing callback for node 69b3729e974c91e64560f5f9 INFO: 172.20.0.1:38994 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43012 - "GET / HTTP/1.0" 200 OK 2026-03-13 02:18:09,157 - lava_callback - INFO - Received callback for node 69b373ad974c91e64560f642 INFO: 172.20.0.1:43028 - "POST /node/69b373ad974c91e64560f642 HTTP/1.0" 202 Accepted 2026-03-13 02:18:09,163 - lava_callback - INFO - Starting async job submit for node 69b373ad974c91e64560f642 2026-03-13 02:18:09,192 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 02:18:09,206 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b373ad974c91e64560f642 HTTP/1.1" 200 None 2026-03-13 02:18:09,207 - lava_callback - DEBUG - Processing node 69b373ad974c91e64560f642: baseline-arm64 2026-03-13 02:18:09,341 - lava_callback - WARNING - Failed to upload log 2026-03-13 02:18:09,415 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:43030 - "GET / HTTP/1.0" 200 OK 2026-03-13 02:18:09,614 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 02:18:09,616 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69b373ad974c91e64560f642/lava-logs.yaml 2026-03-13 02:18:09,619 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 02:18:09,750 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 02:18:09,752 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69b373ad974c91e64560f642/lava_callback.json.gz 2026-03-13 02:18:09,782 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 02:18:09,797 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b373ad974c91e64560f642 HTTP/1.1" 200 None 2026-03-13 02:18:09,798 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 02:18:09,812 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3594d974c91e64560e4b2 HTTP/1.1" 200 None 2026-03-13 02:18:09,813 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 02:18:09,834 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b373ad974c91e64560f642 HTTP/1.1" 200 None 2026-03-13 02:18:09,942 - lava_callback - INFO - Completed processing callback for node 69b373ad974c91e64560f642 INFO: 172.20.0.1:46442 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-13 02:18:15,878 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 02:18:15,906 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:46448 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58676 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59632 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43462 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60612 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39830 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39834 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41280 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41294 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38756 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53178 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48854 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55372 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33450 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33460 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43482 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43496 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54370 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49536 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34990 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58552 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35608 - "GET / HTTP/1.0" 200 OK 2026-03-13 02:21:08,567 - lava_callback - INFO - Received callback for node 69b37462974c91e64560f678 INFO: 172.20.0.1:35610 - "POST /node/69b37462974c91e64560f678 HTTP/1.0" 202 Accepted 2026-03-13 02:21:08,574 - lava_callback - INFO - Starting async job submit for node 69b37462974c91e64560f678 2026-03-13 02:21:08,603 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 02:21:08,642 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b37462974c91e64560f678 HTTP/1.1" 200 None 2026-03-13 02:21:08,643 - lava_callback - DEBUG - Processing node 69b37462974c91e64560f678: baseline-arm64 2026-03-13 02:21:08,789 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 02:21:08,938 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 02:21:08,940 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69b37462974c91e64560f678/log.txt.gz 2026-03-13 02:21:09,011 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 02:21:09,219 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 02:21:09,221 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69b37462974c91e64560f678/lava-logs.yaml 2026-03-13 02:21:09,224 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 02:21:09,373 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 02:21:09,374 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69b37462974c91e64560f678/lava_callback.json.gz 2026-03-13 02:21:09,405 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 02:21:09,419 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b37462974c91e64560f678 HTTP/1.1" 200 None 2026-03-13 02:21:09,420 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 02:21:09,435 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3594d974c91e64560e4b2 HTTP/1.1" 200 None 2026-03-13 02:21:09,436 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 02:21:09,458 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b37462974c91e64560f678 HTTP/1.1" 200 None 2026-03-13 02:21:09,510 - lava_callback - INFO - Completed processing callback for node 69b37462974c91e64560f678 INFO: 172.20.0.1:35616 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43196 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-13 02:21:15,908 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 02:21:15,925 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:43208 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45214 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37784 - "GET / HTTP/1.0" 200 OK 2026-03-13 02:21:45,169 - lava_callback - INFO - Received callback for node 69b370a1974c91e64560f56d INFO: 172.20.0.1:39102 - "POST /node/69b370a1974c91e64560f56d HTTP/1.0" 202 Accepted 2026-03-13 02:21:45,175 - lava_callback - INFO - Starting async job submit for node 69b370a1974c91e64560f56d 2026-03-13 02:21:45,227 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 02:21:45,242 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b370a1974c91e64560f56d HTTP/1.1" 200 None 2026-03-13 02:21:45,243 - lava_callback - DEBUG - Processing node 69b370a1974c91e64560f56d: baseline-arm 2026-03-13 02:21:45,830 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 02:21:45,946 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 02:21:45,948 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69b370a1974c91e64560f56d/log.txt.gz 2026-03-13 02:21:46,281 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:39114 - "GET / HTTP/1.0" 200 OK 2026-03-13 02:21:46,565 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 02:21:46,567 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69b370a1974c91e64560f56d/lava-logs.yaml 2026-03-13 02:21:46,582 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 02:21:46,752 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 02:21:46,754 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69b370a1974c91e64560f56d/lava_callback.json.gz DEBUG: Setup failed for 69b370a1974c91e64560f56d. Transitting job node result: incomplete -> fail 2026-03-13 02:21:46,864 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 02:21:46,878 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b370a1974c91e64560f56d HTTP/1.1" 200 None 2026-03-13 02:21:46,879 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 02:21:46,893 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35934974c91e64560e49d HTTP/1.1" 200 None 2026-03-13 02:21:46,894 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 02:21:46,929 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b370a1974c91e64560f56d HTTP/1.1" 200 None 2026-03-13 02:21:47,034 - lava_callback - INFO - Completed processing callback for node 69b370a1974c91e64560f56d INFO: 172.20.0.1:51484 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35014 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35020 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42764 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-13 02:22:15,927 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 02:22:15,943 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:42770 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51238 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45498 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53128 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49046 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37630 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37634 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53990 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54000 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48386 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51210 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57692 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34986 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59720 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59732 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46726 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46728 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56802 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40578 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54612 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39826 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51048 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51058 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56876 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56884 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44310 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44398 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58014 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55878 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52654 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49200 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49214 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49216 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53738 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36234 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43834 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47844 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42834 - "GET /node/69944f545ae459b9df3dcee1%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:42836 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56948 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56960 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56968 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51812 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55652 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54720 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33170 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48638 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59798 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59806 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59808 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58710 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43368 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53424 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39184 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43742 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45384 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45388 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45394 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38532 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55980 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52824 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34860 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56260 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47746 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47754 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47764 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53734 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50190 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49864 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46880 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59836 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46928 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46942 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46956 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36754 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57252 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51184 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45214 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38190 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42728 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42734 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42748 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38378 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50498 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38232 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59508 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48414 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56058 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56070 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56078 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42088 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54640 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39390 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41500 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51532 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38188 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38204 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38218 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43864 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39504 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56950 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43974 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57914 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52044 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52058 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52070 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59764 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43754 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39602 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49022 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60508 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47862 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47872 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47876 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33470 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40908 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55854 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48898 - "GET / HTTP/1.0" 200 OK 2026-03-13 02:37:00,590 - lava_callback - INFO - Received callback for node 69b37684974c91e64560f702 INFO: 172.20.0.1:34370 - "POST /node/69b37684974c91e64560f702 HTTP/1.0" 202 Accepted 2026-03-13 02:37:00,597 - lava_callback - INFO - Starting async job submit for node 69b37684974c91e64560f702 2026-03-13 02:37:00,632 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 02:37:00,647 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b37684974c91e64560f702 HTTP/1.1" 200 None 2026-03-13 02:37:00,648 - lava_callback - DEBUG - Processing node 69b37684974c91e64560f702: baseline-arm64 2026-03-13 02:37:01,027 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 02:37:01,195 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 02:37:01,197 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69b37684974c91e64560f702/log.txt.gz 2026-03-13 02:37:01,404 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 02:37:01,638 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 02:37:01,641 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69b37684974c91e64560f702/lava-logs.yaml 2026-03-13 02:37:01,650 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 02:37:01,845 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 02:37:01,847 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69b37684974c91e64560f702/lava_callback.json.gz DEBUG: Setup failed for 69b37684974c91e64560f702. Transitting job node result: incomplete -> fail 2026-03-13 02:37:01,920 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 02:37:01,954 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b37684974c91e64560f702 HTTP/1.1" 200 None 2026-03-13 02:37:01,955 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 02:37:01,979 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3594d974c91e64560e4b2 HTTP/1.1" 200 None 2026-03-13 02:37:01,981 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 02:37:02,020 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b37684974c91e64560f702 HTTP/1.1" 200 None 2026-03-13 02:37:02,107 - lava_callback - INFO - Completed processing callback for node 69b37684974c91e64560f702 INFO: 172.20.0.1:34376 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47382 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47392 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-13 02:37:15,948 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 02:37:15,964 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:47398 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35056 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36550 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54292 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55452 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49156 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59044 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59056 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59068 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54390 - "GET / HTTP/1.0" 200 OK 2026-03-13 02:38:29,952 - lava_callback - INFO - Received callback for node 69b36301974c91e64560ec82 INFO: 172.20.0.1:60370 - "POST /node/69b36301974c91e64560ec82 HTTP/1.0" 202 Accepted 2026-03-13 02:38:29,959 - lava_callback - INFO - Starting async job submit for node 69b36301974c91e64560ec82 2026-03-13 02:38:30,017 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 02:38:30,046 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b36301974c91e64560ec82 HTTP/1.1" 200 None 2026-03-13 02:38:30,047 - lava_callback - DEBUG - Processing node 69b36301974c91e64560ec82: ltp-smoke 2026-03-13 02:38:30,516 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 02:38:30,678 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 02:38:30,679 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoke-69b36301974c91e64560ec82/log.txt.gz 2026-03-13 02:38:30,965 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 02:38:31,197 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 02:38:31,200 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-smoke-69b36301974c91e64560ec82/lava-logs.yaml 2026-03-13 02:38:31,211 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 02:38:31,404 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 02:38:31,406 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoke-69b36301974c91e64560ec82/lava_callback.json.gz 2026-03-13 02:38:31,526 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 02:38:31,541 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b36301974c91e64560ec82 HTTP/1.1" 200 None 2026-03-13 02:38:31,542 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 02:38:31,565 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3593e974c91e64560e4a3 HTTP/1.1" 200 None 2026-03-13 02:38:31,567 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 02:38:31,596 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b36301974c91e64560ec82 HTTP/1.1" 200 None 2026-03-13 02:38:31,705 - lava_callback - INFO - Completed processing callback for node 69b36301974c91e64560ec82 INFO: 172.20.0.1:60384 - "GET / HTTP/1.0" 200 OK 2026-03-13 02:38:45,966 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 02:38:45,983 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:34656 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35886 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42150 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58144 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58146 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58152 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44988 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42632 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56442 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50882 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58050 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36106 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36110 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36114 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38876 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43722 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60934 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52662 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59918 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35810 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35814 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35826 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37476 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42672 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54814 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57706 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54722 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60726 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60740 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60750 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47092 - "GET / HTTP/1.0" 200 OK 2026-03-13 02:42:30,598 - lava_callback - INFO - Received callback for node 69b363cd974c91e64560ed64 INFO: 172.20.0.1:56202 - "POST /node/69b363cd974c91e64560ed64 HTTP/1.0" 202 Accepted 2026-03-13 02:42:30,604 - lava_callback - INFO - Starting async job submit for node 69b363cd974c91e64560ed64 2026-03-13 02:42:30,627 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 02:42:30,649 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b363cd974c91e64560ed64 HTTP/1.1" 200 None 2026-03-13 02:42:30,649 - lava_callback - DEBUG - Processing node 69b363cd974c91e64560ed64: baseline-arm64-broonie 2026-03-13 02:42:30,906 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 02:42:31,058 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 02:42:31,060 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69b363cd974c91e64560ed64/log.txt.gz 2026-03-13 02:42:31,216 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 02:42:31,421 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 02:42:31,423 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69b363cd974c91e64560ed64/lava-logs.yaml 2026-03-13 02:42:31,430 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 02:42:31,606 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 02:42:31,608 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69b363cd974c91e64560ed64/lava_callback.json.gz 2026-03-13 02:42:31,631 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 02:42:31,645 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b363cd974c91e64560ed64 HTTP/1.1" 200 None 2026-03-13 02:42:31,647 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 02:42:31,660 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3594d974c91e64560e4b2 HTTP/1.1" 200 None 2026-03-13 02:42:31,662 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 02:42:31,692 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b363cd974c91e64560ed64 HTTP/1.1" 200 None 2026-03-13 02:42:31,693 - lava_callback - INFO - Completed processing callback for node 69b363cd974c91e64560ed64 INFO: 172.20.0.1:56214 - "GET / HTTP/1.0" 200 OK 2026-03-13 02:42:45,986 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 02:42:46,002 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:57414 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35458 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44666 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43856 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43858 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43870 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40684 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46786 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44032 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58562 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39756 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58928 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58940 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58948 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40014 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50472 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34168 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44380 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51198 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52084 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52098 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52106 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41832 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55072 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49226 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51594 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58106 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35766 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35780 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35792 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47762 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58978 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34076 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41436 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54440 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49386 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49390 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49394 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47930 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36942 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42566 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54498 - "GET / HTTP/1.0" 200 OK 2026-03-13 02:48:02,571 - lava_callback - INFO - Received callback for node 69b376e1974c91e64560f716 INFO: 172.20.0.1:57582 - "POST /node/69b376e1974c91e64560f716 HTTP/1.0" 202 Accepted 2026-03-13 02:48:02,578 - lava_callback - INFO - Starting async job submit for node 69b376e1974c91e64560f716 2026-03-13 02:48:02,630 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 02:48:02,645 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b376e1974c91e64560f716 HTTP/1.1" 200 None 2026-03-13 02:48:02,646 - lava_callback - DEBUG - Processing node 69b376e1974c91e64560f716: baseline-arm 2026-03-13 02:48:03,235 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 02:48:03,392 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 02:48:03,394 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69b376e1974c91e64560f716/log.txt.gz 2026-03-13 02:48:03,742 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 02:48:03,988 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 02:48:03,990 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69b376e1974c91e64560f716/lava-logs.yaml 2026-03-13 02:48:04,005 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 02:48:04,194 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 02:48:04,195 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69b376e1974c91e64560f716/lava_callback.json.gz DEBUG: Setup failed for 69b376e1974c91e64560f716. Transitting job node result: incomplete -> fail 2026-03-13 02:48:04,302 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 02:48:04,321 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b376e1974c91e64560f716 HTTP/1.1" 200 None 2026-03-13 02:48:04,322 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 02:48:04,337 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35934974c91e64560e49d HTTP/1.1" 200 None 2026-03-13 02:48:04,338 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 02:48:04,363 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b376e1974c91e64560f716 HTTP/1.1" 200 None 2026-03-13 02:48:04,497 - lava_callback - INFO - Completed processing callback for node 69b376e1974c91e64560f716 INFO: 172.20.0.1:57590 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40342 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40352 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-13 02:48:16,005 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 02:48:16,021 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:40364 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47130 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49150 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54738 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52300 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60446 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39946 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39958 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39972 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33568 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54152 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60376 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42276 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55256 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48590 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48600 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48602 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41530 - "GET / HTTP/1.0" 200 OK 2026-03-13 02:50:32,732 - lava_callback - INFO - Received callback for node 69b0d8639d19175818252cd6 INFO: 172.20.0.1:34540 - "POST /node/69b0d8639d19175818252cd6 HTTP/1.0" 202 Accepted 2026-03-13 02:50:32,738 - lava_callback - INFO - Starting async job submit for node 69b0d8639d19175818252cd6 2026-03-13 02:50:32,738 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 02:50:32,761 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0d8639d19175818252cd6 HTTP/1.1" 200 None 2026-03-13 02:50:32,762 - lava_callback - DEBUG - Processing node 69b0d8639d19175818252cd6: baseline-arm64-baylibre 2026-03-13 02:50:32,762 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0d8639d19175818252cd6 2026-03-13 02:50:32,763 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 02:50:32,776 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0d8639d19175818252cd6 HTTP/1.1" 200 None 2026-03-13 02:50:32,778 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 02:50:32,792 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aafea227cb11dcdfc758f1 HTTP/1.1" 200 None 2026-03-13 02:50:32,793 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 02:50:32,838 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0d8639d19175818252cd6 HTTP/1.1" 200 None 2026-03-13 02:50:32,845 - lava_callback - ERROR - Error processing callback for node 69b0d8639d19175818252cd6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:34542 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56600 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41000 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54360 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36372 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36382 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36396 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45748 - "GET / HTTP/1.0" 200 OK 2026-03-13 02:51:32,700 - lava_callback - INFO - Received callback for node 69b0d8b19d19175818252ce6 INFO: 172.20.0.1:54566 - "POST /node/69b0d8b19d19175818252ce6 HTTP/1.0" 202 Accepted 2026-03-13 02:51:32,706 - lava_callback - INFO - Starting async job submit for node 69b0d8b19d19175818252ce6 2026-03-13 02:51:32,707 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 02:51:32,721 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0d8b19d19175818252ce6 HTTP/1.1" 200 None 2026-03-13 02:51:32,721 - lava_callback - DEBUG - Processing node 69b0d8b19d19175818252ce6: baseline-arm64-baylibre 2026-03-13 02:51:32,722 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0d8b19d19175818252ce6 2026-03-13 02:51:32,723 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 02:51:32,744 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0d8b19d19175818252ce6 HTTP/1.1" 200 None 2026-03-13 02:51:32,745 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 02:51:32,760 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aafea227cb11dcdfc758f1 HTTP/1.1" 200 None 2026-03-13 02:51:32,762 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 02:51:32,783 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0d8b19d19175818252ce6 HTTP/1.1" 200 None 2026-03-13 02:51:32,792 - lava_callback - ERROR - Error processing callback for node 69b0d8b19d19175818252ce6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:54582 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41756 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45400 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43990 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34348 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34362 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34378 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46754 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60624 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54278 - "GET / HTTP/1.0" 200 OK 2026-03-13 02:52:52,813 - lava_callback - INFO - Received callback for node 69b0d9019d19175818252ced INFO: 172.20.0.1:43028 - "POST /node/69b0d9019d19175818252ced HTTP/1.0" 202 Accepted 2026-03-13 02:52:52,819 - lava_callback - INFO - Starting async job submit for node 69b0d9019d19175818252ced 2026-03-13 02:52:52,819 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 02:52:52,835 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0d9019d19175818252ced HTTP/1.1" 200 None 2026-03-13 02:52:52,836 - lava_callback - DEBUG - Processing node 69b0d9019d19175818252ced: baseline-arm64-baylibre 2026-03-13 02:52:52,836 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0d9019d19175818252ced 2026-03-13 02:52:52,837 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 02:52:52,856 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0d9019d19175818252ced HTTP/1.1" 200 None 2026-03-13 02:52:52,857 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 02:52:52,871 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aafea227cb11dcdfc758f1 HTTP/1.1" 200 None 2026-03-13 02:52:52,872 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 02:52:52,893 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0d9019d19175818252ced HTTP/1.1" 200 None 2026-03-13 02:52:52,899 - lava_callback - ERROR - Error processing callback for node 69b0d9019d19175818252ced: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:43044 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38766 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49692 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49706 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49710 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34440 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53930 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52640 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52774 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44126 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43490 - "GET / HTTP/1.0" 200 OK 2026-03-13 02:54:12,873 - lava_callback - INFO - Received callback for node 69b0d94f9d19175818252cfe INFO: 172.20.0.1:43496 - "POST /node/69b0d94f9d19175818252cfe HTTP/1.0" 202 Accepted 2026-03-13 02:54:12,879 - lava_callback - INFO - Starting async job submit for node 69b0d94f9d19175818252cfe 2026-03-13 02:54:12,880 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 02:54:12,894 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0d94f9d19175818252cfe HTTP/1.1" 200 None 2026-03-13 02:54:12,895 - lava_callback - DEBUG - Processing node 69b0d94f9d19175818252cfe: baseline-arm64-baylibre 2026-03-13 02:54:12,895 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0d94f9d19175818252cfe 2026-03-13 02:54:12,896 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 02:54:12,909 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0d94f9d19175818252cfe HTTP/1.1" 200 None 2026-03-13 02:54:12,911 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 02:54:12,931 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aafea227cb11dcdfc758f1 HTTP/1.1" 200 None 2026-03-13 02:54:12,932 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 02:54:12,955 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0d94f9d19175818252cfe HTTP/1.1" 200 None 2026-03-13 02:54:12,956 - lava_callback - ERROR - Error processing callback for node 69b0d94f9d19175818252cfe: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:43502 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43516 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51482 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36284 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54446 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48840 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59002 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39062 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39064 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39068 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56858 - "GET / HTTP/1.0" 200 OK 2026-03-13 02:55:32,942 - lava_callback - INFO - Received callback for node 69b0d99d9d19175818252d0c INFO: 172.20.0.1:56708 - "POST /node/69b0d99d9d19175818252d0c HTTP/1.0" 202 Accepted 2026-03-13 02:55:32,947 - lava_callback - INFO - Starting async job submit for node 69b0d99d9d19175818252d0c 2026-03-13 02:55:32,948 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 02:55:32,962 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0d99d9d19175818252d0c HTTP/1.1" 200 None 2026-03-13 02:55:32,963 - lava_callback - DEBUG - Processing node 69b0d99d9d19175818252d0c: baseline-arm64-baylibre 2026-03-13 02:55:32,963 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0d99d9d19175818252d0c 2026-03-13 02:55:32,964 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 02:55:32,998 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0d99d9d19175818252d0c HTTP/1.1" 200 None 2026-03-13 02:55:32,999 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 02:55:33,012 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aafea227cb11dcdfc758f1 HTTP/1.1" 200 None 2026-03-13 02:55:33,014 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 02:55:33,033 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0d99d9d19175818252d0c HTTP/1.1" 200 None 2026-03-13 02:55:33,040 - lava_callback - ERROR - Error processing callback for node 69b0d99d9d19175818252d0c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:56718 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38580 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53716 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33568 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34532 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34548 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34562 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32794 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53866 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56266 - "GET / HTTP/1.0" 200 OK 2026-03-13 02:56:53,060 - lava_callback - INFO - Received callback for node 69b0d9eb9d19175818252d1d INFO: 172.20.0.1:56452 - "POST /node/69b0d9eb9d19175818252d1d HTTP/1.0" 202 Accepted 2026-03-13 02:56:53,066 - lava_callback - INFO - Starting async job submit for node 69b0d9eb9d19175818252d1d 2026-03-13 02:56:53,067 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 02:56:53,097 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0d9eb9d19175818252d1d HTTP/1.1" 200 None 2026-03-13 02:56:53,098 - lava_callback - DEBUG - Processing node 69b0d9eb9d19175818252d1d: baseline-arm64-baylibre 2026-03-13 02:56:53,098 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0d9eb9d19175818252d1d 2026-03-13 02:56:53,099 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 02:56:53,133 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0d9eb9d19175818252d1d HTTP/1.1" 200 None 2026-03-13 02:56:53,134 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 02:56:53,166 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aafea227cb11dcdfc758f1 HTTP/1.1" 200 None 2026-03-13 02:56:53,168 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 02:56:53,211 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0d9eb9d19175818252d1d HTTP/1.1" 200 None 2026-03-13 02:56:53,212 - lava_callback - ERROR - Error processing callback for node 69b0d9eb9d19175818252d1d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:56458 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60564 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46774 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46790 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46798 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35496 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35198 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54496 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57284 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36310 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46644 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46648 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46656 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35132 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35136 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58558 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48076 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52672 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51646 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51658 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51674 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46870 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56368 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45648 - "GET / HTTP/1.0" 200 OK 2026-03-13 02:59:50,013 - lava_callback - INFO - Received callback for node 69b0da959d19175818252d3f INFO: 172.20.0.1:57070 - "POST /node/69b0da959d19175818252d3f HTTP/1.0" 202 Accepted 2026-03-13 02:59:50,018 - lava_callback - INFO - Starting async job submit for node 69b0da959d19175818252d3f 2026-03-13 02:59:50,019 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 02:59:50,034 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0da959d19175818252d3f HTTP/1.1" 200 None 2026-03-13 02:59:50,035 - lava_callback - DEBUG - Processing node 69b0da959d19175818252d3f: baseline-arm64-clabbe 2026-03-13 02:59:50,035 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0da959d19175818252d3f 2026-03-13 02:59:50,035 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 02:59:50,048 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0da959d19175818252d3f HTTP/1.1" 200 None 2026-03-13 02:59:50,049 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 02:59:50,071 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aafea227cb11dcdfc758f1 HTTP/1.1" 200 None 2026-03-13 02:59:50,073 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 02:59:50,139 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0da959d19175818252d3f HTTP/1.1" 200 None 2026-03-13 02:59:50,140 - lava_callback - ERROR - Error processing callback for node 69b0da959d19175818252d3f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:57082 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39900 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53438 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53446 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53454 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51806 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33630 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33740 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58710 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33382 - "GET / HTTP/1.0" 200 OK 2026-03-13 03:01:10,064 - lava_callback - INFO - Received callback for node 69b0dae39d19175818252d49 INFO: 172.20.0.1:53888 - "POST /node/69b0dae39d19175818252d49 HTTP/1.0" 202 Accepted 2026-03-13 03:01:10,070 - lava_callback - INFO - Starting async job submit for node 69b0dae39d19175818252d49 2026-03-13 03:01:10,071 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:01:10,096 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0dae39d19175818252d49 HTTP/1.1" 200 None 2026-03-13 03:01:10,097 - lava_callback - DEBUG - Processing node 69b0dae39d19175818252d49: baseline-arm64-clabbe 2026-03-13 03:01:10,097 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0dae39d19175818252d49 2026-03-13 03:01:10,098 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:01:10,111 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0dae39d19175818252d49 HTTP/1.1" 200 None 2026-03-13 03:01:10,113 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:01:10,138 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aafea227cb11dcdfc758f1 HTTP/1.1" 200 None 2026-03-13 03:01:10,139 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:01:10,159 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0dae39d19175818252d49 HTTP/1.1" 200 None 2026-03-13 03:01:10,173 - lava_callback - ERROR - Error processing callback for node 69b0dae39d19175818252d49: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:53896 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53904 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53914 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40882 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37752 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44340 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37978 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58466 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44910 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44922 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44930 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40268 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49070 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55052 - "GET / HTTP/1.0" 200 OK 2026-03-13 03:02:49,735 - lava_callback - INFO - Received callback for node 69b3718b974c91e64560f5b8 INFO: 172.20.0.1:45276 - "POST /node/69b3718b974c91e64560f5b8 HTTP/1.0" 202 Accepted 2026-03-13 03:02:49,743 - lava_callback - INFO - Starting async job submit for node 69b3718b974c91e64560f5b8 2026-03-13 03:02:49,785 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:02:49,800 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3718b974c91e64560f5b8 HTTP/1.1" 200 None 2026-03-13 03:02:49,801 - lava_callback - DEBUG - Processing node 69b3718b974c91e64560f5b8: baseline-nfs-arm64 2026-03-13 03:02:50,890 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 03:02:51,086 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 03:02:51,088 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b3718b974c91e64560f5b8/log.txt.gz 2026-03-13 03:02:51,721 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 03:02:51,962 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 03:02:51,965 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b3718b974c91e64560f5b8/lava-logs.yaml 2026-03-13 03:02:51,986 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 03:02:52,174 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 03:02:52,176 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b3718b974c91e64560f5b8/lava_callback.json.gz DEBUG: Setup failed for 69b3718b974c91e64560f5b8. Transitting job node result: incomplete -> fail 2026-03-13 03:02:52,264 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:02:52,286 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3718b974c91e64560f5b8 HTTP/1.1" 200 None 2026-03-13 03:02:52,287 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:02:52,319 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3594d974c91e64560e4b2 HTTP/1.1" 200 None 2026-03-13 03:02:52,321 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:02:52,344 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b3718b974c91e64560f5b8 HTTP/1.1" 200 None 2026-03-13 03:02:52,464 - lava_callback - INFO - Completed processing callback for node 69b3718b974c91e64560f5b8 INFO: 172.20.0.1:45292 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33978 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34366 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34374 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-13 03:03:16,026 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:03:16,046 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:34386 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41032 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47680 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46764 - "GET / HTTP/1.0" 200 OK 2026-03-13 03:03:50,125 - lava_callback - INFO - Received callback for node 69b0db839d19175818252d6a INFO: 172.20.0.1:59478 - "POST /node/69b0db839d19175818252d6a HTTP/1.0" 202 Accepted 2026-03-13 03:03:50,131 - lava_callback - INFO - Starting async job submit for node 69b0db839d19175818252d6a 2026-03-13 03:03:50,132 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:03:50,159 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0db839d19175818252d6a HTTP/1.1" 200 None 2026-03-13 03:03:50,160 - lava_callback - DEBUG - Processing node 69b0db839d19175818252d6a: baseline-arm64-clabbe 2026-03-13 03:03:50,160 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0db839d19175818252d6a 2026-03-13 03:03:50,161 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:03:50,174 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0db839d19175818252d6a HTTP/1.1" 200 None 2026-03-13 03:03:50,175 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:03:50,188 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aafea227cb11dcdfc758f1 HTTP/1.1" 200 None 2026-03-13 03:03:50,189 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:03:50,213 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0db839d19175818252d6a HTTP/1.1" 200 None 2026-03-13 03:03:50,214 - lava_callback - ERROR - Error processing callback for node 69b0db839d19175818252d6a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:59490 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45898 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50978 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50988 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51000 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35968 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56670 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53548 - "GET / HTTP/1.0" 200 OK 2026-03-13 03:04:53,482 - lava_callback - INFO - Received callback for node 69b0dbcb9d19175818252d74 INFO: 172.20.0.1:57244 - "POST /node/69b0dbcb9d19175818252d74 HTTP/1.0" 202 Accepted 2026-03-13 03:04:53,487 - lava_callback - INFO - Starting async job submit for node 69b0dbcb9d19175818252d74 2026-03-13 03:04:53,488 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:04:53,502 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0dbcb9d19175818252d74 HTTP/1.1" 200 None 2026-03-13 03:04:53,503 - lava_callback - DEBUG - Processing node 69b0dbcb9d19175818252d74: baseline-arm64-baylibre 2026-03-13 03:04:53,503 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0dbcb9d19175818252d74 2026-03-13 03:04:53,504 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:04:53,518 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0dbcb9d19175818252d74 HTTP/1.1" 200 None 2026-03-13 03:04:53,520 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:04:53,533 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aafea227cb11dcdfc758f1 HTTP/1.1" 200 None 2026-03-13 03:04:53,534 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:04:53,555 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0dbcb9d19175818252d74 HTTP/1.1" 200 None 2026-03-13 03:04:53,562 - lava_callback - ERROR - Error processing callback for node 69b0dbcb9d19175818252d74: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:57252 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40542 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47280 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47292 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47298 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45250 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46694 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42328 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42800 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36276 - "GET / HTTP/1.0" 200 OK 2026-03-13 03:06:10,240 - lava_callback - INFO - Received callback for node 69b0dc1d9d19175818252d85 INFO: 172.20.0.1:34670 - "POST /node/69b0dc1d9d19175818252d85 HTTP/1.0" 202 Accepted 2026-03-13 03:06:10,246 - lava_callback - INFO - Starting async job submit for node 69b0dc1d9d19175818252d85 2026-03-13 03:06:10,247 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:06:10,267 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0dc1d9d19175818252d85 HTTP/1.1" 200 None 2026-03-13 03:06:10,268 - lava_callback - DEBUG - Processing node 69b0dc1d9d19175818252d85: baseline-arm64-clabbe 2026-03-13 03:06:10,268 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0dc1d9d19175818252d85 2026-03-13 03:06:10,269 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:06:10,282 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0dc1d9d19175818252d85 HTTP/1.1" 200 None 2026-03-13 03:06:10,283 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.20.0.1:34682 - "GET / HTTP/1.0" 200 OK 2026-03-13 03:06:10,311 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aafea227cb11dcdfc758f1 HTTP/1.1" 200 None 2026-03-13 03:06:10,312 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:06:10,333 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0dc1d9d19175818252d85 HTTP/1.1" 200 None 2026-03-13 03:06:10,334 - lava_callback - ERROR - Error processing callback for node 69b0dc1d9d19175818252d85: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:34692 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34694 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56378 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37284 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39452 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49242 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45514 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53694 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53708 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53716 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48582 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54058 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33972 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47132 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44288 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33960 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33968 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33970 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50850 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38804 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50418 - "GET / HTTP/1.0" 200 OK 2026-03-13 03:08:50,360 - lava_callback - INFO - Received callback for node 69b0dcb49d19175818252da3 INFO: 172.20.0.1:60748 - "POST /node/69b0dcb49d19175818252da3 HTTP/1.0" 202 Accepted 2026-03-13 03:08:50,366 - lava_callback - INFO - Starting async job submit for node 69b0dcb49d19175818252da3 2026-03-13 03:08:50,367 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:08:50,381 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0dcb49d19175818252da3 HTTP/1.1" 200 None 2026-03-13 03:08:50,382 - lava_callback - DEBUG - Processing node 69b0dcb49d19175818252da3: baseline-arm64-clabbe 2026-03-13 03:08:50,382 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0dcb49d19175818252da3 2026-03-13 03:08:50,383 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:08:50,396 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0dcb49d19175818252da3 HTTP/1.1" 200 None 2026-03-13 03:08:50,397 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:08:50,411 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aafea227cb11dcdfc758f1 HTTP/1.1" 200 None 2026-03-13 03:08:50,412 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:08:50,432 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0dcb49d19175818252da3 HTTP/1.1" 200 None 2026-03-13 03:08:50,434 - lava_callback - ERROR - Error processing callback for node 69b0dcb49d19175818252da3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:60758 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36682 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50894 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50896 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50904 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40452 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53158 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52360 - "GET / HTTP/1.0" 200 OK 2026-03-13 03:09:53,756 - lava_callback - INFO - Received callback for node 69b0dcfa9d19175818252dab INFO: 172.20.0.1:44846 - "POST /node/69b0dcfa9d19175818252dab HTTP/1.0" 202 Accepted 2026-03-13 03:09:53,762 - lava_callback - INFO - Starting async job submit for node 69b0dcfa9d19175818252dab 2026-03-13 03:09:53,763 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:09:53,785 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0dcfa9d19175818252dab HTTP/1.1" 200 None 2026-03-13 03:09:53,785 - lava_callback - DEBUG - Processing node 69b0dcfa9d19175818252dab: baseline-arm64-baylibre 2026-03-13 03:09:53,786 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0dcfa9d19175818252dab 2026-03-13 03:09:53,786 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:09:53,813 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0dcfa9d19175818252dab HTTP/1.1" 200 None 2026-03-13 03:09:53,814 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:09:53,828 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afd3b0c9209b146d8e1d77 HTTP/1.1" 200 None 2026-03-13 03:09:53,829 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:09:53,849 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0dcfa9d19175818252dab HTTP/1.1" 200 None 2026-03-13 03:09:53,850 - lava_callback - ERROR - Error processing callback for node 69b0dcfa9d19175818252dab: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:44862 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51890 - "GET / HTTP/1.0" 200 OK 2026-03-13 03:10:09,516 - lava_callback - INFO - Received callback for node 69b363d4974c91e64560ed70 INFO: 172.20.0.1:51900 - "POST /node/69b363d4974c91e64560ed70 HTTP/1.0" 202 Accepted 2026-03-13 03:10:09,522 - lava_callback - INFO - Starting async job submit for node 69b363d4974c91e64560ed70 2026-03-13 03:10:09,543 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:10:09,558 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b363d4974c91e64560ed70 HTTP/1.1" 200 None 2026-03-13 03:10:09,559 - lava_callback - DEBUG - Processing node 69b363d4974c91e64560ed70: baseline-arm64-broonie 2026-03-13 03:10:09,800 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 03:10:10,003 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 03:10:10,004 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69b363d4974c91e64560ed70/log.txt.gz 2026-03-13 03:10:10,158 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:36478 - "GET / HTTP/1.0" 200 OK 2026-03-13 03:10:10,371 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 03:10:10,373 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69b363d4974c91e64560ed70/lava-logs.yaml 2026-03-13 03:10:10,380 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 03:10:10,543 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 03:10:10,545 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69b363d4974c91e64560ed70/lava_callback.json.gz 2026-03-13 03:10:10,568 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:10:10,582 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b363d4974c91e64560ed70 HTTP/1.1" 200 None 2026-03-13 03:10:10,584 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:10:10,598 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3594d974c91e64560e4b2 HTTP/1.1" 200 None 2026-03-13 03:10:10,600 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:10:10,625 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b363d4974c91e64560ed70 HTTP/1.1" 200 None 2026-03-13 03:10:10,626 - lava_callback - INFO - Completed processing callback for node 69b363d4974c91e64560ed70 INFO: 172.20.0.1:36492 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-13 03:10:16,049 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:10:16,072 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:36502 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35534 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52964 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35760 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55612 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49094 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41360 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41366 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41378 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39424 - "GET / HTTP/1.0" 200 OK 2026-03-13 03:11:30,455 - lava_callback - INFO - Received callback for node 69b0dd4f9d19175818252dbb INFO: 172.20.0.1:41258 - "POST /node/69b0dd4f9d19175818252dbb HTTP/1.0" 202 Accepted 2026-03-13 03:11:30,461 - lava_callback - INFO - Starting async job submit for node 69b0dd4f9d19175818252dbb 2026-03-13 03:11:30,462 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:11:30,478 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0dd4f9d19175818252dbb HTTP/1.1" 200 None 2026-03-13 03:11:30,478 - lava_callback - DEBUG - Processing node 69b0dd4f9d19175818252dbb: baseline-arm64-clabbe 2026-03-13 03:11:30,479 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0dd4f9d19175818252dbb 2026-03-13 03:11:30,479 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:11:30,511 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0dd4f9d19175818252dbb HTTP/1.1" 200 None 2026-03-13 03:11:30,513 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:11:30,526 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aafea227cb11dcdfc758f1 HTTP/1.1" 200 None 2026-03-13 03:11:30,528 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:11:30,548 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0dd4f9d19175818252dbb HTTP/1.1" 200 None 2026-03-13 03:11:30,549 - lava_callback - ERROR - Error processing callback for node 69b0dd4f9d19175818252dbb: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:41274 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35820 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59476 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42056 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36728 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36744 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36748 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39444 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42146 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50640 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44396 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39280 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56128 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56144 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56152 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53942 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46372 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40136 - "GET / HTTP/1.0" 200 OK 2026-03-13 03:13:53,961 - lava_callback - INFO - Received callback for node 69b0dde39d19175818252dd5 INFO: 172.20.0.1:52426 - "POST /node/69b0dde39d19175818252dd5 HTTP/1.0" 202 Accepted 2026-03-13 03:13:53,966 - lava_callback - INFO - Starting async job submit for node 69b0dde39d19175818252dd5 2026-03-13 03:13:53,967 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:13:53,981 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0dde39d19175818252dd5 HTTP/1.1" 200 None 2026-03-13 03:13:53,982 - lava_callback - DEBUG - Processing node 69b0dde39d19175818252dd5: baseline-arm64-baylibre 2026-03-13 03:13:53,983 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0dde39d19175818252dd5 2026-03-13 03:13:53,983 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:13:54,006 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0dde39d19175818252dd5 HTTP/1.1" 200 None 2026-03-13 03:13:54,007 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:13:54,021 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afd3b0c9209b146d8e1d77 HTTP/1.1" 200 None 2026-03-13 03:13:54,022 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:13:54,070 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0dde39d19175818252dd5 HTTP/1.1" 200 None 2026-03-13 03:13:54,071 - lava_callback - ERROR - Error processing callback for node 69b0dde39d19175818252dd5: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:52430 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54254 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43994 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43996 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44006 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50130 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53908 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44492 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48108 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33682 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38280 - "GET / HTTP/1.0" 200 OK 2026-03-13 03:15:14,045 - lava_callback - INFO - Received callback for node 69b0de329d19175818252de0 INFO: 172.20.0.1:38290 - "POST /node/69b0de329d19175818252de0 HTTP/1.0" 202 Accepted 2026-03-13 03:15:14,050 - lava_callback - INFO - Starting async job submit for node 69b0de329d19175818252de0 2026-03-13 03:15:14,051 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:15:14,066 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0de329d19175818252de0 HTTP/1.1" 200 None 2026-03-13 03:15:14,067 - lava_callback - DEBUG - Processing node 69b0de329d19175818252de0: baseline-arm64-baylibre 2026-03-13 03:15:14,067 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0de329d19175818252de0 2026-03-13 03:15:14,068 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:15:14,103 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0de329d19175818252de0 HTTP/1.1" 200 None 2026-03-13 03:15:14,104 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:15:14,144 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afd3b0c9209b146d8e1d77 HTTP/1.1" 200 None 2026-03-13 03:15:14,146 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:15:14,170 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0de329d19175818252de0 HTTP/1.1" 200 None 2026-03-13 03:15:14,172 - lava_callback - ERROR - Error processing callback for node 69b0de329d19175818252de0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:38294 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38310 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57904 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41334 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36142 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56174 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40102 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38102 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38108 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38120 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52136 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34780 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38048 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35196 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40960 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55434 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55438 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55448 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52290 - "GET / HTTP/1.0" 200 OK 2026-03-13 03:17:34,187 - lava_callback - INFO - Received callback for node 69b0decb9d19175818252dff INFO: 172.20.0.1:56008 - "POST /node/69b0decb9d19175818252dff HTTP/1.0" 202 Accepted 2026-03-13 03:17:34,192 - lava_callback - INFO - Starting async job submit for node 69b0decb9d19175818252dff 2026-03-13 03:17:34,193 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:17:34,208 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0decb9d19175818252dff HTTP/1.1" 200 None 2026-03-13 03:17:34,209 - lava_callback - DEBUG - Processing node 69b0decb9d19175818252dff: baseline-arm64-baylibre 2026-03-13 03:17:34,209 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0decb9d19175818252dff 2026-03-13 03:17:34,210 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:17:34,232 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0decb9d19175818252dff HTTP/1.1" 200 None 2026-03-13 03:17:34,234 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:17:34,247 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afd3b0c9209b146d8e1d77 HTTP/1.1" 200 None 2026-03-13 03:17:34,248 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:17:34,269 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0decb9d19175818252dff HTTP/1.1" 200 None 2026-03-13 03:17:34,274 - lava_callback - ERROR - Error processing callback for node 69b0decb9d19175818252dff: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:56016 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39032 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59660 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55524 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39544 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39550 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39552 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46330 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49862 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51114 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33250 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49452 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43488 - "GET / HTTP/1.0" 200 OK 2026-03-13 03:19:10,804 - lava_callback - INFO - Received callback for node 69b0df1d9d19175818252e0a INFO: 172.20.0.1:43494 - "POST /node/69b0df1d9d19175818252e0a HTTP/1.0" 202 Accepted 2026-03-13 03:19:10,810 - lava_callback - INFO - Starting async job submit for node 69b0df1d9d19175818252e0a 2026-03-13 03:19:10,811 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:19:10,826 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0df1d9d19175818252e0a HTTP/1.1" 200 None 2026-03-13 03:19:10,827 - lava_callback - DEBUG - Processing node 69b0df1d9d19175818252e0a: baseline-arm64-clabbe 2026-03-13 03:19:10,827 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0df1d9d19175818252e0a 2026-03-13 03:19:10,828 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:19:10,871 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0df1d9d19175818252e0a HTTP/1.1" 200 None 2026-03-13 03:19:10,873 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:19:10,898 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afd3b0c9209b146d8e1d77 HTTP/1.1" 200 None 2026-03-13 03:19:10,900 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:19:10,929 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0df1d9d19175818252e0a HTTP/1.1" 200 None 2026-03-13 03:19:10,931 - lava_callback - ERROR - Error processing callback for node 69b0df1d9d19175818252e0a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:43500 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43502 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57550 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32790 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56012 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52704 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40598 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47082 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47084 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47092 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58674 - "GET / HTTP/1.0" 200 OK 2026-03-13 03:20:30,820 - lava_callback - INFO - Received callback for node 69b0df699d19175818252e1c INFO: 172.20.0.1:42534 - "POST /node/69b0df699d19175818252e1c HTTP/1.0" 202 Accepted 2026-03-13 03:20:30,825 - lava_callback - INFO - Starting async job submit for node 69b0df699d19175818252e1c 2026-03-13 03:20:30,826 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:20:30,840 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0df699d19175818252e1c HTTP/1.1" 200 None 2026-03-13 03:20:30,841 - lava_callback - DEBUG - Processing node 69b0df699d19175818252e1c: baseline-arm64-clabbe 2026-03-13 03:20:30,841 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0df699d19175818252e1c 2026-03-13 03:20:30,842 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:20:30,854 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0df699d19175818252e1c HTTP/1.1" 200 None 2026-03-13 03:20:30,856 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:20:30,869 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afd3b0c9209b146d8e1d77 HTTP/1.1" 200 None 2026-03-13 03:20:30,870 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:20:30,891 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0df699d19175818252e1c HTTP/1.1" 200 None 2026-03-13 03:20:30,897 - lava_callback - ERROR - Error processing callback for node 69b0df699d19175818252e1c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:42546 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49790 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54772 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34350 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36444 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36460 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36474 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41338 - "GET / HTTP/1.0" 200 OK 2026-03-13 03:21:30,872 - lava_callback - INFO - Received callback for node 69b0dfb69d19175818252e2e INFO: 172.20.0.1:51042 - "POST /node/69b0dfb69d19175818252e2e HTTP/1.0" 202 Accepted 2026-03-13 03:21:30,877 - lava_callback - INFO - Starting async job submit for node 69b0dfb69d19175818252e2e 2026-03-13 03:21:30,879 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:21:30,893 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0dfb69d19175818252e2e HTTP/1.1" 200 None 2026-03-13 03:21:30,894 - lava_callback - DEBUG - Processing node 69b0dfb69d19175818252e2e: baseline-arm64-clabbe 2026-03-13 03:21:30,894 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0dfb69d19175818252e2e 2026-03-13 03:21:30,895 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:21:30,908 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0dfb69d19175818252e2e HTTP/1.1" 200 None 2026-03-13 03:21:30,910 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:21:30,946 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afd3b0c9209b146d8e1d77 HTTP/1.1" 200 None 2026-03-13 03:21:30,948 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:21:30,968 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0dfb69d19175818252e2e HTTP/1.1" 200 None 2026-03-13 03:21:30,970 - lava_callback - ERROR - Error processing callback for node 69b0dfb69d19175818252e2e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:51054 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58886 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54016 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35434 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38376 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38388 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38390 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51230 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53736 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59006 - "GET / HTTP/1.0" 200 OK 2026-03-13 03:22:50,925 - lava_callback - INFO - Received callback for node 69b0e0059d19175818252e3f INFO: 172.20.0.1:42418 - "POST /node/69b0e0059d19175818252e3f HTTP/1.0" 202 Accepted 2026-03-13 03:22:50,930 - lava_callback - INFO - Starting async job submit for node 69b0e0059d19175818252e3f 2026-03-13 03:22:50,931 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:22:50,965 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0e0059d19175818252e3f HTTP/1.1" 200 None 2026-03-13 03:22:50,966 - lava_callback - DEBUG - Processing node 69b0e0059d19175818252e3f: baseline-arm64-clabbe 2026-03-13 03:22:50,966 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0e0059d19175818252e3f 2026-03-13 03:22:50,967 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:22:50,980 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0e0059d19175818252e3f HTTP/1.1" 200 None 2026-03-13 03:22:50,981 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:22:51,016 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afd3b0c9209b146d8e1d77 HTTP/1.1" 200 None 2026-03-13 03:22:51,018 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:22:51,068 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0e0059d19175818252e3f HTTP/1.1" 200 None 2026-03-13 03:22:51,070 - lava_callback - ERROR - Error processing callback for node 69b0e0059d19175818252e3f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:42434 - "GET / HTTP/1.0" 200 OK 2026-03-13 03:23:06,466 - lava_callback - INFO - Received callback for node 69b37e65974c91e64560f80b INFO: 172.20.0.1:60240 - "POST /node/69b37e65974c91e64560f80b HTTP/1.0" 202 Accepted 2026-03-13 03:23:06,474 - lava_callback - INFO - Starting async job submit for node 69b37e65974c91e64560f80b INFO: 172.20.0.1:60250 - "GET / HTTP/1.0" 200 OK 2026-03-13 03:23:06,520 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:23:06,534 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b37e65974c91e64560f80b HTTP/1.1" 200 None 2026-03-13 03:23:06,535 - lava_callback - DEBUG - Processing node 69b37e65974c91e64560f80b: baseline-nfs-arm64 2026-03-13 03:23:07,611 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 03:23:07,809 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 03:23:07,811 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b37e65974c91e64560f80b/log.txt.gz 2026-03-13 03:23:08,457 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 03:23:08,696 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 03:23:08,698 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b37e65974c91e64560f80b/lava-logs.yaml 2026-03-13 03:23:08,720 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 03:23:08,927 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 03:23:08,929 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b37e65974c91e64560f80b/lava_callback.json.gz DEBUG: Setup failed for 69b37e65974c91e64560f80b. Transitting job node result: incomplete -> fail 2026-03-13 03:23:09,019 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:23:09,033 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b37e65974c91e64560f80b HTTP/1.1" 200 None 2026-03-13 03:23:09,034 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:23:09,047 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3594d974c91e64560e4b2 HTTP/1.1" 200 None 2026-03-13 03:23:09,048 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:23:09,072 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b37e65974c91e64560f80b HTTP/1.1" 200 None 2026-03-13 03:23:09,168 - lava_callback - INFO - Completed processing callback for node 69b37e65974c91e64560f80b INFO: 172.20.0.1:45646 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45660 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-13 03:23:16,077 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:23:16,100 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:45676 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49024 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58732 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56592 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47938 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41676 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52628 - "GET / HTTP/1.0" 200 OK 2026-03-13 03:24:10,968 - lava_callback - INFO - Received callback for node 69b0e0529d19175818252e56 INFO: 172.20.0.1:52632 - "POST /node/69b0e0529d19175818252e56 HTTP/1.0" 202 Accepted 2026-03-13 03:24:10,973 - lava_callback - INFO - Starting async job submit for node 69b0e0529d19175818252e56 2026-03-13 03:24:10,974 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:24:10,987 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0e0529d19175818252e56 HTTP/1.1" 200 None 2026-03-13 03:24:10,988 - lava_callback - DEBUG - Processing node 69b0e0529d19175818252e56: baseline-arm64-clabbe 2026-03-13 03:24:10,988 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0e0529d19175818252e56 2026-03-13 03:24:10,989 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:24:11,002 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0e0529d19175818252e56 HTTP/1.1" 200 None 2026-03-13 03:24:11,004 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:24:11,019 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afd3b0c9209b146d8e1d77 HTTP/1.1" 200 None 2026-03-13 03:24:11,020 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:24:11,041 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0e0529d19175818252e56 HTTP/1.1" 200 None 2026-03-13 03:24:11,049 - lava_callback - ERROR - Error processing callback for node 69b0e0529d19175818252e56: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:52638 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52654 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50864 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45678 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38960 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40304 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35006 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45818 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45824 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45838 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35948 - "GET / HTTP/1.0" 200 OK 2026-03-13 03:25:31,022 - lava_callback - INFO - Received callback for node 69b0e0a19d19175818252e5f INFO: 172.20.0.1:46718 - "POST /node/69b0e0a19d19175818252e5f HTTP/1.0" 202 Accepted 2026-03-13 03:25:31,028 - lava_callback - INFO - Starting async job submit for node 69b0e0a19d19175818252e5f 2026-03-13 03:25:31,028 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:25:31,042 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0e0a19d19175818252e5f HTTP/1.1" 200 None 2026-03-13 03:25:31,043 - lava_callback - DEBUG - Processing node 69b0e0a19d19175818252e5f: baseline-arm64-clabbe 2026-03-13 03:25:31,044 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0e0a19d19175818252e5f 2026-03-13 03:25:31,044 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:25:31,057 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0e0a19d19175818252e5f HTTP/1.1" 200 None 2026-03-13 03:25:31,058 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:25:31,071 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afd3b0c9209b146d8e1d77 HTTP/1.1" 200 None 2026-03-13 03:25:31,073 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:25:31,099 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0e0a19d19175818252e5f HTTP/1.1" 200 None 2026-03-13 03:25:31,101 - lava_callback - ERROR - Error processing callback for node 69b0e0a19d19175818252e5f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:46734 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44998 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36584 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50466 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48570 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48582 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48584 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36556 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37572 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45498 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49076 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46404 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42674 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42684 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42698 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48722 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50296 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51924 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44948 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46566 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53762 - "GET / HTTP/1.0" 200 OK 2026-03-13 03:28:14,741 - lava_callback - INFO - Received callback for node 69b0e1449d19175818252e93 INFO: 172.20.0.1:53778 - "POST /node/69b0e1449d19175818252e93 HTTP/1.0" 202 Accepted 2026-03-13 03:28:14,747 - lava_callback - INFO - Starting async job submit for node 69b0e1449d19175818252e93 2026-03-13 03:28:14,748 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:28:14,806 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0e1449d19175818252e93 HTTP/1.1" 200 None 2026-03-13 03:28:14,807 - lava_callback - DEBUG - Processing node 69b0e1449d19175818252e93: baseline-arm-baylibre 2026-03-13 03:28:14,807 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0e1449d19175818252e93 2026-03-13 03:28:14,808 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:28:14,827 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0e1449d19175818252e93 HTTP/1.1" 200 None 2026-03-13 03:28:14,829 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:28:14,842 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afd3a3c9209b146d8e1d68 HTTP/1.1" 200 None 2026-03-13 03:28:14,843 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:28:14,869 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0e1449d19175818252e93 HTTP/1.1" 200 None 2026-03-13 03:28:14,871 - lava_callback - ERROR - Error processing callback for node 69b0e1449d19175818252e93: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:53784 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53800 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55078 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34462 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34918 - "GET / HTTP/1.0" 200 OK 2026-03-13 03:28:54,721 - lava_callback - INFO - Received callback for node 69b0e1689d19175818252e9f INFO: 172.20.0.1:57642 - "POST /node/69b0e1689d19175818252e9f HTTP/1.0" 202 Accepted 2026-03-13 03:28:54,726 - lava_callback - INFO - Starting async job submit for node 69b0e1689d19175818252e9f 2026-03-13 03:28:54,727 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:28:54,764 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0e1689d19175818252e9f HTTP/1.1" 200 None 2026-03-13 03:28:54,765 - lava_callback - DEBUG - Processing node 69b0e1689d19175818252e9f: baseline-arm-baylibre 2026-03-13 03:28:54,765 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0e1689d19175818252e9f 2026-03-13 03:28:54,766 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:28:54,782 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0e1689d19175818252e9f HTTP/1.1" 200 None 2026-03-13 03:28:54,783 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:28:54,803 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afd3a3c9209b146d8e1d68 HTTP/1.1" 200 None 2026-03-13 03:28:54,804 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:28:54,825 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0e1689d19175818252e9f HTTP/1.1" 200 None 2026-03-13 03:28:54,826 - lava_callback - ERROR - Error processing callback for node 69b0e1689d19175818252e9f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:57646 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55452 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36296 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36310 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36316 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51458 - "GET / HTTP/1.0" 200 OK 2026-03-13 03:29:34,746 - lava_callback - INFO - Received callback for node 69b0e18c9d19175818252eab INFO: 172.20.0.1:49206 - "POST /node/69b0e18c9d19175818252eab HTTP/1.0" 202 Accepted 2026-03-13 03:29:34,752 - lava_callback - INFO - Starting async job submit for node 69b0e18c9d19175818252eab 2026-03-13 03:29:34,753 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:29:34,769 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0e18c9d19175818252eab HTTP/1.1" 200 None 2026-03-13 03:29:34,769 - lava_callback - DEBUG - Processing node 69b0e18c9d19175818252eab: baseline-arm-baylibre 2026-03-13 03:29:34,770 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0e18c9d19175818252eab 2026-03-13 03:29:34,770 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:29:34,783 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0e18c9d19175818252eab HTTP/1.1" 200 None 2026-03-13 03:29:34,784 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:29:34,797 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afd3a3c9209b146d8e1d68 HTTP/1.1" 200 None 2026-03-13 03:29:34,798 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:29:34,819 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0e18c9d19175818252eab HTTP/1.1" 200 None 2026-03-13 03:29:34,824 - lava_callback - ERROR - Error processing callback for node 69b0e18c9d19175818252eab: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:49220 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47530 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47234 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46988 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59374 - "GET / HTTP/1.0" 200 OK 2026-03-13 03:30:14,768 - lava_callback - INFO - Received callback for node 69b0e1b19d19175818252eb9 INFO: 172.20.0.1:59382 - "POST /node/69b0e1b19d19175818252eb9 HTTP/1.0" 202 Accepted 2026-03-13 03:30:14,773 - lava_callback - INFO - Starting async job submit for node 69b0e1b19d19175818252eb9 2026-03-13 03:30:14,774 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:30:14,803 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0e1b19d19175818252eb9 HTTP/1.1" 200 None 2026-03-13 03:30:14,804 - lava_callback - DEBUG - Processing node 69b0e1b19d19175818252eb9: baseline-arm-baylibre 2026-03-13 03:30:14,804 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0e1b19d19175818252eb9 2026-03-13 03:30:14,805 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:30:14,839 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0e1b19d19175818252eb9 HTTP/1.1" 200 None 2026-03-13 03:30:14,840 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:30:14,863 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afd3a3c9209b146d8e1d68 HTTP/1.1" 200 None 2026-03-13 03:30:14,865 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:30:14,894 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0e1b19d19175818252eb9 HTTP/1.1" 200 None 2026-03-13 03:30:14,895 - lava_callback - ERROR - Error processing callback for node 69b0e1b19d19175818252eb9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:59394 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59400 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35286 - "GET / HTTP/1.0" 200 OK 2026-03-13 03:30:34,808 - lava_callback - INFO - Received callback for node 69b0e1d59d19175818252ec8 INFO: 172.20.0.1:35628 - "POST /node/69b0e1d59d19175818252ec8 HTTP/1.0" 202 Accepted 2026-03-13 03:30:34,813 - lava_callback - INFO - Starting async job submit for node 69b0e1d59d19175818252ec8 2026-03-13 03:30:34,814 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:30:34,836 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0e1d59d19175818252ec8 HTTP/1.1" 200 None 2026-03-13 03:30:34,837 - lava_callback - DEBUG - Processing node 69b0e1d59d19175818252ec8: baseline-arm-baylibre 2026-03-13 03:30:34,837 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0e1d59d19175818252ec8 2026-03-13 03:30:34,838 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:30:34,871 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0e1d59d19175818252ec8 HTTP/1.1" 200 None 2026-03-13 03:30:34,872 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:30:34,885 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afd3a3c9209b146d8e1d68 HTTP/1.1" 200 None 2026-03-13 03:30:34,886 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:30:34,906 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0e1d59d19175818252ec8 HTTP/1.1" 200 None 2026-03-13 03:30:34,911 - lava_callback - ERROR - Error processing callback for node 69b0e1d59d19175818252ec8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:35638 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38912 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46330 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47228 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44732 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44742 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44758 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46494 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37388 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59924 - "GET / HTTP/1.0" 200 OK 2026-03-13 03:31:55,508 - lava_callback - INFO - Received callback for node 69b35ee2974c91e64560e9fd INFO: 172.20.0.1:42876 - "POST /node/69b35ee2974c91e64560e9fd HTTP/1.0" 202 Accepted 2026-03-13 03:31:55,514 - lava_callback - INFO - Starting async job submit for node 69b35ee2974c91e64560e9fd 2026-03-13 03:31:55,539 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:31:55,554 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35ee2974c91e64560e9fd HTTP/1.1" 200 None 2026-03-13 03:31:55,555 - lava_callback - DEBUG - Processing node 69b35ee2974c91e64560e9fd: ltp-watchqueue 2026-03-13 03:31:55,894 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 03:31:56,063 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 03:31:56,065 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-watchqueue-69b35ee2974c91e64560e9fd/log.txt.gz 2026-03-13 03:31:56,289 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:42884 - "GET / HTTP/1.0" 200 OK 2026-03-13 03:31:56,523 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 03:31:56,525 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-watchqueue-69b35ee2974c91e64560e9fd/lava-logs.yaml 2026-03-13 03:31:56,534 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 03:31:56,710 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 03:31:56,712 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-watchqueue-69b35ee2974c91e64560e9fd/lava_callback.json.gz 2026-03-13 03:31:56,738 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:31:56,755 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35ee2974c91e64560e9fd HTTP/1.1" 200 None 2026-03-13 03:31:56,757 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:31:56,771 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35934974c91e64560e49d HTTP/1.1" 200 None 2026-03-13 03:31:56,773 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:31:56,797 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b35ee2974c91e64560e9fd HTTP/1.1" 200 None 2026-03-13 03:31:56,798 - lava_callback - INFO - Completed processing callback for node 69b35ee2974c91e64560e9fd INFO: 172.20.0.1:46346 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57740 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57750 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-13 03:32:16,104 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:32:16,120 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:57754 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60342 - "GET / HTTP/1.0" 200 OK 2026-03-13 03:32:31,399 - lava_callback - INFO - Received callback for node 69b0e23f9d19175818252ef8 INFO: 172.20.0.1:52388 - "POST /node/69b0e23f9d19175818252ef8 HTTP/1.0" 202 Accepted 2026-03-13 03:32:31,407 - lava_callback - INFO - Starting async job submit for node 69b0e23f9d19175818252ef8 2026-03-13 03:32:31,409 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:32:31,426 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0e23f9d19175818252ef8 HTTP/1.1" 200 None 2026-03-13 03:32:31,426 - lava_callback - DEBUG - Processing node 69b0e23f9d19175818252ef8: baseline-arm-clabbe 2026-03-13 03:32:31,427 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0e23f9d19175818252ef8 2026-03-13 03:32:31,427 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:32:31,440 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0e23f9d19175818252ef8 HTTP/1.1" 200 None 2026-03-13 03:32:31,441 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:32:31,459 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afd3a3c9209b146d8e1d68 HTTP/1.1" 200 None 2026-03-13 03:32:31,460 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:32:31,481 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0e23f9d19175818252ef8 HTTP/1.1" 200 None 2026-03-13 03:32:31,482 - lava_callback - ERROR - Error processing callback for node 69b0e23f9d19175818252ef8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-13 03:32:35,019 - lava_callback - INFO - Received callback for node 69b35ee0974c91e64560e9fa INFO: 172.20.0.1:52390 - "POST /node/69b35ee0974c91e64560e9fa HTTP/1.0" 202 Accepted 2026-03-13 03:32:35,025 - lava_callback - INFO - Starting async job submit for node 69b35ee0974c91e64560e9fa 2026-03-13 03:32:35,050 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:32:35,071 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35ee0974c91e64560e9fa HTTP/1.1" 200 None 2026-03-13 03:32:35,071 - lava_callback - DEBUG - Processing node 69b35ee0974c91e64560e9fa: ltp-timers 2026-03-13 03:32:35,431 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 03:32:35,571 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 03:32:35,573 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-timers-69b35ee0974c91e64560e9fa/log.txt.gz 2026-03-13 03:32:35,790 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 03:32:35,973 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 03:32:35,975 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-timers-69b35ee0974c91e64560e9fa/lava-logs.yaml 2026-03-13 03:32:35,984 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 03:32:36,174 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 03:32:36,175 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-timers-69b35ee0974c91e64560e9fa/lava_callback.json.gz 2026-03-13 03:32:36,201 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:32:36,218 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35ee0974c91e64560e9fa HTTP/1.1" 200 None 2026-03-13 03:32:36,220 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:32:36,235 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35934974c91e64560e49d HTTP/1.1" 200 None 2026-03-13 03:32:36,237 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:32:36,261 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b35ee0974c91e64560e9fa HTTP/1.1" 200 None 2026-03-13 03:32:36,270 - lava_callback - INFO - Completed processing callback for node 69b35ee0974c91e64560e9fa INFO: 172.20.0.1:52404 - "GET / HTTP/1.0" 200 OK 2026-03-13 03:32:46,123 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:32:46,145 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:48146 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41580 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43730 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43186 - "GET / HTTP/1.0" 200 OK 2026-03-13 03:33:11,362 - lava_callback - INFO - Received callback for node 69b0e2679d19175818252f06 INFO: 172.20.0.1:43198 - "POST /node/69b0e2679d19175818252f06 HTTP/1.0" 202 Accepted 2026-03-13 03:33:11,367 - lava_callback - INFO - Starting async job submit for node 69b0e2679d19175818252f06 2026-03-13 03:33:11,368 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:33:11,385 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0e2679d19175818252f06 HTTP/1.1" 200 None 2026-03-13 03:33:11,386 - lava_callback - DEBUG - Processing node 69b0e2679d19175818252f06: baseline-arm-clabbe 2026-03-13 03:33:11,386 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0e2679d19175818252f06 2026-03-13 03:33:11,387 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:33:11,400 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0e2679d19175818252f06 HTTP/1.1" 200 None 2026-03-13 03:33:11,402 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:33:11,420 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afd3a3c9209b146d8e1d68 HTTP/1.1" 200 None 2026-03-13 03:33:11,421 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:33:11,454 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0e2679d19175818252f06 HTTP/1.1" 200 None 2026-03-13 03:33:11,456 - lava_callback - ERROR - Error processing callback for node 69b0e2679d19175818252f06: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:43206 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43218 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38518 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58922 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57288 - "GET / HTTP/1.0" 200 OK 2026-03-13 03:33:51,422 - lava_callback - INFO - Received callback for node 69b0e28c9d19175818252f1c INFO: 172.20.0.1:39184 - "POST /node/69b0e28c9d19175818252f1c HTTP/1.0" 202 Accepted 2026-03-13 03:33:51,427 - lava_callback - INFO - Starting async job submit for node 69b0e28c9d19175818252f1c 2026-03-13 03:33:51,428 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:33:51,443 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0e28c9d19175818252f1c HTTP/1.1" 200 None 2026-03-13 03:33:51,444 - lava_callback - DEBUG - Processing node 69b0e28c9d19175818252f1c: baseline-arm-clabbe 2026-03-13 03:33:51,444 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0e28c9d19175818252f1c 2026-03-13 03:33:51,445 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:33:51,458 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0e28c9d19175818252f1c HTTP/1.1" 200 None 2026-03-13 03:33:51,459 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:33:51,481 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afd3a3c9209b146d8e1d68 HTTP/1.1" 200 None 2026-03-13 03:33:51,483 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:33:51,509 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0e28c9d19175818252f1c HTTP/1.1" 200 None 2026-03-13 03:33:51,510 - lava_callback - ERROR - Error processing callback for node 69b0e28c9d19175818252f1c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:39200 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56968 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33470 - "GET / HTTP/1.0" 200 OK 2026-03-13 03:34:11,413 - lava_callback - INFO - Received callback for node 69b0e2b09d19175818252f25 INFO: 172.20.0.1:33484 - "POST /node/69b0e2b09d19175818252f25 HTTP/1.0" 202 Accepted 2026-03-13 03:34:11,419 - lava_callback - INFO - Starting async job submit for node 69b0e2b09d19175818252f25 2026-03-13 03:34:11,419 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:34:11,434 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0e2b09d19175818252f25 HTTP/1.1" 200 None 2026-03-13 03:34:11,435 - lava_callback - DEBUG - Processing node 69b0e2b09d19175818252f25: baseline-arm-clabbe 2026-03-13 03:34:11,435 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0e2b09d19175818252f25 2026-03-13 03:34:11,436 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:34:11,463 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0e2b09d19175818252f25 HTTP/1.1" 200 None 2026-03-13 03:34:11,464 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:34:11,478 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afd3a3c9209b146d8e1d68 HTTP/1.1" 200 None 2026-03-13 03:34:11,479 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:34:11,499 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0e2b09d19175818252f25 HTTP/1.1" 200 None 2026-03-13 03:34:11,500 - lava_callback - ERROR - Error processing callback for node 69b0e2b09d19175818252f25: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:33500 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33510 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34952 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39858 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47224 - "GET / HTTP/1.0" 200 OK 2026-03-13 03:34:51,456 - lava_callback - INFO - Received callback for node 69b0e2d49d19175818252f34 INFO: 172.20.0.1:40046 - "POST /node/69b0e2d49d19175818252f34 HTTP/1.0" 202 Accepted 2026-03-13 03:34:51,462 - lava_callback - INFO - Starting async job submit for node 69b0e2d49d19175818252f34 2026-03-13 03:34:51,463 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:34:51,487 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0e2d49d19175818252f34 HTTP/1.1" 200 None 2026-03-13 03:34:51,488 - lava_callback - DEBUG - Processing node 69b0e2d49d19175818252f34: baseline-arm-clabbe 2026-03-13 03:34:51,488 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0e2d49d19175818252f34 2026-03-13 03:34:51,488 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:34:51,501 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0e2d49d19175818252f34 HTTP/1.1" 200 None 2026-03-13 03:34:51,502 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:34:51,516 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afd3a3c9209b146d8e1d68 HTTP/1.1" 200 None 2026-03-13 03:34:51,518 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:34:51,558 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0e2d49d19175818252f34 HTTP/1.1" 200 None 2026-03-13 03:34:51,564 - lava_callback - ERROR - Error processing callback for node 69b0e2d49d19175818252f34: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:40062 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56566 - "GET / HTTP/1.0" 200 OK 2026-03-13 03:35:10,268 - lava_callback - INFO - Received callback for node 69b378a9974c91e64560f72f INFO: 172.20.0.1:53926 - "POST /node/69b378a9974c91e64560f72f HTTP/1.0" 202 Accepted 2026-03-13 03:35:10,274 - lava_callback - INFO - Starting async job submit for node 69b378a9974c91e64560f72f 2026-03-13 03:35:10,332 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:35:10,346 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b378a9974c91e64560f72f HTTP/1.1" 200 None 2026-03-13 03:35:10,347 - lava_callback - DEBUG - Processing node 69b378a9974c91e64560f72f: ltp-smoke INFO: 172.20.0.1:53936 - "GET / HTTP/1.0" 200 OK 2026-03-13 03:35:10,835 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 03:35:11,000 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 03:35:11,002 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoke-69b378a9974c91e64560f72f/log.txt.gz 2026-03-13 03:35:11,328 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 03:35:11,572 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 03:35:11,574 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-smoke-69b378a9974c91e64560f72f/lava-logs.yaml 2026-03-13 03:35:11,585 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 03:35:11,749 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 03:35:11,750 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoke-69b378a9974c91e64560f72f/lava_callback.json.gz 2026-03-13 03:35:11,869 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:35:11,884 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b378a9974c91e64560f72f HTTP/1.1" 200 None 2026-03-13 03:35:11,885 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:35:11,898 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3593e974c91e64560e4a3 HTTP/1.1" 200 None 2026-03-13 03:35:11,900 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:35:11,933 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b378a9974c91e64560f72f HTTP/1.1" 200 None 2026-03-13 03:35:12,062 - lava_callback - INFO - Completed processing callback for node 69b378a9974c91e64560f72f INFO: 172.20.0.1:53944 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-13 03:35:16,148 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:35:16,164 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:53956 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37092 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48566 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50156 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50188 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34450 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47274 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47290 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47292 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49622 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53090 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51354 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33926 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45126 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39638 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39646 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39658 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47116 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49736 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59122 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36974 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45852 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56870 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56884 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56896 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48076 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39770 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60416 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48282 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54722 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39864 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39866 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39874 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35724 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40784 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48682 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57688 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44032 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36338 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36354 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36370 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38816 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41912 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51872 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48408 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54008 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49404 - "GET / HTTP/1.0" 200 OK 2026-03-13 03:41:12,947 - lava_callback - INFO - Received callback for node 69b3684c974c91e64560f0d9 INFO: 172.20.0.1:49414 - "POST /node/69b3684c974c91e64560f0d9 HTTP/1.0" 202 Accepted 2026-03-13 03:41:12,993 - lava_callback - INFO - Starting async job submit for node 69b3684c974c91e64560f0d9 2026-03-13 03:41:13,017 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:41:13,040 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3684c974c91e64560f0d9 HTTP/1.1" 200 None 2026-03-13 03:41:13,041 - lava_callback - DEBUG - Processing node 69b3684c974c91e64560f0d9: baseline-arm-pengutronix 2026-03-13 03:41:13,151 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 03:41:13,315 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 03:41:13,317 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69b3684c974c91e64560f0d9/log.txt.gz 2026-03-13 03:41:13,374 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 03:41:13,557 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 03:41:13,559 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69b3684c974c91e64560f0d9/lava-logs.yaml 2026-03-13 03:41:13,562 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 03:41:13,701 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 03:41:13,703 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69b3684c974c91e64560f0d9/lava_callback.json.gz DEBUG: Setup failed for 69b3684c974c91e64560f0d9. Transitting job node result: incomplete -> fail 2026-03-13 03:41:13,752 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:41:13,789 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3684c974c91e64560f0d9 HTTP/1.1" 200 None 2026-03-13 03:41:13,791 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:41:13,804 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35934974c91e64560e49d HTTP/1.1" 200 None 2026-03-13 03:41:13,806 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:41:13,829 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b3684c974c91e64560f0d9 HTTP/1.1" 200 None 2026-03-13 03:41:13,935 - lava_callback - INFO - Completed processing callback for node 69b3684c974c91e64560f0d9 INFO: 172.20.0.1:49430 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-13 03:41:16,167 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:41:16,183 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:49432 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56452 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35420 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34830 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46264 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51458 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44500 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44506 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44518 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57858 - "GET / HTTP/1.0" 200 OK 2026-03-13 03:42:30,511 - lava_callback - INFO - Received callback for node 69b363cf974c91e64560ed67 INFO: 172.20.0.1:52578 - "POST /node/69b363cf974c91e64560ed67 HTTP/1.0" 202 Accepted 2026-03-13 03:42:30,517 - lava_callback - INFO - Starting async job submit for node 69b363cf974c91e64560ed67 2026-03-13 03:42:30,540 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:42:30,565 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b363cf974c91e64560ed67 HTTP/1.1" 200 None 2026-03-13 03:42:30,566 - lava_callback - DEBUG - Processing node 69b363cf974c91e64560ed67: baseline-arm64-broonie 2026-03-13 03:42:30,805 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 03:42:30,961 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 03:42:30,963 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69b363cf974c91e64560ed67/log.txt.gz 2026-03-13 03:42:31,108 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 03:42:31,318 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 03:42:31,320 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69b363cf974c91e64560ed67/lava-logs.yaml 2026-03-13 03:42:31,327 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 03:42:31,493 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 03:42:31,495 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69b363cf974c91e64560ed67/lava_callback.json.gz 2026-03-13 03:42:31,518 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:42:31,532 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b363cf974c91e64560ed67 HTTP/1.1" 200 None 2026-03-13 03:42:31,533 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:42:31,548 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3594d974c91e64560e4b2 HTTP/1.1" 200 None 2026-03-13 03:42:31,550 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:42:31,582 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b363cf974c91e64560ed67 HTTP/1.1" 200 None 2026-03-13 03:42:31,587 - lava_callback - INFO - Completed processing callback for node 69b363cf974c91e64560ed67 INFO: 172.20.0.1:52590 - "GET / HTTP/1.0" 200 OK 2026-03-13 03:42:46,186 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:42:46,227 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:45426 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42746 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59370 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35320 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35330 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35336 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57830 - "GET / HTTP/1.0" 200 OK 2026-03-13 03:43:26,601 - lava_callback - INFO - Received callback for node 69b38326974c91e64560f84f INFO: 172.20.0.1:57838 - "POST /node/69b38326974c91e64560f84f HTTP/1.0" 202 Accepted 2026-03-13 03:43:26,608 - lava_callback - INFO - Starting async job submit for node 69b38326974c91e64560f84f 2026-03-13 03:43:26,652 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:43:26,667 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b38326974c91e64560f84f HTTP/1.1" 200 None 2026-03-13 03:43:26,668 - lava_callback - DEBUG - Processing node 69b38326974c91e64560f84f: baseline-nfs-arm64 2026-03-13 03:43:27,736 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 03:43:27,932 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 03:43:27,934 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b38326974c91e64560f84f/log.txt.gz 2026-03-13 03:43:28,570 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 03:43:28,799 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 03:43:28,802 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b38326974c91e64560f84f/lava-logs.yaml 2026-03-13 03:43:28,824 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 03:43:28,993 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 03:43:28,995 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b38326974c91e64560f84f/lava_callback.json.gz DEBUG: Setup failed for 69b38326974c91e64560f84f. Transitting job node result: incomplete -> fail 2026-03-13 03:43:29,187 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:43:29,216 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b38326974c91e64560f84f HTTP/1.1" 200 None 2026-03-13 03:43:29,217 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:43:29,235 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3594d974c91e64560e4b2 HTTP/1.1" 200 None 2026-03-13 03:43:29,237 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:43:29,264 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b38326974c91e64560f84f HTTP/1.1" 200 None 2026-03-13 03:43:29,360 - lava_callback - INFO - Completed processing callback for node 69b38326974c91e64560f84f INFO: 172.20.0.1:35544 - "GET / HTTP/1.0" 200 OK 2026-03-13 03:43:46,230 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 03:43:46,246 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:34464 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57980 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55616 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42990 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42996 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43008 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42938 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58364 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50388 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48670 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43654 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44864 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44870 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44876 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37602 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49404 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45890 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38584 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33622 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35908 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35912 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35914 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57740 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40958 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36806 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56042 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39956 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46820 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46822 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46828 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44362 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44726 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41976 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46952 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55624 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55772 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55788 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55796 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60452 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59424 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53278 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44092 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60622 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47192 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47208 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47216 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51226 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44382 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46768 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47154 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48724 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39694 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39708 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39716 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52022 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33296 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57742 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58374 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48610 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41098 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41114 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41130 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59160 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35876 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35690 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50036 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47276 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35394 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35406 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35410 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50636 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32846 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59026 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56666 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40564 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49618 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49624 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49640 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43414 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49698 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51214 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44924 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50104 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57416 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57420 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57424 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34260 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56230 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34572 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54320 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56752 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43002 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43012 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43024 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39042 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41020 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40906 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54784 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39054 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35982 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35996 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36000 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41152 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57326 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47132 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49078 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49894 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38326 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38338 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38354 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34022 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36922 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53892 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48614 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50222 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39652 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39666 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39672 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53776 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43996 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46080 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43326 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60246 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32916 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32926 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:32936 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55256 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51380 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59922 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49224 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35860 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37756 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37760 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37764 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37940 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51598 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46292 - "GET / HTTP/1.0" 200 OK 2026-03-13 04:00:53,245 - lava_callback - INFO - Received callback for node 69b38767974c91e64560f8bc INFO: 172.20.0.1:40906 - "POST /node/69b38767974c91e64560f8bc HTTP/1.0" 202 Accepted 2026-03-13 04:00:53,293 - lava_callback - INFO - Starting async job submit for node 69b38767974c91e64560f8bc 2026-03-13 04:00:53,317 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:00:53,332 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b38767974c91e64560f8bc HTTP/1.1" 200 None 2026-03-13 04:00:53,333 - lava_callback - DEBUG - Processing node 69b38767974c91e64560f8bc: baseline-arm-pengutronix 2026-03-13 04:00:53,440 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 04:00:53,580 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 04:00:53,582 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69b38767974c91e64560f8bc/log.txt.gz 2026-03-13 04:00:53,641 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 04:00:53,837 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 04:00:53,839 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69b38767974c91e64560f8bc/lava-logs.yaml 2026-03-13 04:00:53,842 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 04:00:53,989 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 04:00:53,991 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69b38767974c91e64560f8bc/lava_callback.json.gz DEBUG: Setup failed for 69b38767974c91e64560f8bc. Transitting job node result: incomplete -> fail 2026-03-13 04:00:54,041 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:00:54,055 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b38767974c91e64560f8bc HTTP/1.1" 200 None 2026-03-13 04:00:54,057 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:00:54,070 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35934974c91e64560e49d HTTP/1.1" 200 None 2026-03-13 04:00:54,071 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:00:54,098 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b38767974c91e64560f8bc HTTP/1.1" 200 None 2026-03-13 04:00:54,161 - lava_callback - INFO - Completed processing callback for node 69b38767974c91e64560f8bc INFO: 172.20.0.1:40912 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53796 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39796 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39812 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-13 04:01:16,251 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:01:16,275 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:39816 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43458 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45574 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54194 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46754 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45446 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47516 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47524 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47540 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36316 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54096 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45978 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51044 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47246 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48554 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48564 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48570 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37246 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38554 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53356 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42498 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60062 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53192 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53194 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53202 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55198 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58336 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52614 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44582 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43210 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35252 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35260 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35272 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37298 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53394 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58978 - "GET / HTTP/1.0" 200 OK 2026-03-13 04:05:52,701 - lava_callback - INFO - Received callback for node 69b0ea109d19175818252fd9 INFO: 172.20.0.1:55928 - "POST /node/69b0ea109d19175818252fd9 HTTP/1.0" 202 Accepted 2026-03-13 04:05:52,706 - lava_callback - INFO - Starting async job submit for node 69b0ea109d19175818252fd9 2026-03-13 04:05:52,707 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:05:52,721 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0ea109d19175818252fd9 HTTP/1.1" 200 None 2026-03-13 04:05:52,722 - lava_callback - DEBUG - Processing node 69b0ea109d19175818252fd9: baseline-arm-clabbe 2026-03-13 04:05:52,722 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0ea109d19175818252fd9 2026-03-13 04:05:52,723 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:05:52,735 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0ea109d19175818252fd9 HTTP/1.1" 200 None 2026-03-13 04:05:52,737 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:05:52,750 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afd826ae557f2600f0f036 HTTP/1.1" 200 None 2026-03-13 04:05:52,751 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:05:52,775 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0ea109d19175818252fd9 HTTP/1.1" 200 None 2026-03-13 04:05:52,779 - lava_callback - ERROR - Error processing callback for node 69b0ea109d19175818252fd9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:55930 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40458 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46954 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46964 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46980 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54878 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55874 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42884 - "GET / HTTP/1.0" 200 OK 2026-03-13 04:06:52,722 - lava_callback - INFO - Received callback for node 69b0ea4c9d19175818252fec INFO: 172.20.0.1:57136 - "POST /node/69b0ea4c9d19175818252fec HTTP/1.0" 202 Accepted 2026-03-13 04:06:52,727 - lava_callback - INFO - Starting async job submit for node 69b0ea4c9d19175818252fec 2026-03-13 04:06:52,728 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:06:52,743 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0ea4c9d19175818252fec HTTP/1.1" 200 None 2026-03-13 04:06:52,743 - lava_callback - DEBUG - Processing node 69b0ea4c9d19175818252fec: baseline-arm-clabbe 2026-03-13 04:06:52,744 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0ea4c9d19175818252fec 2026-03-13 04:06:52,744 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:06:52,775 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0ea4c9d19175818252fec HTTP/1.1" 200 None 2026-03-13 04:06:52,776 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:06:52,789 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afd826ae557f2600f0f036 HTTP/1.1" 200 None 2026-03-13 04:06:52,791 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:06:52,821 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0ea4c9d19175818252fec HTTP/1.1" 200 None 2026-03-13 04:06:52,823 - lava_callback - ERROR - Error processing callback for node 69b0ea4c9d19175818252fec: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:57148 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41536 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55408 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55414 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55430 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58892 - "GET / HTTP/1.0" 200 OK 2026-03-13 04:07:32,807 - lava_callback - INFO - Received callback for node 69b0ea719d19175818252ffa INFO: 172.20.0.1:51044 - "POST /node/69b0ea719d19175818252ffa HTTP/1.0" 202 Accepted 2026-03-13 04:07:32,812 - lava_callback - INFO - Starting async job submit for node 69b0ea719d19175818252ffa 2026-03-13 04:07:32,813 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:07:32,841 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0ea719d19175818252ffa HTTP/1.1" 200 None 2026-03-13 04:07:32,842 - lava_callback - DEBUG - Processing node 69b0ea719d19175818252ffa: baseline-arm-clabbe 2026-03-13 04:07:32,843 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0ea719d19175818252ffa 2026-03-13 04:07:32,843 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:07:32,859 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0ea719d19175818252ffa HTTP/1.1" 200 None 2026-03-13 04:07:32,860 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:07:32,874 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afd826ae557f2600f0f036 HTTP/1.1" 200 None 2026-03-13 04:07:32,876 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:07:32,896 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0ea719d19175818252ffa HTTP/1.1" 200 None 2026-03-13 04:07:32,898 - lava_callback - ERROR - Error processing callback for node 69b0ea719d19175818252ffa: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:51060 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50574 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49832 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57906 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50594 - "GET / HTTP/1.0" 200 OK 2026-03-13 04:08:12,857 - lava_callback - INFO - Received callback for node 69b0ea989d1917581825300a INFO: 172.20.0.1:50610 - "POST /node/69b0ea989d1917581825300a HTTP/1.0" 202 Accepted 2026-03-13 04:08:12,863 - lava_callback - INFO - Starting async job submit for node 69b0ea989d1917581825300a 2026-03-13 04:08:12,864 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:08:12,878 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0ea989d1917581825300a HTTP/1.1" 200 None 2026-03-13 04:08:12,879 - lava_callback - DEBUG - Processing node 69b0ea989d1917581825300a: baseline-arm-clabbe 2026-03-13 04:08:12,879 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0ea989d1917581825300a 2026-03-13 04:08:12,880 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:08:12,893 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0ea989d1917581825300a HTTP/1.1" 200 None 2026-03-13 04:08:12,894 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:08:12,908 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afd826ae557f2600f0f036 HTTP/1.1" 200 None 2026-03-13 04:08:12,909 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:08:12,929 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0ea989d1917581825300a HTTP/1.1" 200 None 2026-03-13 04:08:12,930 - lava_callback - ERROR - Error processing callback for node 69b0ea989d1917581825300a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:50614 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50628 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37640 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50500 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39500 - "GET / HTTP/1.0" 200 OK 2026-03-13 04:08:52,833 - lava_callback - INFO - Received callback for node 69b0eac59d1917581825301d INFO: 172.20.0.1:46092 - "POST /node/69b0eac59d1917581825301d HTTP/1.0" 202 Accepted 2026-03-13 04:08:52,839 - lava_callback - INFO - Starting async job submit for node 69b0eac59d1917581825301d 2026-03-13 04:08:52,839 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:08:52,854 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0eac59d1917581825301d HTTP/1.1" 200 None 2026-03-13 04:08:52,854 - lava_callback - DEBUG - Processing node 69b0eac59d1917581825301d: baseline-arm-clabbe 2026-03-13 04:08:52,855 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0eac59d1917581825301d 2026-03-13 04:08:52,855 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:08:52,869 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0eac59d1917581825301d HTTP/1.1" 200 None 2026-03-13 04:08:52,871 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:08:52,884 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afd826ae557f2600f0f036 HTTP/1.1" 200 None 2026-03-13 04:08:52,885 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:08:52,909 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0eac59d1917581825301d HTTP/1.1" 200 None 2026-03-13 04:08:52,912 - lava_callback - ERROR - Error processing callback for node 69b0eac59d1917581825301d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:46108 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51270 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40986 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40988 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40994 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46220 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35358 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49124 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40194 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36400 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60358 - "GET / HTTP/1.0" 200 OK 2026-03-13 04:10:11,317 - lava_callback - INFO - Received callback for node 69b385f1974c91e64560f8a1 INFO: 172.20.0.1:60372 - "POST /node/69b385f1974c91e64560f8a1 HTTP/1.0" 202 Accepted 2026-03-13 04:10:11,324 - lava_callback - INFO - Starting async job submit for node 69b385f1974c91e64560f8a1 2026-03-13 04:10:11,393 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:10:11,406 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b385f1974c91e64560f8a1 HTTP/1.1" 200 None 2026-03-13 04:10:11,407 - lava_callback - DEBUG - Processing node 69b385f1974c91e64560f8a1: ltp-smoke INFO: 172.20.0.1:60382 - "GET / HTTP/1.0" 200 OK 2026-03-13 04:10:11,911 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 04:10:12,072 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 04:10:12,074 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoke-69b385f1974c91e64560f8a1/log.txt.gz 2026-03-13 04:10:12,358 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 04:10:12,635 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 04:10:12,637 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-smoke-69b385f1974c91e64560f8a1/lava-logs.yaml 2026-03-13 04:10:12,651 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 04:10:12,870 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 04:10:12,872 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoke-69b385f1974c91e64560f8a1/lava_callback.json.gz 2026-03-13 04:10:12,991 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:10:13,006 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b385f1974c91e64560f8a1 HTTP/1.1" 200 None 2026-03-13 04:10:13,007 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:10:13,021 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3593e974c91e64560e4a3 HTTP/1.1" 200 None 2026-03-13 04:10:13,022 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:10:13,047 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b385f1974c91e64560f8a1 HTTP/1.1" 200 None 2026-03-13 04:10:13,163 - lava_callback - INFO - Completed processing callback for node 69b385f1974c91e64560f8a1 INFO: 172.20.0.1:60392 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-13 04:10:16,279 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:10:16,296 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:60402 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40320 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37988 - "GET /favicon.ico HTTP/1.0" 404 Not Found INFO: 172.20.0.1:38002 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50602 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49308 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44688 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38104 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38108 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38122 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49640 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52588 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41778 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41436 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50482 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50866 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50872 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50884 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57380 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57192 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46974 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48392 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55870 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47184 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47188 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47194 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35748 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55328 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48206 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44094 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33386 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33146 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33152 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33156 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36186 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37682 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58546 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36666 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32882 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48300 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48314 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48326 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47858 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55896 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37010 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34118 - "GET / HTTP/1.0" 200 OK 2026-03-13 04:15:56,820 - lava_callback - INFO - Received callback for node 69b363cc974c91e64560ed61 INFO: 172.20.0.1:34126 - "POST /node/69b363cc974c91e64560ed61 HTTP/1.0" 202 Accepted 2026-03-13 04:15:56,826 - lava_callback - INFO - Starting async job submit for node 69b363cc974c91e64560ed61 2026-03-13 04:15:56,848 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:15:56,890 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b363cc974c91e64560ed61 HTTP/1.1" 200 None 2026-03-13 04:15:56,890 - lava_callback - DEBUG - Processing node 69b363cc974c91e64560ed61: baseline-arm64-broonie 2026-03-13 04:15:57,169 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 04:15:57,319 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 04:15:57,321 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69b363cc974c91e64560ed61/log.txt.gz 2026-03-13 04:15:57,482 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 04:15:57,756 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 04:15:57,759 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69b363cc974c91e64560ed61/lava-logs.yaml 2026-03-13 04:15:57,766 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 04:15:57,944 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 04:15:57,947 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69b363cc974c91e64560ed61/lava_callback.json.gz 2026-03-13 04:15:57,969 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:15:57,995 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b363cc974c91e64560ed61 HTTP/1.1" 200 None 2026-03-13 04:15:57,997 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:15:58,034 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3594d974c91e64560e4b2 HTTP/1.1" 200 None 2026-03-13 04:15:58,036 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:15:58,068 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b363cc974c91e64560ed61 HTTP/1.1" 200 None 2026-03-13 04:15:58,069 - lava_callback - INFO - Completed processing callback for node 69b363cc974c91e64560ed61 INFO: 172.20.0.1:40224 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54046 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54056 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-13 04:16:16,299 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:16:16,324 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:54072 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44092 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44974 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46988 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60254 - "GET / HTTP/1.0" 200 OK 2026-03-13 04:16:57,260 - lava_callback - INFO - Received callback for node 69b0ecaa9d19175818253058 INFO: 172.20.0.1:60262 - "POST /node/69b0ecaa9d19175818253058 HTTP/1.0" 202 Accepted 2026-03-13 04:16:57,265 - lava_callback - INFO - Starting async job submit for node 69b0ecaa9d19175818253058 2026-03-13 04:16:57,266 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:16:57,283 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0ecaa9d19175818253058 HTTP/1.1" 200 None 2026-03-13 04:16:57,284 - lava_callback - DEBUG - Processing node 69b0ecaa9d19175818253058: baseline-arm64-baylibre 2026-03-13 04:16:57,285 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0ecaa9d19175818253058 2026-03-13 04:16:57,285 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:16:57,298 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0ecaa9d19175818253058 HTTP/1.1" 200 None 2026-03-13 04:16:57,299 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:16:57,321 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afd839ae557f2600f0f045 HTTP/1.1" 200 None 2026-03-13 04:16:57,322 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:16:57,352 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0ecaa9d19175818253058 HTTP/1.1" 200 None 2026-03-13 04:16:57,357 - lava_callback - ERROR - Error processing callback for node 69b0ecaa9d19175818253058: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:59496 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45238 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45244 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45260 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50618 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38856 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39042 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60788 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38664 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51014 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51028 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51030 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37274 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45544 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40554 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44832 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37054 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37742 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37756 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37758 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50194 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51008 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52804 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49834 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42238 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40380 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40394 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40402 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40178 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51962 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42082 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58372 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46116 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48220 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48236 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48246 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53376 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42402 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48942 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60158 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60936 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59932 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59942 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59954 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43660 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55680 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58700 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59394 - "GET / HTTP/1.0" 200 OK 2026-03-13 04:22:57,543 - lava_callback - INFO - Received callback for node 69b0ee169d19175818253066 INFO: 172.20.0.1:59404 - "POST /node/69b0ee169d19175818253066 HTTP/1.0" 202 Accepted 2026-03-13 04:22:57,549 - lava_callback - INFO - Starting async job submit for node 69b0ee169d19175818253066 2026-03-13 04:22:57,549 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:22:57,566 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0ee169d19175818253066 HTTP/1.1" 200 None 2026-03-13 04:22:57,567 - lava_callback - DEBUG - Processing node 69b0ee169d19175818253066: baseline-arm64-baylibre 2026-03-13 04:22:57,567 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0ee169d19175818253066 2026-03-13 04:22:57,567 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:22:57,589 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0ee169d19175818253066 HTTP/1.1" 200 None 2026-03-13 04:22:57,591 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:22:57,608 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afd839ae557f2600f0f045 HTTP/1.1" 200 None 2026-03-13 04:22:57,610 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:22:57,630 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0ee169d19175818253066 HTTP/1.1" 200 None 2026-03-13 04:22:57,635 - lava_callback - ERROR - Error processing callback for node 69b0ee169d19175818253066: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:32954 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60488 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60494 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60496 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33870 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48792 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43040 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37684 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39150 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39714 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39728 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39742 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55124 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39042 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46218 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46448 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39400 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58772 - "GET / HTTP/1.0" 200 OK 2026-03-13 04:25:13,496 - lava_callback - INFO - Received callback for node 69b0ee949d19175818253075 INFO: 172.20.0.1:58788 - "POST /node/69b0ee949d19175818253075 HTTP/1.0" 202 Accepted 2026-03-13 04:25:13,501 - lava_callback - INFO - Starting async job submit for node 69b0ee949d19175818253075 2026-03-13 04:25:13,502 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:25:13,517 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0ee949d19175818253075 HTTP/1.1" 200 None 2026-03-13 04:25:13,518 - lava_callback - DEBUG - Processing node 69b0ee949d19175818253075: baseline-arm64-clabbe 2026-03-13 04:25:13,518 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0ee949d19175818253075 2026-03-13 04:25:13,519 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:25:13,540 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0ee949d19175818253075 HTTP/1.1" 200 None 2026-03-13 04:25:13,542 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:25:13,555 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afd839ae557f2600f0f045 HTTP/1.1" 200 None 2026-03-13 04:25:13,556 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:25:13,577 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0ee949d19175818253075 HTTP/1.1" 200 None 2026-03-13 04:25:13,579 - lava_callback - ERROR - Error processing callback for node 69b0ee949d19175818253075: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:58796 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58808 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49662 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57042 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57522 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50694 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33892 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40176 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40180 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40182 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51926 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35326 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47566 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55132 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37660 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57348 - "GET / HTTP/1.0" 200 OK 2026-03-13 04:27:13,609 - lava_callback - INFO - Received callback for node 69b0ef0e9d19175818253084 INFO: 172.20.0.1:57358 - "POST /node/69b0ef0e9d19175818253084 HTTP/1.0" 202 Accepted 2026-03-13 04:27:13,614 - lava_callback - INFO - Starting async job submit for node 69b0ef0e9d19175818253084 2026-03-13 04:27:13,615 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:27:13,631 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0ef0e9d19175818253084 HTTP/1.1" 200 None 2026-03-13 04:27:13,632 - lava_callback - DEBUG - Processing node 69b0ef0e9d19175818253084: baseline-arm64-clabbe 2026-03-13 04:27:13,632 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0ef0e9d19175818253084 2026-03-13 04:27:13,632 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:27:13,677 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0ef0e9d19175818253084 HTTP/1.1" 200 None 2026-03-13 04:27:13,679 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:27:13,697 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afd839ae557f2600f0f045 HTTP/1.1" 200 None 2026-03-13 04:27:13,699 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:27:13,719 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0ef0e9d19175818253084 HTTP/1.1" 200 None 2026-03-13 04:27:13,721 - lava_callback - ERROR - Error processing callback for node 69b0ef0e9d19175818253084: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:57372 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57380 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45986 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37026 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37930 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37628 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50586 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49376 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49390 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49394 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50192 - "GET / HTTP/1.0" 200 OK 2026-03-13 04:28:33,676 - lava_callback - INFO - Received callback for node 69b0ef5f9d19175818253095 INFO: 172.20.0.1:46074 - "POST /node/69b0ef5f9d19175818253095 HTTP/1.0" 202 Accepted 2026-03-13 04:28:33,681 - lava_callback - INFO - Starting async job submit for node 69b0ef5f9d19175818253095 2026-03-13 04:28:33,682 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:28:33,702 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0ef5f9d19175818253095 HTTP/1.1" 200 None 2026-03-13 04:28:33,702 - lava_callback - DEBUG - Processing node 69b0ef5f9d19175818253095: baseline-arm64-clabbe 2026-03-13 04:28:33,703 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0ef5f9d19175818253095 2026-03-13 04:28:33,703 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:28:33,716 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0ef5f9d19175818253095 HTTP/1.1" 200 None 2026-03-13 04:28:33,718 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:28:33,730 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afd839ae557f2600f0f045 HTTP/1.1" 200 None 2026-03-13 04:28:33,732 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:28:33,760 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0ef5f9d19175818253095 HTTP/1.1" 200 None 2026-03-13 04:28:33,762 - lava_callback - ERROR - Error processing callback for node 69b0ef5f9d19175818253095: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:46086 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48950 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35108 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56844 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36482 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36498 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36510 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44460 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44648 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49762 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45640 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53494 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37046 - "GET / HTTP/1.0" 200 OK 2026-03-13 04:30:13,739 - lava_callback - INFO - Received callback for node 69b0efca9d191758182530aa INFO: 172.20.0.1:37062 - "POST /node/69b0efca9d191758182530aa HTTP/1.0" 202 Accepted 2026-03-13 04:30:13,745 - lava_callback - INFO - Starting async job submit for node 69b0efca9d191758182530aa 2026-03-13 04:30:13,746 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:30:13,775 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0efca9d191758182530aa HTTP/1.1" 200 None 2026-03-13 04:30:13,776 - lava_callback - DEBUG - Processing node 69b0efca9d191758182530aa: baseline-arm64-clabbe 2026-03-13 04:30:13,776 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0efca9d191758182530aa 2026-03-13 04:30:13,777 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:30:13,790 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0efca9d191758182530aa HTTP/1.1" 200 None 2026-03-13 04:30:13,791 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:30:13,804 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afd839ae557f2600f0f045 HTTP/1.1" 200 None 2026-03-13 04:30:13,806 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:30:13,835 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0efca9d191758182530aa HTTP/1.1" 200 None 2026-03-13 04:30:13,836 - lava_callback - ERROR - Error processing callback for node 69b0efca9d191758182530aa: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:37072 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37088 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36876 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43530 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56780 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60728 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46052 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36128 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36132 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36140 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49348 - "GET / HTTP/1.0" 200 OK 2026-03-13 04:31:33,804 - lava_callback - INFO - Received callback for node 69b0f0219d191758182530b2 INFO: 172.20.0.1:34384 - "POST /node/69b0f0219d191758182530b2 HTTP/1.0" 202 Accepted 2026-03-13 04:31:33,809 - lava_callback - INFO - Starting async job submit for node 69b0f0219d191758182530b2 2026-03-13 04:31:33,810 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:31:33,825 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0f0219d191758182530b2 HTTP/1.1" 200 None 2026-03-13 04:31:33,826 - lava_callback - DEBUG - Processing node 69b0f0219d191758182530b2: baseline-arm64-clabbe 2026-03-13 04:31:33,826 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0f0219d191758182530b2 2026-03-13 04:31:33,826 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:31:33,846 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0f0219d191758182530b2 HTTP/1.1" 200 None 2026-03-13 04:31:33,847 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:31:33,861 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afd839ae557f2600f0f045 HTTP/1.1" 200 None 2026-03-13 04:31:33,863 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:31:33,882 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0f0219d191758182530b2 HTTP/1.1" 200 None 2026-03-13 04:31:33,883 - lava_callback - ERROR - Error processing callback for node 69b0f0219d191758182530b2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:34396 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42308 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56900 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33744 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38980 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38984 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38990 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44054 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54600 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40648 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42460 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58474 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43268 - "GET / HTTP/1.0" 200 OK 2026-03-13 04:33:13,845 - lava_callback - INFO - Received callback for node 69b0f07e9d191758182530c8 INFO: 172.20.0.1:43284 - "POST /node/69b0f07e9d191758182530c8 HTTP/1.0" 202 Accepted 2026-03-13 04:33:13,850 - lava_callback - INFO - Starting async job submit for node 69b0f07e9d191758182530c8 2026-03-13 04:33:13,851 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:33:13,866 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0f07e9d191758182530c8 HTTP/1.1" 200 None 2026-03-13 04:33:13,866 - lava_callback - DEBUG - Processing node 69b0f07e9d191758182530c8: baseline-arm64-clabbe 2026-03-13 04:33:13,867 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0f07e9d191758182530c8 2026-03-13 04:33:13,867 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:33:13,905 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0f07e9d191758182530c8 HTTP/1.1" 200 None 2026-03-13 04:33:13,906 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:33:13,930 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afd839ae557f2600f0f045 HTTP/1.1" 200 None 2026-03-13 04:33:13,932 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:33:13,957 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0f07e9d191758182530c8 HTTP/1.1" 200 None 2026-03-13 04:33:13,958 - lava_callback - ERROR - Error processing callback for node 69b0f07e9d191758182530c8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:43292 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43298 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54002 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36534 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45786 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49226 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37804 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55560 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55570 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55574 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41784 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38368 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34844 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40496 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48776 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45100 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45110 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45114 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35690 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46176 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38906 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54660 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54654 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56358 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56366 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56380 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45572 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44520 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40234 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47194 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41048 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45010 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45018 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45024 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54182 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42796 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55394 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53910 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56372 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54966 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54972 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54976 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53590 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60770 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58946 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50810 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36670 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34482 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34490 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34492 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38436 - "GET / HTTP/1.0" 200 OK 2026-03-13 04:39:35,300 - lava_callback - INFO - Received callback for node 69b38c03974c91e64560f949 INFO: 172.20.0.1:50474 - "POST /node/69b38c03974c91e64560f949 HTTP/1.0" 202 Accepted 2026-03-13 04:39:35,349 - lava_callback - INFO - Starting async job submit for node 69b38c03974c91e64560f949 2026-03-13 04:39:35,396 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:39:35,444 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b38c03974c91e64560f949 HTTP/1.1" 200 None 2026-03-13 04:39:35,445 - lava_callback - DEBUG - Processing node 69b38c03974c91e64560f949: baseline-arm-pengutronix 2026-03-13 04:39:35,562 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 04:39:35,726 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 04:39:35,728 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69b38c03974c91e64560f949/log.txt.gz 2026-03-13 04:39:35,786 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 04:39:35,992 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 04:39:35,994 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69b38c03974c91e64560f949/lava-logs.yaml 2026-03-13 04:39:35,998 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 04:39:36,131 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 04:39:36,133 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69b38c03974c91e64560f949/lava_callback.json.gz DEBUG: Setup failed for 69b38c03974c91e64560f949. Transitting job node result: incomplete -> fail 2026-03-13 04:39:36,182 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:39:36,197 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b38c03974c91e64560f949 HTTP/1.1" 200 None 2026-03-13 04:39:36,198 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:39:36,231 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35934974c91e64560e49d HTTP/1.1" 200 None 2026-03-13 04:39:36,233 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:39:36,257 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b38c03974c91e64560f949 HTTP/1.1" 200 None 2026-03-13 04:39:36,302 - lava_callback - INFO - Completed processing callback for node 69b38c03974c91e64560f949 INFO: 172.20.0.1:50488 - "GET / HTTP/1.0" 200 OK 2026-03-13 04:39:46,330 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:39:46,370 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:47098 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59962 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52530 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34754 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34762 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34776 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45148 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43884 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48606 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38236 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58036 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46956 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46962 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46964 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45648 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35272 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56100 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40668 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41752 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34218 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34226 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34240 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60980 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43486 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53754 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45904 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43418 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43358 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43370 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43380 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46194 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55672 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45628 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60422 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39024 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54160 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54170 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54184 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39946 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42340 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40594 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50636 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34072 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51624 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51636 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51650 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52146 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54566 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34088 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57422 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50284 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37056 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37064 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37078 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42764 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38558 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57922 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41644 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34506 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40242 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40250 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40260 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57912 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50964 - "GET / HTTP/1.0" 200 OK 2026-03-13 04:47:39,068 - lava_callback - INFO - Received callback for node 69b0f3dd9d191758182530e2 INFO: 172.20.0.1:50974 - "POST /node/69b0f3dd9d191758182530e2 HTTP/1.0" 202 Accepted 2026-03-13 04:47:39,073 - lava_callback - INFO - Starting async job submit for node 69b0f3dd9d191758182530e2 2026-03-13 04:47:39,075 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:47:39,094 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0f3dd9d191758182530e2 HTTP/1.1" 200 None 2026-03-13 04:47:39,096 - lava_callback - DEBUG - Processing node 69b0f3dd9d191758182530e2: baseline-arm-baylibre 2026-03-13 04:47:39,096 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0f3dd9d191758182530e2 2026-03-13 04:47:39,097 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:47:39,143 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0f3dd9d191758182530e2 HTTP/1.1" 200 None 2026-03-13 04:47:39,145 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:47:39,159 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ada13b8fc97e5ba07d4d75 HTTP/1.1" 200 None 2026-03-13 04:47:39,161 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:47:39,184 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0f3dd9d191758182530e2 HTTP/1.1" 200 None 2026-03-13 04:47:39,190 - lava_callback - ERROR - Error processing callback for node 69b0f3dd9d191758182530e2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:49130 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37872 - "GET / HTTP/1.0" 200 OK 2026-03-13 04:47:57,711 - lava_callback - INFO - Received callback for node 69b363d3974c91e64560ed6d INFO: 172.20.0.1:37882 - "POST /node/69b363d3974c91e64560ed6d HTTP/1.0" 202 Accepted 2026-03-13 04:47:57,717 - lava_callback - INFO - Starting async job submit for node 69b363d3974c91e64560ed6d 2026-03-13 04:47:57,739 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:47:57,754 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b363d3974c91e64560ed6d HTTP/1.1" 200 None 2026-03-13 04:47:57,755 - lava_callback - DEBUG - Processing node 69b363d3974c91e64560ed6d: baseline-arm64-broonie 2026-03-13 04:47:57,989 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 04:47:58,152 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 04:47:58,154 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69b363d3974c91e64560ed6d/log.txt.gz 2026-03-13 04:47:58,301 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 04:47:58,512 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 04:47:58,514 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69b363d3974c91e64560ed6d/lava-logs.yaml 2026-03-13 04:47:58,520 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 04:47:58,680 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 04:47:58,682 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69b363d3974c91e64560ed6d/lava_callback.json.gz 2026-03-13 04:47:58,705 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:47:58,720 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b363d3974c91e64560ed6d HTTP/1.1" 200 None 2026-03-13 04:47:58,721 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:47:58,735 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3594d974c91e64560e4b2 HTTP/1.1" 200 None 2026-03-13 04:47:58,736 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:47:58,771 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b363d3974c91e64560ed6d HTTP/1.1" 200 None 2026-03-13 04:47:58,772 - lava_callback - INFO - Completed processing callback for node 69b363d3974c91e64560ed6d INFO: 172.20.0.1:50548 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35360 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35372 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-13 04:48:16,373 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:48:16,409 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:35382 - "GET / HTTP/1.0" 200 OK 2026-03-13 04:48:18,967 - lava_callback - INFO - Received callback for node 69b0f4049d191758182530f8 INFO: 172.20.0.1:35384 - "POST /node/69b0f4049d191758182530f8 HTTP/1.0" 202 Accepted 2026-03-13 04:48:18,972 - lava_callback - INFO - Starting async job submit for node 69b0f4049d191758182530f8 2026-03-13 04:48:18,973 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:48:18,993 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0f4049d191758182530f8 HTTP/1.1" 200 None 2026-03-13 04:48:18,994 - lava_callback - DEBUG - Processing node 69b0f4049d191758182530f8: baseline-arm-baylibre 2026-03-13 04:48:18,994 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0f4049d191758182530f8 2026-03-13 04:48:18,995 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:48:19,019 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0f4049d191758182530f8 HTTP/1.1" 200 None 2026-03-13 04:48:19,020 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:48:19,053 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ada13b8fc97e5ba07d4d75 HTTP/1.1" 200 None 2026-03-13 04:48:19,055 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:48:19,075 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0f4049d191758182530f8 HTTP/1.1" 200 None 2026-03-13 04:48:19,079 - lava_callback - ERROR - Error processing callback for node 69b0f4049d191758182530f8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:41260 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56788 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49506 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48294 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59460 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44320 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44328 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44338 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36750 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45728 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47458 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49008 - "GET / HTTP/1.0" 200 OK 2026-03-13 04:49:59,084 - lava_callback - INFO - Received callback for node 69b0f4729d19175818253122 INFO: 172.20.0.1:49014 - "POST /node/69b0f4729d19175818253122 HTTP/1.0" 202 Accepted 2026-03-13 04:49:59,089 - lava_callback - INFO - Starting async job submit for node 69b0f4729d19175818253122 2026-03-13 04:49:59,090 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:49:59,104 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0f4729d19175818253122 HTTP/1.1" 200 None 2026-03-13 04:49:59,105 - lava_callback - DEBUG - Processing node 69b0f4729d19175818253122: baseline-arm-baylibre 2026-03-13 04:49:59,105 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0f4729d19175818253122 2026-03-13 04:49:59,106 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:49:59,130 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0f4729d19175818253122 HTTP/1.1" 200 None 2026-03-13 04:49:59,132 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:49:59,145 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ada13b8fc97e5ba07d4d75 HTTP/1.1" 200 None 2026-03-13 04:49:59,146 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:49:59,172 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0f4729d19175818253122 HTTP/1.1" 200 None 2026-03-13 04:49:59,173 - lava_callback - ERROR - Error processing callback for node 69b0f4729d19175818253122: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:50132 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47166 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47170 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47180 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50740 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49100 - "GET / HTTP/1.0" 200 OK 2026-03-13 04:50:39,077 - lava_callback - INFO - Received callback for node 69b0f4969d19175818253131 INFO: 172.20.0.1:49114 - "POST /node/69b0f4969d19175818253131 HTTP/1.0" 202 Accepted 2026-03-13 04:50:39,082 - lava_callback - INFO - Starting async job submit for node 69b0f4969d19175818253131 2026-03-13 04:50:39,083 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:50:39,097 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0f4969d19175818253131 HTTP/1.1" 200 None 2026-03-13 04:50:39,098 - lava_callback - DEBUG - Processing node 69b0f4969d19175818253131: baseline-arm-baylibre 2026-03-13 04:50:39,098 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0f4969d19175818253131 2026-03-13 04:50:39,098 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:50:39,113 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0f4969d19175818253131 HTTP/1.1" 200 None 2026-03-13 04:50:39,115 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:50:39,138 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ada13b8fc97e5ba07d4d75 HTTP/1.1" 200 None 2026-03-13 04:50:39,140 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:50:39,175 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0f4969d19175818253131 HTTP/1.1" 200 None 2026-03-13 04:50:39,180 - lava_callback - ERROR - Error processing callback for node 69b0f4969d19175818253131: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:53704 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48942 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52394 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46344 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46348 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46358 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54774 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37682 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45922 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58340 - "GET / HTTP/1.0" 200 OK 2026-03-13 04:51:59,205 - lava_callback - INFO - Received callback for node 69b0f4e19d1917581825314e INFO: 172.20.0.1:58352 - "POST /node/69b0f4e19d1917581825314e HTTP/1.0" 202 Accepted 2026-03-13 04:51:59,211 - lava_callback - INFO - Starting async job submit for node 69b0f4e19d1917581825314e 2026-03-13 04:51:59,212 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:51:59,228 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0f4e19d1917581825314e HTTP/1.1" 200 None 2026-03-13 04:51:59,229 - lava_callback - DEBUG - Processing node 69b0f4e19d1917581825314e: baseline-arm-baylibre 2026-03-13 04:51:59,229 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0f4e19d1917581825314e 2026-03-13 04:51:59,230 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:51:59,244 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0f4e19d1917581825314e HTTP/1.1" 200 None 2026-03-13 04:51:59,246 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:51:59,259 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ada13b8fc97e5ba07d4d75 HTTP/1.1" 200 None 2026-03-13 04:51:59,260 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:51:59,282 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0f4e19d1917581825314e HTTP/1.1" 200 None 2026-03-13 04:51:59,283 - lava_callback - ERROR - Error processing callback for node 69b0f4e19d1917581825314e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:50324 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43732 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43742 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43746 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37668 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58774 - "GET / HTTP/1.0" 200 OK 2026-03-13 04:52:39,201 - lava_callback - INFO - Received callback for node 69b0f5069d1917581825315c INFO: 172.20.0.1:58778 - "POST /node/69b0f5069d1917581825315c HTTP/1.0" 202 Accepted 2026-03-13 04:52:39,206 - lava_callback - INFO - Starting async job submit for node 69b0f5069d1917581825315c 2026-03-13 04:52:39,207 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:52:39,232 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0f5069d1917581825315c HTTP/1.1" 200 None 2026-03-13 04:52:39,233 - lava_callback - DEBUG - Processing node 69b0f5069d1917581825315c: baseline-arm-baylibre 2026-03-13 04:52:39,233 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0f5069d1917581825315c 2026-03-13 04:52:39,234 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:52:39,256 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0f5069d1917581825315c HTTP/1.1" 200 None 2026-03-13 04:52:39,257 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:52:39,271 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ada13b8fc97e5ba07d4d75 HTTP/1.1" 200 None 2026-03-13 04:52:39,272 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:52:39,305 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0f5069d1917581825315c HTTP/1.1" 200 None 2026-03-13 04:52:39,315 - lava_callback - ERROR - Error processing callback for node 69b0f5069d1917581825315c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:33314 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44568 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49802 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48404 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48420 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48426 - "GET / HTTP/1.0" 200 OK 2026-03-13 04:53:19,268 - lava_callback - INFO - Received callback for node 69b0f52a9d1917581825316d INFO: 172.20.0.1:48442 - "POST /node/69b0f52a9d1917581825316d HTTP/1.0" 202 Accepted 2026-03-13 04:53:19,273 - lava_callback - INFO - Starting async job submit for node 69b0f52a9d1917581825316d 2026-03-13 04:53:19,274 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:53:19,289 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0f52a9d1917581825316d HTTP/1.1" 200 None 2026-03-13 04:53:19,290 - lava_callback - DEBUG - Processing node 69b0f52a9d1917581825316d: baseline-arm-baylibre 2026-03-13 04:53:19,290 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0f52a9d1917581825316d 2026-03-13 04:53:19,290 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:53:19,303 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0f52a9d1917581825316d HTTP/1.1" 200 None 2026-03-13 04:53:19,305 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:53:19,320 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ada13b8fc97e5ba07d4d75 HTTP/1.1" 200 None 2026-03-13 04:53:19,322 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:53:19,342 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0f52a9d1917581825316d HTTP/1.1" 200 None 2026-03-13 04:53:19,348 - lava_callback - ERROR - Error processing callback for node 69b0f52a9d1917581825316d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:34664 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47014 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40230 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37658 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40954 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58974 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58976 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58980 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52008 - "GET / HTTP/1.0" 200 OK 2026-03-13 04:54:34,749 - lava_callback - INFO - Received callback for node 69b0f5779d1917581825318d INFO: 172.20.0.1:45238 - "POST /node/69b0f5779d1917581825318d HTTP/1.0" 202 Accepted 2026-03-13 04:54:34,755 - lava_callback - INFO - Starting async job submit for node 69b0f5779d1917581825318d 2026-03-13 04:54:34,756 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:54:34,770 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0f5779d1917581825318d HTTP/1.1" 200 None 2026-03-13 04:54:34,771 - lava_callback - DEBUG - Processing node 69b0f5779d1917581825318d: baseline-arm-clabbe 2026-03-13 04:54:34,772 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0f5779d1917581825318d 2026-03-13 04:54:34,772 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:54:34,786 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0f5779d1917581825318d HTTP/1.1" 200 None 2026-03-13 04:54:34,787 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:54:34,800 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ada13b8fc97e5ba07d4d75 HTTP/1.1" 200 None 2026-03-13 04:54:34,801 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:54:34,821 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0f5779d1917581825318d HTTP/1.1" 200 None 2026-03-13 04:54:34,823 - lava_callback - ERROR - Error processing callback for node 69b0f5779d1917581825318d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:45254 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54600 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46624 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48080 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55374 - "GET / HTTP/1.0" 200 OK 2026-03-13 04:55:14,588 - lava_callback - INFO - Received callback for node 69b0f59d9d1917581825319f INFO: 172.20.0.1:55386 - "POST /node/69b0f59d9d1917581825319f HTTP/1.0" 202 Accepted 2026-03-13 04:55:14,593 - lava_callback - INFO - Starting async job submit for node 69b0f59d9d1917581825319f 2026-03-13 04:55:14,594 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:55:14,608 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0f59d9d1917581825319f HTTP/1.1" 200 None 2026-03-13 04:55:14,609 - lava_callback - DEBUG - Processing node 69b0f59d9d1917581825319f: baseline-arm-clabbe 2026-03-13 04:55:14,609 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0f59d9d1917581825319f 2026-03-13 04:55:14,610 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:55:14,622 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0f59d9d1917581825319f HTTP/1.1" 200 None 2026-03-13 04:55:14,624 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:55:14,638 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ada13b8fc97e5ba07d4d75 HTTP/1.1" 200 None 2026-03-13 04:55:14,639 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:55:14,668 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0f59d9d1917581825319f HTTP/1.1" 200 None 2026-03-13 04:55:14,674 - lava_callback - ERROR - Error processing callback for node 69b0f59d9d1917581825319f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:55400 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55412 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59188 - "GET / HTTP/1.0" 200 OK 2026-03-13 04:55:34,743 - lava_callback - INFO - Received callback for node 69b0f5c39d191758182531af INFO: 172.20.0.1:34116 - "POST /node/69b0f5c39d191758182531af HTTP/1.0" 202 Accepted 2026-03-13 04:55:34,749 - lava_callback - INFO - Starting async job submit for node 69b0f5c39d191758182531af 2026-03-13 04:55:34,749 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:55:34,765 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0f5c39d191758182531af HTTP/1.1" 200 None 2026-03-13 04:55:34,766 - lava_callback - DEBUG - Processing node 69b0f5c39d191758182531af: baseline-arm-clabbe 2026-03-13 04:55:34,766 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0f5c39d191758182531af 2026-03-13 04:55:34,767 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:55:34,780 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0f5c39d191758182531af HTTP/1.1" 200 None 2026-03-13 04:55:34,781 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:55:34,806 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ada13b8fc97e5ba07d4d75 HTTP/1.1" 200 None 2026-03-13 04:55:34,807 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:55:34,832 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0f5c39d191758182531af HTTP/1.1" 200 None 2026-03-13 04:55:34,833 - lava_callback - ERROR - Error processing callback for node 69b0f5c39d191758182531af: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:34132 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49254 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35830 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35742 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44666 - "GET / HTTP/1.0" 200 OK 2026-03-13 04:56:14,793 - lava_callback - INFO - Received callback for node 69b0f5e89d191758182531c4 INFO: 172.20.0.1:44678 - "POST /node/69b0f5e89d191758182531c4 HTTP/1.0" 202 Accepted 2026-03-13 04:56:14,798 - lava_callback - INFO - Starting async job submit for node 69b0f5e89d191758182531c4 2026-03-13 04:56:14,799 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:56:14,814 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0f5e89d191758182531c4 HTTP/1.1" 200 None 2026-03-13 04:56:14,815 - lava_callback - DEBUG - Processing node 69b0f5e89d191758182531c4: baseline-arm-clabbe 2026-03-13 04:56:14,815 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0f5e89d191758182531c4 2026-03-13 04:56:14,816 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:56:14,830 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0f5e89d191758182531c4 HTTP/1.1" 200 None 2026-03-13 04:56:14,831 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:56:14,845 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ada13b8fc97e5ba07d4d75 HTTP/1.1" 200 None 2026-03-13 04:56:14,846 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:56:14,870 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0f5e89d191758182531c4 HTTP/1.1" 200 None 2026-03-13 04:56:14,876 - lava_callback - ERROR - Error processing callback for node 69b0f5e89d191758182531c4: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:44682 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44690 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59236 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49792 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37488 - "GET / HTTP/1.0" 200 OK 2026-03-13 04:56:54,803 - lava_callback - INFO - Received callback for node 69b0f60e9d191758182531d6 INFO: 172.20.0.1:60256 - "POST /node/69b0f60e9d191758182531d6 HTTP/1.0" 202 Accepted 2026-03-13 04:56:54,809 - lava_callback - INFO - Starting async job submit for node 69b0f60e9d191758182531d6 2026-03-13 04:56:54,810 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:56:54,824 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0f60e9d191758182531d6 HTTP/1.1" 200 None 2026-03-13 04:56:54,825 - lava_callback - DEBUG - Processing node 69b0f60e9d191758182531d6: baseline-arm-clabbe 2026-03-13 04:56:54,825 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0f60e9d191758182531d6 2026-03-13 04:56:54,826 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:56:54,839 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0f60e9d191758182531d6 HTTP/1.1" 200 None 2026-03-13 04:56:54,840 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:56:54,858 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ada13b8fc97e5ba07d4d75 HTTP/1.1" 200 None 2026-03-13 04:56:54,859 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:56:54,899 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0f60e9d191758182531d6 HTTP/1.1" 200 None 2026-03-13 04:56:54,903 - lava_callback - ERROR - Error processing callback for node 69b0f60e9d191758182531d6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:60272 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40210 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51884 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51886 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51896 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49614 - "GET / HTTP/1.0" 200 OK 2026-03-13 04:57:34,812 - lava_callback - INFO - Received callback for node 69b0f6339d191758182531e7 INFO: 172.20.0.1:49972 - "POST /node/69b0f6339d191758182531e7 HTTP/1.0" 202 Accepted 2026-03-13 04:57:34,817 - lava_callback - INFO - Starting async job submit for node 69b0f6339d191758182531e7 2026-03-13 04:57:34,818 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:57:34,832 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0f6339d191758182531e7 HTTP/1.1" 200 None 2026-03-13 04:57:34,832 - lava_callback - DEBUG - Processing node 69b0f6339d191758182531e7: baseline-arm-clabbe 2026-03-13 04:57:34,833 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0f6339d191758182531e7 2026-03-13 04:57:34,833 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:57:34,847 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0f6339d191758182531e7 HTTP/1.1" 200 None 2026-03-13 04:57:34,848 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:57:34,861 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ada13b8fc97e5ba07d4d75 HTTP/1.1" 200 None 2026-03-13 04:57:34,863 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:57:34,894 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0f6339d191758182531e7 HTTP/1.1" 200 None 2026-03-13 04:57:34,897 - lava_callback - ERROR - Error processing callback for node 69b0f6339d191758182531e7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:49978 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52524 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36810 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49340 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39612 - "GET / HTTP/1.0" 200 OK 2026-03-13 04:58:14,840 - lava_callback - INFO - Received callback for node 69b0f6579d191758182531f9 INFO: 172.20.0.1:39624 - "POST /node/69b0f6579d191758182531f9 HTTP/1.0" 202 Accepted 2026-03-13 04:58:14,845 - lava_callback - INFO - Starting async job submit for node 69b0f6579d191758182531f9 2026-03-13 04:58:14,846 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:58:14,871 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0f6579d191758182531f9 HTTP/1.1" 200 None 2026-03-13 04:58:14,872 - lava_callback - DEBUG - Processing node 69b0f6579d191758182531f9: baseline-arm-clabbe 2026-03-13 04:58:14,872 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0f6579d191758182531f9 2026-03-13 04:58:14,873 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:58:14,895 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0f6579d191758182531f9 HTTP/1.1" 200 None 2026-03-13 04:58:14,896 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:58:14,909 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ada13b8fc97e5ba07d4d75 HTTP/1.1" 200 None 2026-03-13 04:58:14,911 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:58:14,931 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0f6579d191758182531f9 HTTP/1.1" 200 None 2026-03-13 04:58:14,937 - lava_callback - ERROR - Error processing callback for node 69b0f6579d191758182531f9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:39638 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39646 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48540 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57324 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43462 - "GET / HTTP/1.0" 200 OK 2026-03-13 04:58:54,850 - lava_callback - INFO - Received callback for node 69b0f67c9d1917581825320c INFO: 172.20.0.1:52074 - "POST /node/69b0f67c9d1917581825320c HTTP/1.0" 202 Accepted 2026-03-13 04:58:54,855 - lava_callback - INFO - Starting async job submit for node 69b0f67c9d1917581825320c 2026-03-13 04:58:54,856 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:58:54,871 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0f67c9d1917581825320c HTTP/1.1" 200 None 2026-03-13 04:58:54,871 - lava_callback - DEBUG - Processing node 69b0f67c9d1917581825320c: baseline-arm-clabbe 2026-03-13 04:58:54,872 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0f67c9d1917581825320c 2026-03-13 04:58:54,872 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:58:54,885 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0f67c9d1917581825320c HTTP/1.1" 200 None 2026-03-13 04:58:54,886 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:58:54,899 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ada13b8fc97e5ba07d4d75 HTTP/1.1" 200 None 2026-03-13 04:58:54,901 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:58:54,921 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0f67c9d1917581825320c HTTP/1.1" 200 None 2026-03-13 04:58:54,922 - lava_callback - ERROR - Error processing callback for node 69b0f67c9d1917581825320c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:52080 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41960 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34476 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34486 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34494 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54470 - "GET / HTTP/1.0" 200 OK 2026-03-13 04:59:34,949 - lava_callback - INFO - Received callback for node 69b0f6a09d19175818253217 INFO: 172.20.0.1:38734 - "POST /node/69b0f6a09d19175818253217 HTTP/1.0" 202 Accepted 2026-03-13 04:59:34,954 - lava_callback - INFO - Starting async job submit for node 69b0f6a09d19175818253217 2026-03-13 04:59:34,955 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:59:34,974 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0f6a09d19175818253217 HTTP/1.1" 200 None 2026-03-13 04:59:34,975 - lava_callback - DEBUG - Processing node 69b0f6a09d19175818253217: baseline-arm-clabbe 2026-03-13 04:59:34,975 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0f6a09d19175818253217 2026-03-13 04:59:34,976 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:59:35,011 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0f6a09d19175818253217 HTTP/1.1" 200 None 2026-03-13 04:59:35,012 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:59:35,025 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ada13b8fc97e5ba07d4d75 HTTP/1.1" 200 None 2026-03-13 04:59:35,027 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 04:59:35,052 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0f6a09d19175818253217 HTTP/1.1" 200 None 2026-03-13 04:59:35,061 - lava_callback - ERROR - Error processing callback for node 69b0f6a09d19175818253217: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:38740 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58674 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36302 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50374 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43034 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43042 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43056 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55016 - "GET / HTTP/1.0" 200 OK 2026-03-13 05:00:34,958 - lava_callback - INFO - Received callback for node 69b0f6e99d19175818253231 INFO: 172.20.0.1:34834 - "POST /node/69b0f6e99d19175818253231 HTTP/1.0" 202 Accepted 2026-03-13 05:00:34,963 - lava_callback - INFO - Starting async job submit for node 69b0f6e99d19175818253231 2026-03-13 05:00:34,964 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:00:35,032 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0f6e99d19175818253231 HTTP/1.1" 200 None 2026-03-13 05:00:35,033 - lava_callback - DEBUG - Processing node 69b0f6e99d19175818253231: baseline-arm-clabbe 2026-03-13 05:00:35,033 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0f6e99d19175818253231 2026-03-13 05:00:35,034 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:00:35,058 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0f6e99d19175818253231 HTTP/1.1" 200 None 2026-03-13 05:00:35,059 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:00:35,072 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ada13b8fc97e5ba07d4d75 HTTP/1.1" 200 None 2026-03-13 05:00:35,074 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:00:35,118 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0f6e99d19175818253231 HTTP/1.1" 200 None 2026-03-13 05:00:35,129 - lava_callback - ERROR - Error processing callback for node 69b0f6e99d19175818253231: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:34850 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54696 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44742 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34656 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39548 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39550 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39560 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42478 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38162 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43302 - "GET / HTTP/1.0" 200 OK 2026-03-13 05:01:54,991 - lava_callback - INFO - Received callback for node 69b0f7359d19175818253244 INFO: 172.20.0.1:45344 - "POST /node/69b0f7359d19175818253244 HTTP/1.0" 202 Accepted 2026-03-13 05:01:54,997 - lava_callback - INFO - Starting async job submit for node 69b0f7359d19175818253244 2026-03-13 05:01:54,997 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:01:55,013 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0f7359d19175818253244 HTTP/1.1" 200 None 2026-03-13 05:01:55,014 - lava_callback - DEBUG - Processing node 69b0f7359d19175818253244: baseline-arm-clabbe 2026-03-13 05:01:55,014 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0f7359d19175818253244 2026-03-13 05:01:55,014 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:01:55,027 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0f7359d19175818253244 HTTP/1.1" 200 None 2026-03-13 05:01:55,029 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:01:55,042 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ada13b8fc97e5ba07d4d75 HTTP/1.1" 200 None 2026-03-13 05:01:55,043 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:01:55,064 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0f7359d19175818253244 HTTP/1.1" 200 None 2026-03-13 05:01:55,065 - lava_callback - ERROR - Error processing callback for node 69b0f7359d19175818253244: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:45352 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52118 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53900 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53916 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53918 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45112 - "GET / HTTP/1.0" 200 OK 2026-03-13 05:02:35,000 - lava_callback - INFO - Received callback for node 69b0f75a9d19175818253255 INFO: 172.20.0.1:33506 - "POST /node/69b0f75a9d19175818253255 HTTP/1.0" 202 Accepted 2026-03-13 05:02:35,007 - lava_callback - INFO - Starting async job submit for node 69b0f75a9d19175818253255 2026-03-13 05:02:35,008 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:02:35,022 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0f75a9d19175818253255 HTTP/1.1" 200 None 2026-03-13 05:02:35,023 - lava_callback - DEBUG - Processing node 69b0f75a9d19175818253255: baseline-arm-clabbe 2026-03-13 05:02:35,023 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0f75a9d19175818253255 2026-03-13 05:02:35,024 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:02:35,037 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0f75a9d19175818253255 HTTP/1.1" 200 None 2026-03-13 05:02:35,038 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:02:35,057 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ada13b8fc97e5ba07d4d75 HTTP/1.1" 200 None 2026-03-13 05:02:35,059 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:02:35,084 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0f75a9d19175818253255 HTTP/1.1" 200 None 2026-03-13 05:02:35,085 - lava_callback - ERROR - Error processing callback for node 69b0f75a9d19175818253255: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:33512 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51370 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34066 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35942 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56812 - "GET / HTTP/1.0" 200 OK 2026-03-13 05:03:15,073 - lava_callback - INFO - Received callback for node 69b0f77f9d1917581825326b INFO: 172.20.0.1:56822 - "POST /node/69b0f77f9d1917581825326b HTTP/1.0" 202 Accepted 2026-03-13 05:03:15,078 - lava_callback - INFO - Starting async job submit for node 69b0f77f9d1917581825326b 2026-03-13 05:03:15,079 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:03:15,097 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0f77f9d1917581825326b HTTP/1.1" 200 None 2026-03-13 05:03:15,097 - lava_callback - DEBUG - Processing node 69b0f77f9d1917581825326b: baseline-arm-clabbe 2026-03-13 05:03:15,098 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0f77f9d1917581825326b 2026-03-13 05:03:15,098 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:03:15,111 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0f77f9d1917581825326b HTTP/1.1" 200 None 2026-03-13 05:03:15,113 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:03:15,133 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ada13b8fc97e5ba07d4d75 HTTP/1.1" 200 None 2026-03-13 05:03:15,135 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.20.0.1:56832 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-13 05:03:15,158 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0f77f9d1917581825326b HTTP/1.1" 200 None 2026-03-13 05:03:15,159 - lava_callback - ERROR - Error processing callback for node 69b0f77f9d1917581825326b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:56846 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34770 - "GET / HTTP/1.0" 200 OK 2026-03-13 05:03:35,091 - lava_callback - INFO - Received callback for node 69b0f7a49d19175818253281 INFO: 172.20.0.1:57976 - "POST /node/69b0f7a49d19175818253281 HTTP/1.0" 202 Accepted 2026-03-13 05:03:35,096 - lava_callback - INFO - Starting async job submit for node 69b0f7a49d19175818253281 2026-03-13 05:03:35,097 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:03:35,112 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0f7a49d19175818253281 HTTP/1.1" 200 None 2026-03-13 05:03:35,113 - lava_callback - DEBUG - Processing node 69b0f7a49d19175818253281: baseline-arm-clabbe 2026-03-13 05:03:35,114 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0f7a49d19175818253281 2026-03-13 05:03:35,114 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:03:35,153 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0f7a49d19175818253281 HTTP/1.1" 200 None 2026-03-13 05:03:35,155 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:03:35,168 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ada13b8fc97e5ba07d4d75 HTTP/1.1" 200 None 2026-03-13 05:03:35,170 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:03:35,199 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0f7a49d19175818253281 HTTP/1.1" 200 None 2026-03-13 05:03:35,204 - lava_callback - ERROR - Error processing callback for node 69b0f7a49d19175818253281: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:57978 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54796 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56242 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42104 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39414 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39420 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-13 05:04:15,408 - lava_callback - INFO - Received callback for node 69b0f7c89d1917581825328c INFO: 172.20.0.1:39422 - "POST /node/69b0f7c89d1917581825328c HTTP/1.0" 202 Accepted 2026-03-13 05:04:15,414 - lava_callback - INFO - Starting async job submit for node 69b0f7c89d1917581825328c 2026-03-13 05:04:15,416 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:04:15,432 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0f7c89d1917581825328c HTTP/1.1" 200 None 2026-03-13 05:04:15,433 - lava_callback - DEBUG - Processing node 69b0f7c89d1917581825328c: baseline-arm-clabbe 2026-03-13 05:04:15,433 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0f7c89d1917581825328c 2026-03-13 05:04:15,434 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:04:16,242 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0f7c89d1917581825328c HTTP/1.1" 200 None 2026-03-13 05:04:16,245 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:04:16,260 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ada13b8fc97e5ba07d4d75 HTTP/1.1" 200 None 2026-03-13 05:04:16,262 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:04:16,285 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0f7c89d1917581825328c HTTP/1.1" 200 None 2026-03-13 05:04:16,287 - lava_callback - ERROR - Error processing callback for node 69b0f7c89d1917581825328c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:39426 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43268 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53434 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49258 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42192 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35612 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44288 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44300 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44306 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59374 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36774 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33184 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57370 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55866 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44218 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44220 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44234 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55226 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51156 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48100 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52964 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37098 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53380 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53394 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53404 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60286 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43224 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37928 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46156 - "GET / HTTP/1.0" 200 OK 2026-03-13 05:08:00,086 - lava_callback - INFO - Received callback for node 69b0f8a59d191758182532a8 INFO: 172.20.0.1:36522 - "POST /node/69b0f8a59d191758182532a8 HTTP/1.0" 202 Accepted 2026-03-13 05:08:00,092 - lava_callback - INFO - Starting async job submit for node 69b0f8a59d191758182532a8 2026-03-13 05:08:00,092 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:08:00,106 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0f8a59d191758182532a8 HTTP/1.1" 200 None 2026-03-13 05:08:00,107 - lava_callback - DEBUG - Processing node 69b0f8a59d191758182532a8: baseline-arm64-baylibre 2026-03-13 05:08:00,108 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0f8a59d191758182532a8 2026-03-13 05:08:00,108 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:08:00,127 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0f8a59d191758182532a8 HTTP/1.1" 200 None 2026-03-13 05:08:00,129 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:08:00,143 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ada1488fc97e5ba07d4d8d HTTP/1.1" 200 None 2026-03-13 05:08:00,145 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:08:00,171 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0f8a59d191758182532a8 HTTP/1.1" 200 None 2026-03-13 05:08:00,172 - lava_callback - ERROR - Error processing callback for node 69b0f8a59d191758182532a8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:36528 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46468 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46480 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46492 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51068 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37080 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49008 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33416 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59094 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59374 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59376 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59378 - "GET / HTTP/1.0" 200 OK 2026-03-13 05:09:20,200 - lava_callback - INFO - Received callback for node 69b0f8f59d191758182532bd INFO: 172.20.0.1:34198 - "POST /node/69b0f8f59d191758182532bd HTTP/1.0" 202 Accepted 2026-03-13 05:09:20,207 - lava_callback - INFO - Starting async job submit for node 69b0f8f59d191758182532bd 2026-03-13 05:09:20,208 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:09:20,222 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0f8f59d191758182532bd HTTP/1.1" 200 None 2026-03-13 05:09:20,223 - lava_callback - DEBUG - Processing node 69b0f8f59d191758182532bd: baseline-arm64-baylibre 2026-03-13 05:09:20,224 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0f8f59d191758182532bd 2026-03-13 05:09:20,224 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:09:20,239 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0f8f59d191758182532bd HTTP/1.1" 200 None 2026-03-13 05:09:20,241 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:09:20,255 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ada1488fc97e5ba07d4d8d HTTP/1.1" 200 None 2026-03-13 05:09:20,257 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:09:20,279 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0f8f59d191758182532bd HTTP/1.1" 200 None 2026-03-13 05:09:20,281 - lava_callback - ERROR - Error processing callback for node 69b0f8f59d191758182532bd: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:34210 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44296 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59046 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44470 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36940 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35040 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35042 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35056 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50028 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57368 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34562 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49034 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57460 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47892 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47894 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47906 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45776 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44222 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43680 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40318 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46198 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49054 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49058 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-13 05:12:15,436 - lava_callback - INFO - Received callback for node 69b0f99f9d191758182532da INFO: 172.20.0.1:49062 - "POST /node/69b0f99f9d191758182532da HTTP/1.0" 202 Accepted 2026-03-13 05:12:15,441 - lava_callback - INFO - Starting async job submit for node 69b0f99f9d191758182532da 2026-03-13 05:12:15,443 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:12:15,478 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0f99f9d191758182532da HTTP/1.1" 200 None 2026-03-13 05:12:15,478 - lava_callback - DEBUG - Processing node 69b0f99f9d191758182532da: baseline-arm64-clabbe 2026-03-13 05:12:15,479 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0f99f9d191758182532da 2026-03-13 05:12:15,479 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:12:15,494 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0f99f9d191758182532da HTTP/1.1" 200 None 2026-03-13 05:12:15,495 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:12:15,509 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ada1488fc97e5ba07d4d8d HTTP/1.1" 200 None 2026-03-13 05:12:15,510 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:12:15,531 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0f99f9d191758182532da HTTP/1.1" 200 None 2026-03-13 05:12:15,536 - lava_callback - ERROR - Error processing callback for node 69b0f99f9d191758182532da: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:49074 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40820 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50530 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39904 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42796 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41642 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35148 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35152 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35166 - "GET / HTTP/1.0" 200 OK 2026-03-13 05:13:20,376 - lava_callback - INFO - Received callback for node 69b0f9e99d191758182532ee INFO: 172.20.0.1:56422 - "POST /node/69b0f9e99d191758182532ee HTTP/1.0" 202 Accepted 2026-03-13 05:13:20,381 - lava_callback - INFO - Starting async job submit for node 69b0f9e99d191758182532ee 2026-03-13 05:13:20,382 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:13:20,424 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0f9e99d191758182532ee HTTP/1.1" 200 None 2026-03-13 05:13:20,426 - lava_callback - DEBUG - Processing node 69b0f9e99d191758182532ee: baseline-arm64-baylibre 2026-03-13 05:13:20,426 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0f9e99d191758182532ee 2026-03-13 05:13:20,427 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:13:20,449 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0f9e99d191758182532ee HTTP/1.1" 200 None 2026-03-13 05:13:20,451 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:13:20,470 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ada1488fc97e5ba07d4d8d HTTP/1.1" 200 None 2026-03-13 05:13:20,472 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:13:20,492 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0f9e99d191758182532ee HTTP/1.1" 200 None 2026-03-13 05:13:20,500 - lava_callback - ERROR - Error processing callback for node 69b0f9e99d191758182532ee: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:56434 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55534 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39666 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35100 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42604 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45218 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45220 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45236 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49386 - "GET / HTTP/1.0" 200 OK 2026-03-13 05:14:35,544 - lava_callback - INFO - Received callback for node 69b0fa389d19175818253306 INFO: 172.20.0.1:38308 - "POST /node/69b0fa389d19175818253306 HTTP/1.0" 202 Accepted 2026-03-13 05:14:35,550 - lava_callback - INFO - Starting async job submit for node 69b0fa389d19175818253306 2026-03-13 05:14:35,551 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:14:35,568 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0fa389d19175818253306 HTTP/1.1" 200 None 2026-03-13 05:14:35,569 - lava_callback - DEBUG - Processing node 69b0fa389d19175818253306: baseline-arm64-clabbe 2026-03-13 05:14:35,569 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0fa389d19175818253306 2026-03-13 05:14:35,570 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:14:35,600 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0fa389d19175818253306 HTTP/1.1" 200 None 2026-03-13 05:14:35,602 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:14:35,620 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ada1488fc97e5ba07d4d8d HTTP/1.1" 200 None 2026-03-13 05:14:35,622 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:14:35,642 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0fa389d19175818253306 HTTP/1.1" 200 None 2026-03-13 05:14:35,644 - lava_callback - ERROR - Error processing callback for node 69b0fa389d19175818253306: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:38318 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48434 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47004 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59230 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44806 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44822 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44834 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46076 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58466 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51414 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58456 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33814 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47122 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47128 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47136 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48498 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34024 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42096 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58706 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50118 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52318 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52322 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-13 05:17:15,660 - lava_callback - INFO - Received callback for node 69b0fad39d19175818253326 INFO: 172.20.0.1:52330 - "POST /node/69b0fad39d19175818253326 HTTP/1.0" 202 Accepted 2026-03-13 05:17:15,665 - lava_callback - INFO - Starting async job submit for node 69b0fad39d19175818253326 2026-03-13 05:17:15,666 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:17:15,681 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0fad39d19175818253326 HTTP/1.1" 200 None 2026-03-13 05:17:15,682 - lava_callback - DEBUG - Processing node 69b0fad39d19175818253326: baseline-arm64-clabbe 2026-03-13 05:17:15,682 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0fad39d19175818253326 2026-03-13 05:17:15,683 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:17:15,718 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0fad39d19175818253326 HTTP/1.1" 200 None 2026-03-13 05:17:15,720 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:17:15,734 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ada1488fc97e5ba07d4d8d HTTP/1.1" 200 None 2026-03-13 05:17:15,735 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:17:15,762 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0fad39d19175818253326 HTTP/1.1" 200 None 2026-03-13 05:17:15,768 - lava_callback - ERROR - Error processing callback for node 69b0fad39d19175818253326: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:52334 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56358 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60354 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50472 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45884 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51282 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39710 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39726 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39732 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58478 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41820 - "GET / HTTP/1.0" 200 OK 2026-03-13 05:18:40,730 - lava_callback - INFO - Received callback for node 69b0fb1f9d19175818253330 INFO: 172.20.0.1:60988 - "POST /node/69b0fb1f9d19175818253330 HTTP/1.0" 202 Accepted 2026-03-13 05:18:40,738 - lava_callback - INFO - Starting async job submit for node 69b0fb1f9d19175818253330 2026-03-13 05:18:40,739 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:18:40,753 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0fb1f9d19175818253330 HTTP/1.1" 200 None 2026-03-13 05:18:40,754 - lava_callback - DEBUG - Processing node 69b0fb1f9d19175818253330: baseline-arm64-baylibre 2026-03-13 05:18:40,754 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0fb1f9d19175818253330 2026-03-13 05:18:40,754 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:18:40,789 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0fb1f9d19175818253330 HTTP/1.1" 200 None 2026-03-13 05:18:40,790 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:18:40,804 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ada1488fc97e5ba07d4d8d HTTP/1.1" 200 None 2026-03-13 05:18:40,806 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:18:40,830 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0fb1f9d19175818253330 HTTP/1.1" 200 None 2026-03-13 05:18:40,836 - lava_callback - ERROR - Error processing callback for node 69b0fb1f9d19175818253330: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:60994 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36358 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49848 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47094 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47096 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47102 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55708 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47450 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60088 - "GET / HTTP/1.0" 200 OK 2026-03-13 05:19:55,789 - lava_callback - INFO - Received callback for node 69b0fb759d19175818253341 INFO: 172.20.0.1:35716 - "POST /node/69b0fb759d19175818253341 HTTP/1.0" 202 Accepted 2026-03-13 05:19:55,795 - lava_callback - INFO - Starting async job submit for node 69b0fb759d19175818253341 2026-03-13 05:19:55,796 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:19:55,810 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0fb759d19175818253341 HTTP/1.1" 200 None 2026-03-13 05:19:55,811 - lava_callback - DEBUG - Processing node 69b0fb759d19175818253341: baseline-arm64-clabbe 2026-03-13 05:19:55,811 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0fb759d19175818253341 2026-03-13 05:19:55,811 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:19:55,843 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0fb759d19175818253341 HTTP/1.1" 200 None 2026-03-13 05:19:55,844 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:19:55,861 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ada1488fc97e5ba07d4d8d HTTP/1.1" 200 None 2026-03-13 05:19:55,863 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:19:55,892 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0fb759d19175818253341 HTTP/1.1" 200 None 2026-03-13 05:19:55,894 - lava_callback - ERROR - Error processing callback for node 69b0fb759d19175818253341: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:35724 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49640 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34962 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34970 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34978 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38406 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47158 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42492 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36422 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48028 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41216 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41230 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41242 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54662 - "GET / HTTP/1.0" 200 OK 2026-03-13 05:21:35,854 - lava_callback - INFO - Received callback for node 69b0fbcb9d19175818253353 INFO: 172.20.0.1:55068 - "POST /node/69b0fbcb9d19175818253353 HTTP/1.0" 202 Accepted 2026-03-13 05:21:35,859 - lava_callback - INFO - Starting async job submit for node 69b0fbcb9d19175818253353 2026-03-13 05:21:35,860 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:21:35,874 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0fbcb9d19175818253353 HTTP/1.1" 200 None 2026-03-13 05:21:35,875 - lava_callback - DEBUG - Processing node 69b0fbcb9d19175818253353: baseline-arm64-clabbe 2026-03-13 05:21:35,875 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0fbcb9d19175818253353 2026-03-13 05:21:35,876 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:21:35,891 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0fbcb9d19175818253353 HTTP/1.1" 200 None 2026-03-13 05:21:35,893 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:21:35,908 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ada1488fc97e5ba07d4d8d HTTP/1.1" 200 None 2026-03-13 05:21:35,909 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:21:35,943 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0fbcb9d19175818253353 HTTP/1.1" 200 None 2026-03-13 05:21:35,945 - lava_callback - ERROR - Error processing callback for node 69b0fbcb9d19175818253353: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:55070 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49022 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58590 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38510 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56426 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56440 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56446 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56556 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56856 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44876 - "GET / HTTP/1.0" 200 OK 2026-03-13 05:22:55,896 - lava_callback - INFO - Received callback for node 69b0fc1c9d19175818253368 INFO: 172.20.0.1:54106 - "POST /node/69b0fc1c9d19175818253368 HTTP/1.0" 202 Accepted 2026-03-13 05:22:55,901 - lava_callback - INFO - Starting async job submit for node 69b0fc1c9d19175818253368 2026-03-13 05:22:55,902 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:22:55,917 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0fc1c9d19175818253368 HTTP/1.1" 200 None 2026-03-13 05:22:55,918 - lava_callback - DEBUG - Processing node 69b0fc1c9d19175818253368: baseline-arm64-clabbe 2026-03-13 05:22:55,918 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0fc1c9d19175818253368 2026-03-13 05:22:55,919 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:22:55,962 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0fc1c9d19175818253368 HTTP/1.1" 200 None 2026-03-13 05:22:55,964 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:22:55,978 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ada1488fc97e5ba07d4d8d HTTP/1.1" 200 None 2026-03-13 05:22:55,979 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:22:56,000 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0fc1c9d19175818253368 HTTP/1.1" 200 None 2026-03-13 05:22:56,006 - lava_callback - ERROR - Error processing callback for node 69b0fc1c9d19175818253368: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:54118 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37718 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45208 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45218 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45228 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48388 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55970 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54798 - "GET / HTTP/1.0" 200 OK 2026-03-13 05:23:55,964 - lava_callback - INFO - Received callback for node 69b0fc699d19175818253370 INFO: 172.20.0.1:39822 - "POST /node/69b0fc699d19175818253370 HTTP/1.0" 202 Accepted 2026-03-13 05:23:55,969 - lava_callback - INFO - Starting async job submit for node 69b0fc699d19175818253370 2026-03-13 05:23:55,970 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:23:56,009 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0fc699d19175818253370 HTTP/1.1" 200 None 2026-03-13 05:23:56,010 - lava_callback - DEBUG - Processing node 69b0fc699d19175818253370: baseline-arm64-clabbe 2026-03-13 05:23:56,010 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0fc699d19175818253370 2026-03-13 05:23:56,011 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:23:56,025 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0fc699d19175818253370 HTTP/1.1" 200 None 2026-03-13 05:23:56,026 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:23:56,039 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ada1488fc97e5ba07d4d8d HTTP/1.1" 200 None 2026-03-13 05:23:56,040 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:23:56,061 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0fc699d19175818253370 HTTP/1.1" 200 None 2026-03-13 05:23:56,062 - lava_callback - ERROR - Error processing callback for node 69b0fc699d19175818253370: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:39830 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57594 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33348 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33350 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33352 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60914 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41216 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40902 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60786 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44008 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39504 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39514 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-13 05:25:16,285 - lava_callback - INFO - Received callback for node 69b0fcb39d19175818253389 INFO: 172.20.0.1:39524 - "POST /node/69b0fcb39d19175818253389 HTTP/1.0" 202 Accepted 2026-03-13 05:25:16,290 - lava_callback - INFO - Starting async job submit for node 69b0fcb39d19175818253389 2026-03-13 05:25:16,291 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:25:16,313 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0fcb39d19175818253389 HTTP/1.1" 200 None 2026-03-13 05:25:16,314 - lava_callback - DEBUG - Processing node 69b0fcb39d19175818253389: baseline-arm64-clabbe 2026-03-13 05:25:16,314 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0fcb39d19175818253389 2026-03-13 05:25:16,315 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:25:16,347 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0fcb39d19175818253389 HTTP/1.1" 200 None 2026-03-13 05:25:16,348 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:25:16,370 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ada1488fc97e5ba07d4d8d HTTP/1.1" 200 None 2026-03-13 05:25:16,371 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:25:16,405 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0fcb39d19175818253389 HTTP/1.1" 200 None 2026-03-13 05:25:16,417 - lava_callback - ERROR - Error processing callback for node 69b0fcb39d19175818253389: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:39528 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48708 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38952 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47260 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46590 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53524 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56128 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56138 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56140 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35158 - "GET / HTTP/1.0" 200 OK 2026-03-13 05:26:36,048 - lava_callback - INFO - Received callback for node 69b0fd019d19175818253393 INFO: 172.20.0.1:48026 - "POST /node/69b0fd019d19175818253393 HTTP/1.0" 202 Accepted 2026-03-13 05:26:36,055 - lava_callback - INFO - Starting async job submit for node 69b0fd019d19175818253393 2026-03-13 05:26:36,056 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:26:36,092 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0fd019d19175818253393 HTTP/1.1" 200 None 2026-03-13 05:26:36,093 - lava_callback - DEBUG - Processing node 69b0fd019d19175818253393: baseline-arm64-clabbe 2026-03-13 05:26:36,093 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0fd019d19175818253393 2026-03-13 05:26:36,094 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:26:36,108 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0fd019d19175818253393 HTTP/1.1" 200 None 2026-03-13 05:26:36,109 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:26:36,138 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ada1488fc97e5ba07d4d8d HTTP/1.1" 200 None 2026-03-13 05:26:36,140 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:26:36,161 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0fd019d19175818253393 HTTP/1.1" 200 None 2026-03-13 05:26:36,166 - lava_callback - ERROR - Error processing callback for node 69b0fd019d19175818253393: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:48034 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48370 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60350 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50822 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54394 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54408 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54410 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58788 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38624 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55472 - "GET / HTTP/1.0" 200 OK 2026-03-13 05:27:56,078 - lava_callback - INFO - Received callback for node 69b0fd519d191758182533a7 INFO: 172.20.0.1:55300 - "POST /node/69b0fd519d191758182533a7 HTTP/1.0" 202 Accepted 2026-03-13 05:27:56,084 - lava_callback - INFO - Starting async job submit for node 69b0fd519d191758182533a7 2026-03-13 05:27:56,084 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:27:56,100 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0fd519d191758182533a7 HTTP/1.1" 200 None 2026-03-13 05:27:56,100 - lava_callback - DEBUG - Processing node 69b0fd519d191758182533a7: baseline-arm64-clabbe 2026-03-13 05:27:56,101 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0fd519d191758182533a7 2026-03-13 05:27:56,101 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:27:56,113 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0fd519d191758182533a7 HTTP/1.1" 200 None 2026-03-13 05:27:56,115 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:27:56,144 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ada1488fc97e5ba07d4d8d HTTP/1.1" 200 None 2026-03-13 05:27:56,145 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:27:56,168 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0fd519d191758182533a7 HTTP/1.1" 200 None 2026-03-13 05:27:56,179 - lava_callback - ERROR - Error processing callback for node 69b0fd519d191758182533a7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:55314 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43576 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35856 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35870 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35872 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55204 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57344 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52376 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40362 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48502 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37116 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37122 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-13 05:29:16,128 - lava_callback - INFO - Received callback for node 69b0fda39d191758182533be INFO: 172.20.0.1:37132 - "POST /node/69b0fda39d191758182533be HTTP/1.0" 202 Accepted 2026-03-13 05:29:16,134 - lava_callback - INFO - Starting async job submit for node 69b0fda39d191758182533be 2026-03-13 05:29:16,135 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:29:16,183 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0fda39d191758182533be HTTP/1.1" 200 None 2026-03-13 05:29:16,184 - lava_callback - DEBUG - Processing node 69b0fda39d191758182533be: baseline-arm64-clabbe 2026-03-13 05:29:16,184 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0fda39d191758182533be 2026-03-13 05:29:16,185 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:29:16,202 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0fda39d191758182533be HTTP/1.1" 200 None 2026-03-13 05:29:16,203 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:29:16,216 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ada1488fc97e5ba07d4d8d HTTP/1.1" 200 None 2026-03-13 05:29:16,218 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:29:16,246 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0fda39d191758182533be HTTP/1.1" 200 None 2026-03-13 05:29:16,247 - lava_callback - ERROR - Error processing callback for node 69b0fda39d191758182533be: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:37142 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35564 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34626 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36992 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32886 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56056 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40762 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40778 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40788 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58528 - "GET / HTTP/1.0" 200 OK 2026-03-13 05:30:36,220 - lava_callback - INFO - Received callback for node 69b0fdf49d191758182533c5 INFO: 172.20.0.1:33394 - "POST /node/69b0fdf49d191758182533c5 HTTP/1.0" 202 Accepted 2026-03-13 05:30:36,226 - lava_callback - INFO - Starting async job submit for node 69b0fdf49d191758182533c5 2026-03-13 05:30:36,227 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:30:36,245 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0fdf49d191758182533c5 HTTP/1.1" 200 None 2026-03-13 05:30:36,246 - lava_callback - DEBUG - Processing node 69b0fdf49d191758182533c5: baseline-arm64-clabbe 2026-03-13 05:30:36,246 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0fdf49d191758182533c5 2026-03-13 05:30:36,247 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:30:36,281 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0fdf49d191758182533c5 HTTP/1.1" 200 None 2026-03-13 05:30:36,283 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:30:36,321 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ada1488fc97e5ba07d4d8d HTTP/1.1" 200 None 2026-03-13 05:30:36,322 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:30:36,343 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0fdf49d191758182533c5 HTTP/1.1" 200 None 2026-03-13 05:30:36,345 - lava_callback - ERROR - Error processing callback for node 69b0fdf49d191758182533c5: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:33408 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38082 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35494 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51412 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39850 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39854 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39856 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40898 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46366 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40858 - "GET / HTTP/1.0" 200 OK 2026-03-13 05:31:56,311 - lava_callback - INFO - Received callback for node 69b0fe479d191758182533d8 INFO: 172.20.0.1:51484 - "POST /node/69b0fe479d191758182533d8 HTTP/1.0" 202 Accepted 2026-03-13 05:31:56,316 - lava_callback - INFO - Starting async job submit for node 69b0fe479d191758182533d8 2026-03-13 05:31:56,317 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:31:56,344 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0fe479d191758182533d8 HTTP/1.1" 200 None 2026-03-13 05:31:56,346 - lava_callback - DEBUG - Processing node 69b0fe479d191758182533d8: baseline-arm64-clabbe 2026-03-13 05:31:56,346 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0fe479d191758182533d8 2026-03-13 05:31:56,348 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:31:56,371 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0fe479d191758182533d8 HTTP/1.1" 200 None 2026-03-13 05:31:56,373 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:31:56,390 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ada1488fc97e5ba07d4d8d HTTP/1.1" 200 None 2026-03-13 05:31:56,392 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:31:56,413 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0fe479d191758182533d8 HTTP/1.1" 200 None 2026-03-13 05:31:56,414 - lava_callback - ERROR - Error processing callback for node 69b0fe479d191758182533d8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:51490 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35878 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51634 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51636 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51642 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42336 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34008 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57862 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52848 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36200 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45430 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45440 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-13 05:33:16,358 - lava_callback - INFO - Received callback for node 69b0fe949d191758182533ed INFO: 172.20.0.1:45448 - "POST /node/69b0fe949d191758182533ed HTTP/1.0" 202 Accepted 2026-03-13 05:33:16,364 - lava_callback - INFO - Starting async job submit for node 69b0fe949d191758182533ed 2026-03-13 05:33:16,365 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:33:16,379 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0fe949d191758182533ed HTTP/1.1" 200 None 2026-03-13 05:33:16,380 - lava_callback - DEBUG - Processing node 69b0fe949d191758182533ed: baseline-arm64-clabbe 2026-03-13 05:33:16,380 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0fe949d191758182533ed 2026-03-13 05:33:16,380 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:33:16,394 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0fe949d191758182533ed HTTP/1.1" 200 None 2026-03-13 05:33:16,395 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:33:16,408 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ada1488fc97e5ba07d4d8d HTTP/1.1" 200 None 2026-03-13 05:33:16,410 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:33:16,442 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0fe949d191758182533ed HTTP/1.1" 200 None 2026-03-13 05:33:16,444 - lava_callback - ERROR - Error processing callback for node 69b0fe949d191758182533ed: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:45454 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47050 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56888 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46792 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46062 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51844 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54092 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54094 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54096 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39318 - "GET / HTTP/1.0" 200 OK 2026-03-13 05:34:36,416 - lava_callback - INFO - Received callback for node 69b0fedd9d191758182533f5 INFO: 172.20.0.1:52692 - "POST /node/69b0fedd9d191758182533f5 HTTP/1.0" 202 Accepted 2026-03-13 05:34:36,421 - lava_callback - INFO - Starting async job submit for node 69b0fedd9d191758182533f5 2026-03-13 05:34:36,422 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:34:36,445 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0fedd9d191758182533f5 HTTP/1.1" 200 None 2026-03-13 05:34:36,446 - lava_callback - DEBUG - Processing node 69b0fedd9d191758182533f5: baseline-arm64-clabbe 2026-03-13 05:34:36,446 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0fedd9d191758182533f5 2026-03-13 05:34:36,447 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:34:36,460 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0fedd9d191758182533f5 HTTP/1.1" 200 None 2026-03-13 05:34:36,462 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:34:36,475 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ada1488fc97e5ba07d4d8d HTTP/1.1" 200 None 2026-03-13 05:34:36,477 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:34:36,497 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0fedd9d191758182533f5 HTTP/1.1" 200 None 2026-03-13 05:34:36,498 - lava_callback - ERROR - Error processing callback for node 69b0fedd9d191758182533f5: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:52706 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46968 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36202 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48284 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59510 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59520 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59528 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57990 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39482 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60314 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51028 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37414 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50630 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50634 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50636 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36200 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43764 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35436 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57830 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37020 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44636 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44650 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44656 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50732 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39464 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47056 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39614 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34766 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46150 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46158 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46166 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45760 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45754 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39856 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34354 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52314 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53042 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53048 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53060 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39442 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34062 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51754 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40560 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49068 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57342 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57344 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57354 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53830 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37986 - "GET / HTTP/1.0" 200 OK 2026-03-13 05:40:39,973 - lava_callback - INFO - Received callback for node 69b363d1974c91e64560ed6a INFO: 172.20.0.1:45764 - "POST /node/69b363d1974c91e64560ed6a HTTP/1.0" 202 Accepted 2026-03-13 05:40:39,979 - lava_callback - INFO - Starting async job submit for node 69b363d1974c91e64560ed6a 2026-03-13 05:40:40,003 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:40:40,035 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b363d1974c91e64560ed6a HTTP/1.1" 200 None 2026-03-13 05:40:40,036 - lava_callback - DEBUG - Processing node 69b363d1974c91e64560ed6a: baseline-arm64-broonie 2026-03-13 05:40:40,318 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 05:40:40,480 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 05:40:40,482 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69b363d1974c91e64560ed6a/log.txt.gz 2026-03-13 05:40:40,639 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 05:40:40,886 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 05:40:40,889 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69b363d1974c91e64560ed6a/lava-logs.yaml 2026-03-13 05:40:40,897 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 05:40:41,052 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 05:40:41,054 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69b363d1974c91e64560ed6a/lava_callback.json.gz 2026-03-13 05:40:41,077 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:40:41,091 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b363d1974c91e64560ed6a HTTP/1.1" 200 None 2026-03-13 05:40:41,093 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:40:41,106 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3594d974c91e64560e4b2 HTTP/1.1" 200 None 2026-03-13 05:40:41,108 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:40:41,139 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b363d1974c91e64560ed6a HTTP/1.1" 200 None 2026-03-13 05:40:41,143 - lava_callback - INFO - Completed processing callback for node 69b363d1974c91e64560ed6a 2026-03-13 05:40:46,422 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 05:40:46,439 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:45774 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46260 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43942 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38842 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38856 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38862 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47142 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52310 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47930 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40304 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60808 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56702 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56710 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56718 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58800 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54990 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33302 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49244 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48586 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60108 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60120 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60126 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37378 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51834 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40790 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42362 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55128 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47596 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47612 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47624 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40784 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46468 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54214 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56834 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39916 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49948 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49958 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49960 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54104 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38760 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46340 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42638 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34718 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55304 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55310 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55320 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40424 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55356 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42988 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52798 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59232 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56566 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56582 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56590 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41566 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44422 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36304 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41048 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42374 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38316 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38322 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38326 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33144 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49790 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55336 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38228 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36682 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36606 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36610 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36612 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58242 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35608 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60866 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44290 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46120 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53390 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53392 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53402 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49646 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48602 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33548 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50486 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58676 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51266 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51274 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51276 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55880 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42794 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59616 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47126 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34008 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42284 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42286 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42296 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49418 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46192 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57120 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44580 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40040 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37120 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37136 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37138 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44032 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36678 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49164 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44670 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36056 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56868 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56880 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56894 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57902 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48584 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34676 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38300 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43586 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52316 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52322 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52332 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35696 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55252 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57902 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42980 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43398 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48226 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48238 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48240 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58846 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58002 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57598 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45888 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43702 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35058 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35074 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35082 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34196 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52292 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46636 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37942 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59616 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38392 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38398 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38412 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46010 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43262 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40454 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48090 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52696 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33952 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33956 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33964 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59072 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49284 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58100 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59122 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45916 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47482 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47496 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47500 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58420 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58150 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48764 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37732 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41788 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48642 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48650 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48654 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34192 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48054 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34304 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45394 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42822 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45852 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45864 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45880 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56168 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49712 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35508 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43370 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33576 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47814 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47830 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47832 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53372 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38654 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39494 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58382 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33352 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53022 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53030 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53038 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40160 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48080 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54246 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55922 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36910 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37072 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37084 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37092 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38750 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43254 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54878 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38506 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54854 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41798 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41800 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41808 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35882 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47450 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52358 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50000 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48782 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38362 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38378 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38392 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48928 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42324 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35130 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51044 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51472 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55964 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55972 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55976 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51096 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53306 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36004 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50346 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43042 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42250 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42262 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42276 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55898 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58344 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39828 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53122 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44560 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33898 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33904 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33916 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52288 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33044 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58692 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48140 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41810 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37402 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37410 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37414 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50626 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49962 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36904 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56582 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40998 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53688 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53696 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53704 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40672 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40084 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40004 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54476 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42580 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47850 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47854 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47864 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51130 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44452 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54000 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50008 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41390 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52240 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52252 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52260 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60082 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54850 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38530 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49148 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60792 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38060 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38066 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38078 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43728 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40798 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57182 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38224 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46008 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56216 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56226 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56238 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39576 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55710 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35412 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42730 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44406 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45260 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45262 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45266 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51404 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39078 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37998 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47190 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52054 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40520 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40528 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40532 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39272 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49184 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59542 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40712 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53458 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41370 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41384 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41386 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44468 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37200 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41708 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51074 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54696 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39876 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39884 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39888 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40394 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57146 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56834 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35556 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60824 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55454 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55464 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55474 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34742 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45414 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55050 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45394 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40132 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60314 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60326 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60330 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46466 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53298 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45696 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50516 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58188 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48022 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48024 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48038 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38422 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41456 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41264 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58788 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47404 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46448 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46456 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46464 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49578 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49380 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54026 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38850 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58456 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52984 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52996 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52998 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58518 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56574 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48950 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59856 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37610 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60952 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60964 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60976 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60990 - "GET /node/69a62d087da105ad098359e3%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:37116 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36532 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57208 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50186 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51904 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48702 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48712 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48720 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33404 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59920 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53424 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47802 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44618 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42398 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42412 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42428 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33840 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39082 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60500 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45226 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40428 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33096 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33112 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33116 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58342 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57118 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54716 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33756 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35506 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52018 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52024 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52036 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34822 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38118 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56352 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43070 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57756 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49302 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49304 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49314 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44250 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44726 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51876 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56554 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48372 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51836 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51838 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51844 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49604 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47782 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36586 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53894 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54768 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44244 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44256 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44262 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45040 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41056 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47726 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53890 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52088 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43180 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43196 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43204 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58930 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55100 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56180 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37140 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57104 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51258 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51266 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51280 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53870 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43798 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36388 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57230 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55014 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38092 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38098 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38114 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52454 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59760 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51372 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45330 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38222 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36100 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36106 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36114 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53538 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33724 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55704 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54120 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44128 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53706 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53718 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53722 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60994 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60076 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51254 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51054 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43842 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50208 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50212 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50222 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43494 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34326 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60248 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56456 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60228 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45846 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45848 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45856 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58600 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57424 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37428 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44338 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45152 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38108 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38110 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38122 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33054 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32882 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59244 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54428 - "GET / HTTP/1.0" 200 OK 2026-03-13 06:42:05,487 - lava_callback - INFO - Received callback for node 69b10eb69d19175818253449 INFO: 172.20.0.1:58122 - "POST /node/69b10eb69d19175818253449 HTTP/1.0" 202 Accepted 2026-03-13 06:42:05,493 - lava_callback - INFO - Starting async job submit for node 69b10eb69d19175818253449 2026-03-13 06:42:05,494 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 06:42:05,509 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b10eb69d19175818253449 HTTP/1.1" 200 None 2026-03-13 06:42:05,510 - lava_callback - DEBUG - Processing node 69b10eb69d19175818253449: baseline-arm-baylibre 2026-03-13 06:42:05,510 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b10eb69d19175818253449 2026-03-13 06:42:05,511 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 06:42:05,524 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b10eb69d19175818253449 HTTP/1.1" 200 None 2026-03-13 06:42:05,526 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 06:42:05,539 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b4fc9d19175818251fd6 HTTP/1.1" 200 None 2026-03-13 06:42:05,540 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 06:42:05,561 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b10eb69d19175818253449 HTTP/1.1" 200 None 2026-03-13 06:42:05,570 - lava_callback - ERROR - Error processing callback for node 69b10eb69d19175818253449: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:58136 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43436 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43438 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43446 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58828 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44702 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41034 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43742 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38242 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41546 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41556 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41558 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38082 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49516 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40114 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49252 - "GET / HTTP/1.0" 200 OK 2026-03-13 06:43:59,341 - lava_callback - INFO - Received callback for node 69b10f249d19175818253473 INFO: 172.20.0.1:49266 - "POST /node/69b10f249d19175818253473 HTTP/1.0" 202 Accepted 2026-03-13 06:43:59,347 - lava_callback - INFO - Starting async job submit for node 69b10f249d19175818253473 2026-03-13 06:43:59,348 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 06:43:59,362 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b10f249d19175818253473 HTTP/1.1" 200 None 2026-03-13 06:43:59,363 - lava_callback - DEBUG - Processing node 69b10f249d19175818253473: baseline-arm-clabbe 2026-03-13 06:43:59,364 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b10f249d19175818253473 2026-03-13 06:43:59,364 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 06:43:59,378 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b10f249d19175818253473 HTTP/1.1" 200 None 2026-03-13 06:43:59,379 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 06:43:59,394 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b4fc9d19175818251fd6 HTTP/1.1" 200 None 2026-03-13 06:43:59,396 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 06:43:59,421 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b10f249d19175818253473 HTTP/1.1" 200 None 2026-03-13 06:43:59,422 - lava_callback - ERROR - Error processing callback for node 69b10f249d19175818253473: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:34406 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51630 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51636 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51650 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46972 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46270 - "GET / HTTP/1.0" 200 OK 2026-03-13 06:44:39,335 - lava_callback - INFO - Received callback for node 69b10f4a9d19175818253481 INFO: 172.20.0.1:46284 - "POST /node/69b10f4a9d19175818253481 HTTP/1.0" 202 Accepted 2026-03-13 06:44:39,341 - lava_callback - INFO - Starting async job submit for node 69b10f4a9d19175818253481 2026-03-13 06:44:39,341 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 06:44:39,355 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b10f4a9d19175818253481 HTTP/1.1" 200 None 2026-03-13 06:44:39,356 - lava_callback - DEBUG - Processing node 69b10f4a9d19175818253481: baseline-arm-clabbe 2026-03-13 06:44:39,357 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b10f4a9d19175818253481 2026-03-13 06:44:39,357 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 06:44:39,398 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b10f4a9d19175818253481 HTTP/1.1" 200 None 2026-03-13 06:44:39,400 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 06:44:39,415 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b4fc9d19175818251fd6 HTTP/1.1" 200 None 2026-03-13 06:44:39,417 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 06:44:39,439 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b10f4a9d19175818253481 HTTP/1.1" 200 None 2026-03-13 06:44:39,445 - lava_callback - ERROR - Error processing callback for node 69b10f4a9d19175818253481: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:39474 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47358 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40542 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56896 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56902 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56912 - "GET / HTTP/1.0" 200 OK 2026-03-13 06:45:19,383 - lava_callback - INFO - Received callback for node 69b10f6e9d19175818253491 INFO: 172.20.0.1:56924 - "POST /node/69b10f6e9d19175818253491 HTTP/1.0" 202 Accepted 2026-03-13 06:45:19,389 - lava_callback - INFO - Starting async job submit for node 69b10f6e9d19175818253491 2026-03-13 06:45:19,390 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 06:45:19,407 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b10f6e9d19175818253491 HTTP/1.1" 200 None 2026-03-13 06:45:19,408 - lava_callback - DEBUG - Processing node 69b10f6e9d19175818253491: baseline-arm-clabbe 2026-03-13 06:45:19,408 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b10f6e9d19175818253491 2026-03-13 06:45:19,408 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 06:45:19,423 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b10f6e9d19175818253491 HTTP/1.1" 200 None 2026-03-13 06:45:19,424 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 06:45:19,438 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b4fc9d19175818251fd6 HTTP/1.1" 200 None 2026-03-13 06:45:19,439 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 06:45:19,459 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b10f6e9d19175818253491 HTTP/1.1" 200 None 2026-03-13 06:45:19,461 - lava_callback - ERROR - Error processing callback for node 69b10f6e9d19175818253491: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:46378 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57132 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46674 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42722 - "GET / HTTP/1.0" 200 OK 2026-03-13 06:45:59,385 - lava_callback - INFO - Received callback for node 69b10f929d191758182534a3 INFO: 172.20.0.1:42724 - "POST /node/69b10f929d191758182534a3 HTTP/1.0" 202 Accepted 2026-03-13 06:45:59,390 - lava_callback - INFO - Starting async job submit for node 69b10f929d191758182534a3 2026-03-13 06:45:59,391 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 06:45:59,405 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b10f929d191758182534a3 HTTP/1.1" 200 None 2026-03-13 06:45:59,406 - lava_callback - DEBUG - Processing node 69b10f929d191758182534a3: baseline-arm-clabbe 2026-03-13 06:45:59,406 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b10f929d191758182534a3 2026-03-13 06:45:59,407 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 06:45:59,423 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b10f929d191758182534a3 HTTP/1.1" 200 None 2026-03-13 06:45:59,424 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 06:45:59,439 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b4fc9d19175818251fd6 HTTP/1.1" 200 None 2026-03-13 06:45:59,440 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 06:45:59,460 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b10f929d191758182534a3 HTTP/1.1" 200 None 2026-03-13 06:45:59,461 - lava_callback - ERROR - Error processing callback for node 69b10f929d191758182534a3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:40968 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55900 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55912 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55926 - "GET / HTTP/1.0" 200 OK 2026-03-13 06:46:19,606 - lava_callback - INFO - Received callback for node 69b10fb79d191758182534b0 INFO: 172.20.0.1:55930 - "POST /node/69b10fb79d191758182534b0 HTTP/1.0" 202 Accepted 2026-03-13 06:46:19,611 - lava_callback - INFO - Starting async job submit for node 69b10fb79d191758182534b0 2026-03-13 06:46:19,612 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 06:46:19,627 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b10fb79d191758182534b0 HTTP/1.1" 200 None 2026-03-13 06:46:19,627 - lava_callback - DEBUG - Processing node 69b10fb79d191758182534b0: baseline-arm-clabbe 2026-03-13 06:46:19,628 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b10fb79d191758182534b0 2026-03-13 06:46:19,628 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 06:46:19,641 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b10fb79d191758182534b0 HTTP/1.1" 200 None 2026-03-13 06:46:19,643 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 06:46:19,656 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b4fc9d19175818251fd6 HTTP/1.1" 200 None 2026-03-13 06:46:19,657 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 06:46:19,678 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b10fb79d191758182534b0 HTTP/1.1" 200 None 2026-03-13 06:46:19,682 - lava_callback - ERROR - Error processing callback for node 69b10fb79d191758182534b0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:33864 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57290 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48858 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40760 - "GET / HTTP/1.0" 200 OK 2026-03-13 06:47:05,634 - lava_callback - INFO - Received callback for node 69b10fd79d191758182534be INFO: 172.20.0.1:59644 - "POST /node/69b10fd79d191758182534be HTTP/1.0" 202 Accepted 2026-03-13 06:47:05,639 - lava_callback - INFO - Starting async job submit for node 69b10fd79d191758182534be 2026-03-13 06:47:05,640 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 06:47:05,655 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b10fd79d191758182534be HTTP/1.1" 200 None 2026-03-13 06:47:05,656 - lava_callback - DEBUG - Processing node 69b10fd79d191758182534be: baseline-arm-baylibre 2026-03-13 06:47:05,656 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b10fd79d191758182534be 2026-03-13 06:47:05,656 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 06:47:05,689 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b10fd79d191758182534be HTTP/1.1" 200 None 2026-03-13 06:47:05,690 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 06:47:05,703 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11668ef4424e487b2ea2 HTTP/1.1" 200 None 2026-03-13 06:47:05,705 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 06:47:05,725 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b10fd79d191758182534be HTTP/1.1" 200 None 2026-03-13 06:47:05,726 - lava_callback - ERROR - Error processing callback for node 69b10fd79d191758182534be: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:59656 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55688 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55702 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55710 - "GET / HTTP/1.0" 200 OK 2026-03-13 06:47:25,590 - lava_callback - INFO - Received callback for node 69b10ffa9d191758182534cb INFO: 172.20.0.1:55512 - "POST /node/69b10ffa9d191758182534cb HTTP/1.0" 202 Accepted 2026-03-13 06:47:25,596 - lava_callback - INFO - Starting async job submit for node 69b10ffa9d191758182534cb 2026-03-13 06:47:25,597 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 06:47:25,612 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b10ffa9d191758182534cb HTTP/1.1" 200 None 2026-03-13 06:47:25,613 - lava_callback - DEBUG - Processing node 69b10ffa9d191758182534cb: baseline-arm-baylibre 2026-03-13 06:47:25,613 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b10ffa9d191758182534cb 2026-03-13 06:47:25,614 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 06:47:25,632 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b10ffa9d191758182534cb HTTP/1.1" 200 None 2026-03-13 06:47:25,633 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 06:47:25,647 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11668ef4424e487b2ea2 HTTP/1.1" 200 None 2026-03-13 06:47:25,649 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 06:47:25,679 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b10ffa9d191758182534cb HTTP/1.1" 200 None 2026-03-13 06:47:25,680 - lava_callback - ERROR - Error processing callback for node 69b10ffa9d191758182534cb: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:55526 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38850 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50162 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40498 - "GET / HTTP/1.0" 200 OK 2026-03-13 06:48:05,622 - lava_callback - INFO - Received callback for node 69b1101e9d191758182534dd INFO: 172.20.0.1:55216 - "POST /node/69b1101e9d191758182534dd HTTP/1.0" 202 Accepted 2026-03-13 06:48:05,627 - lava_callback - INFO - Starting async job submit for node 69b1101e9d191758182534dd 2026-03-13 06:48:05,628 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 06:48:05,644 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b1101e9d191758182534dd HTTP/1.1" 200 None 2026-03-13 06:48:05,645 - lava_callback - DEBUG - Processing node 69b1101e9d191758182534dd: baseline-arm-baylibre 2026-03-13 06:48:05,645 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b1101e9d191758182534dd 2026-03-13 06:48:05,645 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 06:48:05,658 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b1101e9d191758182534dd HTTP/1.1" 200 None 2026-03-13 06:48:05,660 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 06:48:05,673 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11668ef4424e487b2ea2 HTTP/1.1" 200 None 2026-03-13 06:48:05,674 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 06:48:05,694 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b1101e9d191758182534dd HTTP/1.1" 200 None 2026-03-13 06:48:05,700 - lava_callback - ERROR - Error processing callback for node 69b1101e9d191758182534dd: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:55226 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47706 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47720 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47726 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43658 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54198 - "GET / HTTP/1.0" 200 OK 2026-03-13 06:48:45,724 - lava_callback - INFO - Received callback for node 69b110439d191758182534e9 INFO: 172.20.0.1:33664 - "POST /node/69b110439d191758182534e9 HTTP/1.0" 202 Accepted 2026-03-13 06:48:45,730 - lava_callback - INFO - Starting async job submit for node 69b110439d191758182534e9 2026-03-13 06:48:45,731 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 06:48:45,745 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b110439d191758182534e9 HTTP/1.1" 200 None 2026-03-13 06:48:45,747 - lava_callback - DEBUG - Processing node 69b110439d191758182534e9: baseline-arm-baylibre 2026-03-13 06:48:45,747 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b110439d191758182534e9 2026-03-13 06:48:45,748 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 06:48:45,763 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b110439d191758182534e9 HTTP/1.1" 200 None 2026-03-13 06:48:45,765 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 06:48:45,781 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11668ef4424e487b2ea2 HTTP/1.1" 200 None 2026-03-13 06:48:45,783 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 06:48:45,809 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b110439d191758182534e9 HTTP/1.1" 200 None 2026-03-13 06:48:45,815 - lava_callback - ERROR - Error processing callback for node 69b110439d191758182534e9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:33678 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51466 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47454 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56286 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56296 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56302 - "GET / HTTP/1.0" 200 OK 2026-03-13 06:49:25,694 - lava_callback - INFO - Received callback for node 69b110689d191758182534f5 INFO: 172.20.0.1:47092 - "POST /node/69b110689d191758182534f5 HTTP/1.0" 202 Accepted 2026-03-13 06:49:25,700 - lava_callback - INFO - Starting async job submit for node 69b110689d191758182534f5 2026-03-13 06:49:25,701 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 06:49:25,786 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b110689d191758182534f5 HTTP/1.1" 200 None 2026-03-13 06:49:25,787 - lava_callback - DEBUG - Processing node 69b110689d191758182534f5: baseline-arm-baylibre 2026-03-13 06:49:25,787 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b110689d191758182534f5 2026-03-13 06:49:25,788 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 06:49:25,802 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b110689d191758182534f5 HTTP/1.1" 200 None 2026-03-13 06:49:25,803 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 06:49:25,816 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11668ef4424e487b2ea2 HTTP/1.1" 200 None 2026-03-13 06:49:25,818 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 06:49:25,839 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b110689d191758182534f5 HTTP/1.1" 200 None 2026-03-13 06:49:25,843 - lava_callback - ERROR - Error processing callback for node 69b110689d191758182534f5: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:47106 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60068 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53324 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59630 - "GET / HTTP/1.0" 200 OK 2026-03-13 06:50:05,750 - lava_callback - INFO - Received callback for node 69b110899d19175818253505 INFO: 172.20.0.1:46504 - "POST /node/69b110899d19175818253505 HTTP/1.0" 202 Accepted 2026-03-13 06:50:05,756 - lava_callback - INFO - Starting async job submit for node 69b110899d19175818253505 2026-03-13 06:50:05,757 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 06:50:05,781 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b110899d19175818253505 HTTP/1.1" 200 None 2026-03-13 06:50:05,782 - lava_callback - DEBUG - Processing node 69b110899d19175818253505: baseline-arm-baylibre 2026-03-13 06:50:05,782 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b110899d19175818253505 2026-03-13 06:50:05,783 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 06:50:05,805 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b110899d19175818253505 HTTP/1.1" 200 None 2026-03-13 06:50:05,806 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 06:50:05,820 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11668ef4424e487b2ea2 HTTP/1.1" 200 None 2026-03-13 06:50:05,822 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 06:50:05,842 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b110899d19175818253505 HTTP/1.1" 200 None 2026-03-13 06:50:05,848 - lava_callback - ERROR - Error processing callback for node 69b110899d19175818253505: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:46506 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32796 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32810 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:32826 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44466 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32982 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59288 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50356 - "GET / HTTP/1.0" 200 OK 2026-03-13 06:51:05,803 - lava_callback - INFO - Received callback for node 69b110d29d19175818253525 INFO: 172.20.0.1:38760 - "POST /node/69b110d29d19175818253525 HTTP/1.0" 202 Accepted 2026-03-13 06:51:05,809 - lava_callback - INFO - Starting async job submit for node 69b110d29d19175818253525 2026-03-13 06:51:05,810 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 06:51:05,832 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b110d29d19175818253525 HTTP/1.1" 200 None 2026-03-13 06:51:05,832 - lava_callback - DEBUG - Processing node 69b110d29d19175818253525: baseline-arm-baylibre 2026-03-13 06:51:05,833 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b110d29d19175818253525 2026-03-13 06:51:05,833 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 06:51:05,846 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b110d29d19175818253525 HTTP/1.1" 200 None 2026-03-13 06:51:05,848 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 06:51:05,862 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11668ef4424e487b2ea2 HTTP/1.1" 200 None 2026-03-13 06:51:05,864 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 06:51:05,892 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b110d29d19175818253525 HTTP/1.1" 200 None 2026-03-13 06:51:05,893 - lava_callback - ERROR - Error processing callback for node 69b110d29d19175818253525: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:38762 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38092 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38104 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38106 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42112 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51310 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53214 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56222 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47332 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51586 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51602 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51614 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40726 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59700 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36326 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49564 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60704 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46964 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46974 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46986 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47294 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57898 - "GET / HTTP/1.0" 200 OK 2026-03-13 06:53:39,717 - lava_callback - INFO - Received callback for node 69b1115f9d19175818253565 INFO: 172.20.0.1:59952 - "POST /node/69b1115f9d19175818253565 HTTP/1.0" 202 Accepted 2026-03-13 06:53:39,722 - lava_callback - INFO - Starting async job submit for node 69b1115f9d19175818253565 2026-03-13 06:53:39,723 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 06:53:39,741 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b1115f9d19175818253565 HTTP/1.1" 200 None 2026-03-13 06:53:39,742 - lava_callback - DEBUG - Processing node 69b1115f9d19175818253565: baseline-arm-clabbe 2026-03-13 06:53:39,742 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b1115f9d19175818253565 2026-03-13 06:53:39,743 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 06:53:39,759 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b1115f9d19175818253565 HTTP/1.1" 200 None 2026-03-13 06:53:39,761 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 06:53:39,775 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11668ef4424e487b2ea2 HTTP/1.1" 200 None 2026-03-13 06:53:39,776 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 06:53:39,797 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b1115f9d19175818253565 HTTP/1.1" 200 None 2026-03-13 06:53:39,806 - lava_callback - ERROR - Error processing callback for node 69b1115f9d19175818253565: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:59958 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42416 - "GET / HTTP/1.0" 200 OK 2026-03-13 06:53:59,782 - lava_callback - INFO - Received callback for node 69b111849d19175818253578 INFO: 172.20.0.1:40396 - "POST /node/69b111849d19175818253578 HTTP/1.0" 202 Accepted 2026-03-13 06:53:59,788 - lava_callback - INFO - Starting async job submit for node 69b111849d19175818253578 2026-03-13 06:53:59,789 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 06:53:59,812 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b111849d19175818253578 HTTP/1.1" 200 None 2026-03-13 06:53:59,813 - lava_callback - DEBUG - Processing node 69b111849d19175818253578: baseline-arm-clabbe 2026-03-13 06:53:59,813 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b111849d19175818253578 2026-03-13 06:53:59,813 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 06:53:59,838 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b111849d19175818253578 HTTP/1.1" 200 None 2026-03-13 06:53:59,839 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 06:53:59,852 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11668ef4424e487b2ea2 HTTP/1.1" 200 None 2026-03-13 06:53:59,854 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 06:53:59,875 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b111849d19175818253578 HTTP/1.1" 200 None 2026-03-13 06:53:59,886 - lava_callback - ERROR - Error processing callback for node 69b111849d19175818253578: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:40404 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59498 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59504 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59516 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57128 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52688 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44708 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35490 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50760 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55186 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55202 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55218 - "GET / HTTP/1.0" 200 OK 2026-03-13 06:55:19,763 - lava_callback - INFO - Received callback for node 69b111cb9d1917581825359b INFO: 172.20.0.1:54638 - "POST /node/69b111cb9d1917581825359b HTTP/1.0" 202 Accepted 2026-03-13 06:55:19,768 - lava_callback - INFO - Starting async job submit for node 69b111cb9d1917581825359b 2026-03-13 06:55:19,769 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 06:55:19,783 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b111cb9d1917581825359b HTTP/1.1" 200 None 2026-03-13 06:55:19,784 - lava_callback - DEBUG - Processing node 69b111cb9d1917581825359b: baseline-arm-clabbe 2026-03-13 06:55:19,785 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b111cb9d1917581825359b 2026-03-13 06:55:19,785 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 06:55:19,807 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b111cb9d1917581825359b HTTP/1.1" 200 None 2026-03-13 06:55:19,808 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 06:55:19,822 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11668ef4424e487b2ea2 HTTP/1.1" 200 None 2026-03-13 06:55:19,823 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 06:55:19,844 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b111cb9d1917581825359b HTTP/1.1" 200 None 2026-03-13 06:55:19,845 - lava_callback - ERROR - Error processing callback for node 69b111cb9d1917581825359b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:54650 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53444 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51568 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51618 - "GET / HTTP/1.0" 200 OK 2026-03-13 06:55:59,760 - lava_callback - INFO - Received callback for node 69b111ef9d191758182535a6 INFO: 172.20.0.1:53316 - "POST /node/69b111ef9d191758182535a6 HTTP/1.0" 202 Accepted 2026-03-13 06:55:59,766 - lava_callback - INFO - Starting async job submit for node 69b111ef9d191758182535a6 2026-03-13 06:55:59,767 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 06:55:59,781 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b111ef9d191758182535a6 HTTP/1.1" 200 None 2026-03-13 06:55:59,782 - lava_callback - DEBUG - Processing node 69b111ef9d191758182535a6: baseline-arm-clabbe 2026-03-13 06:55:59,782 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b111ef9d191758182535a6 2026-03-13 06:55:59,783 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 06:55:59,801 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b111ef9d191758182535a6 HTTP/1.1" 200 None 2026-03-13 06:55:59,803 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 06:55:59,817 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11668ef4424e487b2ea2 HTTP/1.1" 200 None 2026-03-13 06:55:59,818 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 06:55:59,838 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b111ef9d191758182535a6 HTTP/1.1" 200 None 2026-03-13 06:55:59,845 - lava_callback - ERROR - Error processing callback for node 69b111ef9d191758182535a6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:53326 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34288 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34296 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34310 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47122 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56820 - "GET / HTTP/1.0" 200 OK 2026-03-13 06:56:40,257 - lava_callback - INFO - Received callback for node 69b112139d191758182535b3 INFO: 172.20.0.1:40568 - "POST /node/69b112139d191758182535b3 HTTP/1.0" 202 Accepted 2026-03-13 06:56:40,262 - lava_callback - INFO - Starting async job submit for node 69b112139d191758182535b3 2026-03-13 06:56:40,263 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 06:56:40,279 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b112139d191758182535b3 HTTP/1.1" 200 None 2026-03-13 06:56:40,280 - lava_callback - DEBUG - Processing node 69b112139d191758182535b3: baseline-arm-clabbe 2026-03-13 06:56:40,280 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b112139d191758182535b3 2026-03-13 06:56:40,281 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 06:56:40,295 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b112139d191758182535b3 HTTP/1.1" 200 None 2026-03-13 06:56:40,296 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 06:56:40,314 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11668ef4424e487b2ea2 HTTP/1.1" 200 None 2026-03-13 06:56:40,316 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 06:56:40,338 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b112139d191758182535b3 HTTP/1.1" 200 None 2026-03-13 06:56:40,341 - lava_callback - ERROR - Error processing callback for node 69b112139d191758182535b3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:40572 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48038 - "GET / HTTP/1.0" 200 OK 2026-03-13 06:56:59,760 - lava_callback - INFO - Received callback for node 69b112389d191758182535bd INFO: 172.20.0.1:38648 - "POST /node/69b112389d191758182535bd HTTP/1.0" 202 Accepted 2026-03-13 06:56:59,765 - lava_callback - INFO - Starting async job submit for node 69b112389d191758182535bd 2026-03-13 06:56:59,766 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 06:56:59,781 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b112389d191758182535bd HTTP/1.1" 200 None 2026-03-13 06:56:59,782 - lava_callback - DEBUG - Processing node 69b112389d191758182535bd: baseline-arm-clabbe 2026-03-13 06:56:59,782 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b112389d191758182535bd 2026-03-13 06:56:59,783 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 06:56:59,799 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b112389d191758182535bd HTTP/1.1" 200 None 2026-03-13 06:56:59,800 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 06:56:59,813 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11668ef4424e487b2ea2 HTTP/1.1" 200 None 2026-03-13 06:56:59,815 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 06:56:59,841 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b112389d191758182535bd HTTP/1.1" 200 None 2026-03-13 06:56:59,842 - lava_callback - ERROR - Error processing callback for node 69b112389d191758182535bd: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:38660 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50426 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50434 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50450 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59280 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54158 - "GET / HTTP/1.0" 200 OK 2026-03-13 06:57:39,843 - lava_callback - INFO - Received callback for node 69b1125c9d191758182535c7 INFO: 172.20.0.1:41108 - "POST /node/69b1125c9d191758182535c7 HTTP/1.0" 202 Accepted 2026-03-13 06:57:39,849 - lava_callback - INFO - Starting async job submit for node 69b1125c9d191758182535c7 2026-03-13 06:57:39,850 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 06:57:39,864 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b1125c9d191758182535c7 HTTP/1.1" 200 None 2026-03-13 06:57:39,865 - lava_callback - DEBUG - Processing node 69b1125c9d191758182535c7: baseline-arm-clabbe 2026-03-13 06:57:39,865 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b1125c9d191758182535c7 2026-03-13 06:57:39,866 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 06:57:39,905 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b1125c9d191758182535c7 HTTP/1.1" 200 None 2026-03-13 06:57:39,907 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 06:57:39,920 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11668ef4424e487b2ea2 HTTP/1.1" 200 None 2026-03-13 06:57:39,921 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 06:57:39,949 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b1125c9d191758182535c7 HTTP/1.1" 200 None 2026-03-13 06:57:39,951 - lava_callback - ERROR - Error processing callback for node 69b1125c9d191758182535c7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:41114 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43668 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59194 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48194 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48202 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48208 - "GET / HTTP/1.0" 200 OK 2026-03-13 06:58:19,845 - lava_callback - INFO - Received callback for node 69b112809d191758182535e5 INFO: 172.20.0.1:37036 - "POST /node/69b112809d191758182535e5 HTTP/1.0" 202 Accepted 2026-03-13 06:58:19,851 - lava_callback - INFO - Starting async job submit for node 69b112809d191758182535e5 2026-03-13 06:58:19,852 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 06:58:19,865 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b112809d191758182535e5 HTTP/1.1" 200 None 2026-03-13 06:58:19,866 - lava_callback - DEBUG - Processing node 69b112809d191758182535e5: baseline-arm-clabbe 2026-03-13 06:58:19,866 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b112809d191758182535e5 2026-03-13 06:58:19,867 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 06:58:19,884 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b112809d191758182535e5 HTTP/1.1" 200 None 2026-03-13 06:58:19,886 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 06:58:19,899 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11668ef4424e487b2ea2 HTTP/1.1" 200 None 2026-03-13 06:58:19,901 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 06:58:19,921 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b112809d191758182535e5 HTTP/1.1" 200 None 2026-03-13 06:58:19,930 - lava_callback - ERROR - Error processing callback for node 69b112809d191758182535e5: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:37048 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34634 - "GET / HTTP/1.0" 200 OK 2026-03-13 06:58:46,242 - lava_callback - INFO - Received callback for node 69b112a09d191758182535ec INFO: 172.20.0.1:40594 - "POST /node/69b112a09d191758182535ec HTTP/1.0" 202 Accepted 2026-03-13 06:58:46,248 - lava_callback - INFO - Starting async job submit for node 69b112a09d191758182535ec 2026-03-13 06:58:46,248 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 06:58:46,269 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b112a09d191758182535ec HTTP/1.1" 200 None 2026-03-13 06:58:46,270 - lava_callback - DEBUG - Processing node 69b112a09d191758182535ec: baseline-arm-baylibre 2026-03-13 06:58:46,270 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b112a09d191758182535ec 2026-03-13 06:58:46,271 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 06:58:46,285 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b112a09d191758182535ec HTTP/1.1" 200 None 2026-03-13 06:58:46,286 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 06:58:46,300 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e34efa461f55e59cdb7 HTTP/1.1" 200 None 2026-03-13 06:58:46,301 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 06:58:46,321 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b112a09d191758182535ec HTTP/1.1" 200 None 2026-03-13 06:58:46,328 - lava_callback - ERROR - Error processing callback for node 69b112a09d191758182535ec: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:40596 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39450 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46536 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42604 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42610 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42626 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35156 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58108 - "GET / HTTP/1.0" 200 OK 2026-03-13 06:59:39,914 - lava_callback - INFO - Received callback for node 69b112c89d1917581825360a INFO: 172.20.0.1:38090 - "POST /node/69b112c89d1917581825360a HTTP/1.0" 202 Accepted 2026-03-13 06:59:39,919 - lava_callback - INFO - Starting async job submit for node 69b112c89d1917581825360a 2026-03-13 06:59:39,920 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 06:59:39,935 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b112c89d1917581825360a HTTP/1.1" 200 None 2026-03-13 06:59:39,935 - lava_callback - DEBUG - Processing node 69b112c89d1917581825360a: baseline-arm-clabbe 2026-03-13 06:59:39,936 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b112c89d1917581825360a 2026-03-13 06:59:39,936 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 06:59:39,949 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b112c89d1917581825360a HTTP/1.1" 200 None 2026-03-13 06:59:39,950 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 06:59:39,964 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11668ef4424e487b2ea2 HTTP/1.1" 200 None 2026-03-13 06:59:39,966 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 06:59:39,991 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b112c89d1917581825360a HTTP/1.1" 200 None 2026-03-13 06:59:39,992 - lava_callback - ERROR - Error processing callback for node 69b112c89d1917581825360a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:38096 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59154 - "GET / HTTP/1.0" 200 OK 2026-03-13 07:00:06,274 - lava_callback - INFO - Received callback for node 69b112e89d19175818253621 INFO: 172.20.0.1:35962 - "POST /node/69b112e89d19175818253621 HTTP/1.0" 202 Accepted 2026-03-13 07:00:06,280 - lava_callback - INFO - Starting async job submit for node 69b112e89d19175818253621 2026-03-13 07:00:06,281 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:00:06,296 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b112e89d19175818253621 HTTP/1.1" 200 None 2026-03-13 07:00:06,296 - lava_callback - DEBUG - Processing node 69b112e89d19175818253621: baseline-arm-baylibre 2026-03-13 07:00:06,297 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b112e89d19175818253621 2026-03-13 07:00:06,297 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:00:06,311 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b112e89d19175818253621 HTTP/1.1" 200 None 2026-03-13 07:00:06,312 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:00:06,326 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e34efa461f55e59cdb7 HTTP/1.1" 200 None 2026-03-13 07:00:06,328 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:00:06,360 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b112e89d19175818253621 HTTP/1.1" 200 None 2026-03-13 07:00:06,361 - lava_callback - ERROR - Error processing callback for node 69b112e89d19175818253621: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:35976 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35042 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35050 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35060 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45224 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53194 - "GET / HTTP/1.0" 200 OK 2026-03-13 07:00:46,283 - lava_callback - INFO - Received callback for node 69b1130c9d19175818253634 INFO: 172.20.0.1:40368 - "POST /node/69b1130c9d19175818253634 HTTP/1.0" 202 Accepted 2026-03-13 07:00:46,289 - lava_callback - INFO - Starting async job submit for node 69b1130c9d19175818253634 2026-03-13 07:00:46,289 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:00:46,310 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b1130c9d19175818253634 HTTP/1.1" 200 None 2026-03-13 07:00:46,311 - lava_callback - DEBUG - Processing node 69b1130c9d19175818253634: baseline-arm-baylibre 2026-03-13 07:00:46,311 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b1130c9d19175818253634 2026-03-13 07:00:46,312 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:00:46,325 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b1130c9d19175818253634 HTTP/1.1" 200 None 2026-03-13 07:00:46,327 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:00:46,341 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e34efa461f55e59cdb7 HTTP/1.1" 200 None 2026-03-13 07:00:46,342 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:00:46,369 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b1130c9d19175818253634 HTTP/1.1" 200 None 2026-03-13 07:00:46,370 - lava_callback - ERROR - Error processing callback for node 69b1130c9d19175818253634: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:40374 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34560 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50502 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35022 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35036 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35048 - "GET / HTTP/1.0" 200 OK 2026-03-13 07:01:20,052 - lava_callback - INFO - Received callback for node 69b113349d19175818253650 INFO: 172.20.0.1:32910 - "POST /node/69b113349d19175818253650 HTTP/1.0" 202 Accepted 2026-03-13 07:01:20,058 - lava_callback - INFO - Starting async job submit for node 69b113349d19175818253650 2026-03-13 07:01:20,059 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:01:20,073 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b113349d19175818253650 HTTP/1.1" 200 None 2026-03-13 07:01:20,074 - lava_callback - DEBUG - Processing node 69b113349d19175818253650: baseline-arm-clabbe 2026-03-13 07:01:20,074 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b113349d19175818253650 2026-03-13 07:01:20,075 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:01:20,088 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b113349d19175818253650 HTTP/1.1" 200 None 2026-03-13 07:01:20,090 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:01:20,104 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11668ef4424e487b2ea2 HTTP/1.1" 200 None 2026-03-13 07:01:20,106 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:01:20,132 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b113349d19175818253650 HTTP/1.1" 200 None 2026-03-13 07:01:20,133 - lava_callback - ERROR - Error processing callback for node 69b113349d19175818253650: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:32920 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42888 - "GET / HTTP/1.0" 200 OK 2026-03-13 07:01:42,838 - lava_callback - INFO - Received callback for node 69b363ca974c91e64560ed5e INFO: 172.20.0.1:33656 - "POST /node/69b363ca974c91e64560ed5e HTTP/1.0" 202 Accepted 2026-03-13 07:01:42,844 - lava_callback - INFO - Starting async job submit for node 69b363ca974c91e64560ed5e 2026-03-13 07:01:42,866 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:01:42,891 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b363ca974c91e64560ed5e HTTP/1.1" 200 None 2026-03-13 07:01:42,892 - lava_callback - DEBUG - Processing node 69b363ca974c91e64560ed5e: baseline-arm64-broonie 2026-03-13 07:01:43,165 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 07:01:43,326 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 07:01:43,327 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69b363ca974c91e64560ed5e/log.txt.gz 2026-03-13 07:01:43,497 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 07:01:43,836 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 07:01:43,838 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69b363ca974c91e64560ed5e/lava-logs.yaml 2026-03-13 07:01:43,845 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-13 07:01:44,150 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-13 07:01:44,152 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69b363ca974c91e64560ed5e/lava_callback.json.gz 2026-03-13 07:01:44,174 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:01:44,188 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b363ca974c91e64560ed5e HTTP/1.1" 200 None 2026-03-13 07:01:44,190 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:01:44,207 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3594d974c91e64560e4b2 HTTP/1.1" 200 None 2026-03-13 07:01:44,209 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:01:44,233 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b363ca974c91e64560ed5e HTTP/1.1" 200 None 2026-03-13 07:01:44,239 - lava_callback - INFO - Completed processing callback for node 69b363ca974c91e64560ed5e 2026-03-13 07:01:46,379 - lava_callback - INFO - Received callback for node 69b113559d1917581825365b INFO: 172.20.0.1:33672 - "POST /node/69b113559d1917581825365b HTTP/1.0" 202 Accepted 2026-03-13 07:01:46,384 - lava_callback - INFO - Starting async job submit for node 69b113559d1917581825365b 2026-03-13 07:01:46,385 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:01:46,411 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b113559d1917581825365b HTTP/1.1" 200 None 2026-03-13 07:01:46,412 - lava_callback - DEBUG - Processing node 69b113559d1917581825365b: baseline-arm-baylibre 2026-03-13 07:01:46,412 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b113559d1917581825365b 2026-03-13 07:01:46,413 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:01:46,436 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b113559d1917581825365b HTTP/1.1" 200 None 2026-03-13 07:01:46,437 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:01:46,451 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e34efa461f55e59cdb7 HTTP/1.1" 200 None 2026-03-13 07:01:46,452 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:01:46,458 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:01:46,478 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-13 07:01:46,484 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b113559d1917581825365b HTTP/1.1" 200 None 2026-03-13 07:01:46,486 - lava_callback - ERROR - Error processing callback for node 69b113559d1917581825365b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:33678 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44134 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51144 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50662 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50674 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50684 - "GET / HTTP/1.0" 200 OK 2026-03-13 07:02:26,388 - lava_callback - INFO - Received callback for node 69b113799d19175818253668 INFO: 172.20.0.1:43164 - "POST /node/69b113799d19175818253668 HTTP/1.0" 202 Accepted 2026-03-13 07:02:26,393 - lava_callback - INFO - Starting async job submit for node 69b113799d19175818253668 2026-03-13 07:02:26,394 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:02:26,409 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b113799d19175818253668 HTTP/1.1" 200 None 2026-03-13 07:02:26,409 - lava_callback - DEBUG - Processing node 69b113799d19175818253668: baseline-arm-baylibre 2026-03-13 07:02:26,410 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b113799d19175818253668 2026-03-13 07:02:26,410 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:02:26,424 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b113799d19175818253668 HTTP/1.1" 200 None 2026-03-13 07:02:26,426 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:02:26,439 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e34efa461f55e59cdb7 HTTP/1.1" 200 None 2026-03-13 07:02:26,441 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:02:26,469 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b113799d19175818253668 HTTP/1.1" 200 None 2026-03-13 07:02:26,470 - lava_callback - ERROR - Error processing callback for node 69b113799d19175818253668: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:43172 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36570 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50340 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49500 - "GET / HTTP/1.0" 200 OK 2026-03-13 07:03:00,068 - lava_callback - INFO - Received callback for node 69b1139e9d19175818253679 INFO: 172.20.0.1:59066 - "POST /node/69b1139e9d19175818253679 HTTP/1.0" 202 Accepted 2026-03-13 07:03:00,074 - lava_callback - INFO - Starting async job submit for node 69b1139e9d19175818253679 2026-03-13 07:03:00,075 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:03:00,091 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b1139e9d19175818253679 HTTP/1.1" 200 None 2026-03-13 07:03:00,092 - lava_callback - DEBUG - Processing node 69b1139e9d19175818253679: baseline-arm-clabbe 2026-03-13 07:03:00,093 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b1139e9d19175818253679 2026-03-13 07:03:00,093 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:03:00,107 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b1139e9d19175818253679 HTTP/1.1" 200 None 2026-03-13 07:03:00,108 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:03:00,121 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e34efa461f55e59cdb7 HTTP/1.1" 200 None 2026-03-13 07:03:00,123 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:03:00,151 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b1139e9d19175818253679 HTTP/1.1" 200 None 2026-03-13 07:03:00,153 - lava_callback - ERROR - Error processing callback for node 69b1139e9d19175818253679: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:59078 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51330 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51342 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51350 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34048 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54850 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33916 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40170 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52450 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50648 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50652 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50654 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45878 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38912 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35822 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33024 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49010 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41936 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41946 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41948 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45916 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42230 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55838 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44656 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56172 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55132 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55144 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55148 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43864 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40110 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40142 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34178 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37300 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36684 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36686 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36692 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42102 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45842 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47274 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57636 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50940 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54848 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54860 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54866 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39628 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41528 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54568 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53886 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43402 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41682 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41696 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41710 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35120 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57394 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44968 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39368 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50418 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50266 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50270 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50274 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47278 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35242 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52954 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56272 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33668 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36946 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36960 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36962 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60716 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43960 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51308 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49190 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34062 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59604 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59616 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59622 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49664 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46254 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41354 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54318 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60876 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38440 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38444 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38458 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37292 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51776 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56884 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47700 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44230 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58512 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58526 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58530 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60324 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46566 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48180 - "GET / HTTP/1.0" 200 OK 2026-03-13 07:14:47,129 - lava_callback - INFO - Received callback for node 69b116629d19175818253708 INFO: 172.20.0.1:48188 - "POST /node/69b116629d19175818253708 HTTP/1.0" 202 Accepted 2026-03-13 07:14:47,137 - lava_callback - INFO - Starting async job submit for node 69b116629d19175818253708 2026-03-13 07:14:47,139 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:14:47,156 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b116629d19175818253708 HTTP/1.1" 200 None 2026-03-13 07:14:47,158 - lava_callback - DEBUG - Processing node 69b116629d19175818253708: baseline-arm64-baylibre 2026-03-13 07:14:47,158 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b116629d19175818253708 2026-03-13 07:14:47,159 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:14:47,174 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b116629d19175818253708 HTTP/1.1" 200 None 2026-03-13 07:14:47,176 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:14:47,189 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b5149d19175818251feb HTTP/1.1" 200 None 2026-03-13 07:14:47,190 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:14:47,210 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b116629d19175818253708 HTTP/1.1" 200 None 2026-03-13 07:14:47,212 - lava_callback - ERROR - Error processing callback for node 69b116629d19175818253708: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:52350 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51462 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38846 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38850 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38858 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46846 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35614 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60946 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54444 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37000 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46094 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46102 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46104 - "GET / HTTP/1.0" 200 OK 2026-03-13 07:16:20,645 - lava_callback - INFO - Received callback for node 69b116b39d19175818253715 INFO: 172.20.0.1:49102 - "POST /node/69b116b39d19175818253715 HTTP/1.0" 202 Accepted 2026-03-13 07:16:20,650 - lava_callback - INFO - Starting async job submit for node 69b116b39d19175818253715 2026-03-13 07:16:20,651 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:16:20,665 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b116b39d19175818253715 HTTP/1.1" 200 None 2026-03-13 07:16:20,666 - lava_callback - DEBUG - Processing node 69b116b39d19175818253715: baseline-arm64-clabbe 2026-03-13 07:16:20,667 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b116b39d19175818253715 2026-03-13 07:16:20,667 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:16:20,682 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b116b39d19175818253715 HTTP/1.1" 200 None 2026-03-13 07:16:20,683 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:16:20,702 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b5149d19175818251feb HTTP/1.1" 200 None 2026-03-13 07:16:20,704 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:16:20,727 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b116b39d19175818253715 HTTP/1.1" 200 None 2026-03-13 07:16:20,737 - lava_callback - ERROR - Error processing callback for node 69b116b39d19175818253715: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:49104 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49224 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46110 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57312 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36490 - "GET / HTTP/1.0" 200 OK 2026-03-13 07:17:07,317 - lava_callback - INFO - Received callback for node 69b116eb9d19175818253722 INFO: 172.20.0.1:36502 - "POST /node/69b116eb9d19175818253722 HTTP/1.0" 202 Accepted 2026-03-13 07:17:07,323 - lava_callback - INFO - Starting async job submit for node 69b116eb9d19175818253722 2026-03-13 07:17:07,324 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:17:07,338 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b116eb9d19175818253722 HTTP/1.1" 200 None 2026-03-13 07:17:07,339 - lava_callback - DEBUG - Processing node 69b116eb9d19175818253722: baseline-arm-baylibre 2026-03-13 07:17:07,340 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b116eb9d19175818253722 2026-03-13 07:17:07,340 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:17:07,357 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b116eb9d19175818253722 HTTP/1.1" 200 None 2026-03-13 07:17:07,358 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:17:07,371 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e34efa461f55e59cdb7 HTTP/1.1" 200 None 2026-03-13 07:17:07,373 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:17:07,395 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b116eb9d19175818253722 HTTP/1.1" 200 None 2026-03-13 07:17:07,398 - lava_callback - ERROR - Error processing callback for node 69b116eb9d19175818253722: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:60402 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60418 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60430 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42330 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36676 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40046 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45682 - "GET / HTTP/1.0" 200 OK 2026-03-13 07:18:00,661 - lava_callback - INFO - Received callback for node 69b117219d1917581825372f INFO: 172.20.0.1:46018 - "POST /node/69b117219d1917581825372f HTTP/1.0" 202 Accepted 2026-03-13 07:18:00,667 - lava_callback - INFO - Starting async job submit for node 69b117219d1917581825372f 2026-03-13 07:18:00,668 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:18:00,698 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b117219d1917581825372f HTTP/1.1" 200 None 2026-03-13 07:18:00,699 - lava_callback - DEBUG - Processing node 69b117219d1917581825372f: baseline-arm64-clabbe 2026-03-13 07:18:00,699 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b117219d1917581825372f 2026-03-13 07:18:00,700 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:18:00,717 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b117219d1917581825372f HTTP/1.1" 200 None 2026-03-13 07:18:00,718 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:18:00,732 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b5149d19175818251feb HTTP/1.1" 200 None 2026-03-13 07:18:00,734 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:18:00,755 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b117219d1917581825372f HTTP/1.1" 200 None 2026-03-13 07:18:00,762 - lava_callback - ERROR - Error processing callback for node 69b117219d1917581825372f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:46032 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46698 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46712 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46722 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39334 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44136 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35518 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32998 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32934 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41994 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42002 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42004 - "GET / HTTP/1.0" 200 OK 2026-03-13 07:19:20,745 - lava_callback - INFO - Received callback for node 69b1176d9d19175818253744 INFO: 172.20.0.1:52912 - "POST /node/69b1176d9d19175818253744 HTTP/1.0" 202 Accepted 2026-03-13 07:19:20,750 - lava_callback - INFO - Starting async job submit for node 69b1176d9d19175818253744 2026-03-13 07:19:20,751 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:19:20,775 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b1176d9d19175818253744 HTTP/1.1" 200 None 2026-03-13 07:19:20,776 - lava_callback - DEBUG - Processing node 69b1176d9d19175818253744: baseline-arm64-clabbe 2026-03-13 07:19:20,777 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b1176d9d19175818253744 2026-03-13 07:19:20,777 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:19:20,791 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b1176d9d19175818253744 HTTP/1.1" 200 None 2026-03-13 07:19:20,793 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:19:20,806 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b5149d19175818251feb HTTP/1.1" 200 None 2026-03-13 07:19:20,808 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:19:20,828 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b1176d9d19175818253744 HTTP/1.1" 200 None 2026-03-13 07:19:20,830 - lava_callback - ERROR - Error processing callback for node 69b1176d9d19175818253744: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:52920 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41966 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32810 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48918 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40850 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34992 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35004 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35018 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55608 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46610 - "GET / HTTP/1.0" 200 OK 2026-03-13 07:20:40,729 - lava_callback - INFO - Received callback for node 69b117b89d1917581825374a INFO: 172.20.0.1:44280 - "POST /node/69b117b89d1917581825374a HTTP/1.0" 202 Accepted 2026-03-13 07:20:40,734 - lava_callback - INFO - Starting async job submit for node 69b117b89d1917581825374a 2026-03-13 07:20:40,735 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:20:40,751 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b117b89d1917581825374a HTTP/1.1" 200 None 2026-03-13 07:20:40,752 - lava_callback - DEBUG - Processing node 69b117b89d1917581825374a: baseline-arm64-clabbe 2026-03-13 07:20:40,752 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b117b89d1917581825374a 2026-03-13 07:20:40,753 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:20:40,780 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b117b89d1917581825374a HTTP/1.1" 200 None 2026-03-13 07:20:40,782 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:20:40,805 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b5149d19175818251feb HTTP/1.1" 200 None 2026-03-13 07:20:40,808 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:20:40,830 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b117b89d1917581825374a HTTP/1.1" 200 None 2026-03-13 07:20:40,840 - lava_callback - ERROR - Error processing callback for node 69b117b89d1917581825374a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:44288 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58042 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55200 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52136 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52146 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52156 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34432 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57714 - "GET / HTTP/1.0" 200 OK 2026-03-13 07:21:40,839 - lava_callback - INFO - Received callback for node 69b118029d1917581825375e INFO: 172.20.0.1:46052 - "POST /node/69b118029d1917581825375e HTTP/1.0" 202 Accepted 2026-03-13 07:21:40,845 - lava_callback - INFO - Starting async job submit for node 69b118029d1917581825375e 2026-03-13 07:21:40,846 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:21:40,861 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b118029d1917581825375e HTTP/1.1" 200 None 2026-03-13 07:21:40,862 - lava_callback - DEBUG - Processing node 69b118029d1917581825375e: baseline-arm64-clabbe 2026-03-13 07:21:40,862 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b118029d1917581825375e 2026-03-13 07:21:40,863 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:21:40,876 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b118029d1917581825375e HTTP/1.1" 200 None 2026-03-13 07:21:40,878 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:21:40,894 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b5149d19175818251feb HTTP/1.1" 200 None 2026-03-13 07:21:40,896 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:21:40,934 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b118029d1917581825375e HTTP/1.1" 200 None 2026-03-13 07:21:40,936 - lava_callback - ERROR - Error processing callback for node 69b118029d1917581825375e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:46058 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52116 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40600 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51976 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51978 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51992 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42140 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46622 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48474 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40254 - "GET / HTTP/1.0" 200 OK 2026-03-13 07:23:00,877 - lava_callback - INFO - Received callback for node 69b1184d9d19175818253769 INFO: 172.20.0.1:34670 - "POST /node/69b1184d9d19175818253769 HTTP/1.0" 202 Accepted 2026-03-13 07:23:00,883 - lava_callback - INFO - Starting async job submit for node 69b1184d9d19175818253769 2026-03-13 07:23:00,883 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:23:00,898 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b1184d9d19175818253769 HTTP/1.1" 200 None 2026-03-13 07:23:00,899 - lava_callback - DEBUG - Processing node 69b1184d9d19175818253769: baseline-arm64-clabbe 2026-03-13 07:23:00,900 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b1184d9d19175818253769 2026-03-13 07:23:00,900 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:23:00,926 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b1184d9d19175818253769 HTTP/1.1" 200 None 2026-03-13 07:23:00,928 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:23:00,941 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b5149d19175818251feb HTTP/1.1" 200 None 2026-03-13 07:23:00,942 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:23:00,962 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b1184d9d19175818253769 HTTP/1.1" 200 None 2026-03-13 07:23:00,964 - lava_callback - ERROR - Error processing callback for node 69b1184d9d19175818253769: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:34674 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38762 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38766 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38782 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52596 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47918 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36388 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51938 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50892 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37344 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37358 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37366 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60876 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58518 - "GET / HTTP/1.0" 200 OK 2026-03-13 07:24:40,953 - lava_callback - INFO - Received callback for node 69b118ab9d1917581825377d INFO: 172.20.0.1:38698 - "POST /node/69b118ab9d1917581825377d HTTP/1.0" 202 Accepted 2026-03-13 07:24:40,959 - lava_callback - INFO - Starting async job submit for node 69b118ab9d1917581825377d 2026-03-13 07:24:40,959 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:24:40,975 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b118ab9d1917581825377d HTTP/1.1" 200 None 2026-03-13 07:24:40,976 - lava_callback - DEBUG - Processing node 69b118ab9d1917581825377d: baseline-arm-clabbe 2026-03-13 07:24:40,976 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b118ab9d1917581825377d 2026-03-13 07:24:40,977 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:24:40,991 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b118ab9d1917581825377d HTTP/1.1" 200 None 2026-03-13 07:24:40,992 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:24:41,008 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e34efa461f55e59cdb7 HTTP/1.1" 200 None 2026-03-13 07:24:41,010 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:24:41,039 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b118ab9d1917581825377d HTTP/1.1" 200 None 2026-03-13 07:24:41,042 - lava_callback - ERROR - Error processing callback for node 69b118ab9d1917581825377d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:38700 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34448 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36440 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40920 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40924 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40930 - "GET / HTTP/1.0" 200 OK 2026-03-13 07:25:20,924 - lava_callback - INFO - Received callback for node 69b118d09d1917581825378c INFO: 172.20.0.1:44786 - "POST /node/69b118d09d1917581825378c HTTP/1.0" 202 Accepted 2026-03-13 07:25:20,929 - lava_callback - INFO - Starting async job submit for node 69b118d09d1917581825378c 2026-03-13 07:25:20,930 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:25:20,945 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b118d09d1917581825378c HTTP/1.1" 200 None 2026-03-13 07:25:20,946 - lava_callback - DEBUG - Processing node 69b118d09d1917581825378c: baseline-arm-clabbe 2026-03-13 07:25:20,946 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b118d09d1917581825378c 2026-03-13 07:25:20,946 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:25:20,959 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b118d09d1917581825378c HTTP/1.1" 200 None 2026-03-13 07:25:20,961 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:25:21,002 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e34efa461f55e59cdb7 HTTP/1.1" 200 None 2026-03-13 07:25:21,004 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:25:21,039 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b118d09d1917581825378c HTTP/1.1" 200 None 2026-03-13 07:25:21,044 - lava_callback - ERROR - Error processing callback for node 69b118d09d1917581825378c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:44802 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35656 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33834 - "GET / HTTP/1.0" 200 OK 2026-03-13 07:25:47,806 - lava_callback - INFO - Received callback for node 69b118ef9d19175818253799 INFO: 172.20.0.1:33846 - "POST /node/69b118ef9d19175818253799 HTTP/1.0" 202 Accepted 2026-03-13 07:25:47,812 - lava_callback - INFO - Starting async job submit for node 69b118ef9d19175818253799 2026-03-13 07:25:47,813 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:25:47,827 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b118ef9d19175818253799 HTTP/1.1" 200 None 2026-03-13 07:25:47,828 - lava_callback - DEBUG - Processing node 69b118ef9d19175818253799: baseline-arm-baylibre 2026-03-13 07:25:47,829 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b118ef9d19175818253799 2026-03-13 07:25:47,829 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:25:47,844 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b118ef9d19175818253799 HTTP/1.1" 200 None 2026-03-13 07:25:47,846 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:25:47,860 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e34efa461f55e59cdb7 HTTP/1.1" 200 None 2026-03-13 07:25:47,862 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:25:47,890 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b118ef9d19175818253799 HTTP/1.1" 200 None 2026-03-13 07:25:47,892 - lava_callback - ERROR - Error processing callback for node 69b118ef9d19175818253799: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:36426 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40604 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42872 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42882 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42886 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39508 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58220 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60312 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58378 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44326 - "GET / HTTP/1.0" 200 OK 2026-03-13 07:27:07,847 - lava_callback - INFO - Received callback for node 69b119369d191758182537ac INFO: 172.20.0.1:44330 - "POST /node/69b119369d191758182537ac HTTP/1.0" 202 Accepted 2026-03-13 07:27:07,852 - lava_callback - INFO - Starting async job submit for node 69b119369d191758182537ac 2026-03-13 07:27:07,853 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:27:07,874 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b119369d191758182537ac HTTP/1.1" 200 None 2026-03-13 07:27:07,875 - lava_callback - DEBUG - Processing node 69b119369d191758182537ac: baseline-arm-baylibre 2026-03-13 07:27:07,875 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b119369d191758182537ac 2026-03-13 07:27:07,876 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:27:07,889 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b119369d191758182537ac HTTP/1.1" 200 None 2026-03-13 07:27:07,890 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:27:07,903 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e34efa461f55e59cdb7 HTTP/1.1" 200 None 2026-03-13 07:27:07,905 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:27:07,939 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b119369d191758182537ac HTTP/1.1" 200 None 2026-03-13 07:27:07,941 - lava_callback - ERROR - Error processing callback for node 69b119369d191758182537ac: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:40626 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40632 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40638 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53402 - "GET / HTTP/1.0" 200 OK 2026-03-13 07:27:27,832 - lava_callback - INFO - Received callback for node 69b1195b9d191758182537b8 INFO: 172.20.0.1:53410 - "POST /node/69b1195b9d191758182537b8 HTTP/1.0" 202 Accepted 2026-03-13 07:27:27,837 - lava_callback - INFO - Starting async job submit for node 69b1195b9d191758182537b8 2026-03-13 07:27:27,838 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:27:27,852 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b1195b9d191758182537b8 HTTP/1.1" 200 None 2026-03-13 07:27:27,853 - lava_callback - DEBUG - Processing node 69b1195b9d191758182537b8: baseline-arm-baylibre 2026-03-13 07:27:27,853 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b1195b9d191758182537b8 2026-03-13 07:27:27,854 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:27:27,867 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b1195b9d191758182537b8 HTTP/1.1" 200 None 2026-03-13 07:27:27,869 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:27:27,882 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e34efa461f55e59cdb7 HTTP/1.1" 200 None 2026-03-13 07:27:27,883 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:27:27,908 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b1195b9d191758182537b8 HTTP/1.1" 200 None 2026-03-13 07:27:27,910 - lava_callback - ERROR - Error processing callback for node 69b1195b9d191758182537b8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:35846 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58372 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35984 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47246 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41520 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41534 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41544 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35040 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33542 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38000 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56200 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52798 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44184 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44196 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44198 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58080 - "GET / HTTP/1.0" 200 OK 2026-03-13 07:29:27,959 - lava_callback - INFO - Received callback for node 69b119c49d191758182537df INFO: 172.20.0.1:58082 - "POST /node/69b119c49d191758182537df HTTP/1.0" 202 Accepted 2026-03-13 07:29:27,964 - lava_callback - INFO - Starting async job submit for node 69b119c49d191758182537df 2026-03-13 07:29:27,965 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:29:27,979 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b119c49d191758182537df HTTP/1.1" 200 None 2026-03-13 07:29:27,980 - lava_callback - DEBUG - Processing node 69b119c49d191758182537df: baseline-arm-baylibre 2026-03-13 07:29:27,981 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b119c49d191758182537df 2026-03-13 07:29:27,981 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:29:27,995 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b119c49d191758182537df HTTP/1.1" 200 None 2026-03-13 07:29:27,996 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:29:28,017 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e34efa461f55e59cdb7 HTTP/1.1" 200 None 2026-03-13 07:29:28,019 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:29:28,039 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b119c49d191758182537df HTTP/1.1" 200 None 2026-03-13 07:29:28,041 - lava_callback - ERROR - Error processing callback for node 69b119c49d191758182537df: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:42244 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60402 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57816 - "GET / HTTP/1.0" 200 OK 2026-03-13 07:30:01,626 - lava_callback - INFO - Received callback for node 69b119ea9d191758182537eb INFO: 172.20.0.1:45710 - "POST /node/69b119ea9d191758182537eb HTTP/1.0" 202 Accepted 2026-03-13 07:30:01,632 - lava_callback - INFO - Starting async job submit for node 69b119ea9d191758182537eb 2026-03-13 07:30:01,633 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:30:01,647 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b119ea9d191758182537eb HTTP/1.1" 200 None 2026-03-13 07:30:01,648 - lava_callback - DEBUG - Processing node 69b119ea9d191758182537eb: baseline-arm-clabbe 2026-03-13 07:30:01,649 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b119ea9d191758182537eb 2026-03-13 07:30:01,649 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:30:01,663 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b119ea9d191758182537eb HTTP/1.1" 200 None 2026-03-13 07:30:01,664 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:30:01,678 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e34efa461f55e59cdb7 HTTP/1.1" 200 None 2026-03-13 07:30:01,679 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:30:01,701 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b119ea9d191758182537eb HTTP/1.1" 200 None 2026-03-13 07:30:01,703 - lava_callback - ERROR - Error processing callback for node 69b119ea9d191758182537eb: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:45716 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45636 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45640 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45642 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52184 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48956 - "GET / HTTP/1.0" 200 OK 2026-03-13 07:30:41,119 - lava_callback - INFO - Received callback for node 69b11a0e9d191758182537ff INFO: 172.20.0.1:46476 - "POST /node/69b11a0e9d191758182537ff HTTP/1.0" 202 Accepted 2026-03-13 07:30:41,124 - lava_callback - INFO - Starting async job submit for node 69b11a0e9d191758182537ff 2026-03-13 07:30:41,125 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:30:41,144 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b11a0e9d191758182537ff HTTP/1.1" 200 None 2026-03-13 07:30:41,145 - lava_callback - DEBUG - Processing node 69b11a0e9d191758182537ff: baseline-arm-clabbe 2026-03-13 07:30:41,145 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b11a0e9d191758182537ff 2026-03-13 07:30:41,146 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:30:41,160 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b11a0e9d191758182537ff HTTP/1.1" 200 None 2026-03-13 07:30:41,162 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:30:41,181 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e34efa461f55e59cdb7 HTTP/1.1" 200 None 2026-03-13 07:30:41,183 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:30:41,204 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b11a0e9d191758182537ff HTTP/1.1" 200 None 2026-03-13 07:30:41,205 - lava_callback - ERROR - Error processing callback for node 69b11a0e9d191758182537ff: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:46484 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35602 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49072 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50472 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50488 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50494 - "GET / HTTP/1.0" 200 OK 2026-03-13 07:31:21,184 - lava_callback - INFO - Received callback for node 69b11a339d19175818253811 INFO: 172.20.0.1:42860 - "POST /node/69b11a339d19175818253811 HTTP/1.0" 202 Accepted 2026-03-13 07:31:21,190 - lava_callback - INFO - Starting async job submit for node 69b11a339d19175818253811 2026-03-13 07:31:21,191 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:31:21,211 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b11a339d19175818253811 HTTP/1.1" 200 None 2026-03-13 07:31:21,213 - lava_callback - DEBUG - Processing node 69b11a339d19175818253811: baseline-arm-clabbe 2026-03-13 07:31:21,213 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b11a339d19175818253811 2026-03-13 07:31:21,213 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:31:21,228 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b11a339d19175818253811 HTTP/1.1" 200 None 2026-03-13 07:31:21,230 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:31:21,245 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e34efa461f55e59cdb7 HTTP/1.1" 200 None 2026-03-13 07:31:21,246 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:31:21,268 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b11a339d19175818253811 HTTP/1.1" 200 None 2026-03-13 07:31:21,274 - lava_callback - ERROR - Error processing callback for node 69b11a339d19175818253811: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:42868 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56724 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52216 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47070 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47444 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59714 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59724 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59734 - "GET / HTTP/1.0" 200 OK 2026-03-13 07:32:21,238 - lava_callback - INFO - Received callback for node 69b11a7b9d1917581825382f INFO: 172.20.0.1:35182 - "POST /node/69b11a7b9d1917581825382f HTTP/1.0" 202 Accepted 2026-03-13 07:32:21,243 - lava_callback - INFO - Starting async job submit for node 69b11a7b9d1917581825382f 2026-03-13 07:32:21,244 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:32:21,258 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b11a7b9d1917581825382f HTTP/1.1" 200 None 2026-03-13 07:32:21,259 - lava_callback - DEBUG - Processing node 69b11a7b9d1917581825382f: baseline-arm-clabbe 2026-03-13 07:32:21,259 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b11a7b9d1917581825382f 2026-03-13 07:32:21,260 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:32:21,273 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b11a7b9d1917581825382f HTTP/1.1" 200 None 2026-03-13 07:32:21,275 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:32:21,358 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e34efa461f55e59cdb7 HTTP/1.1" 200 None 2026-03-13 07:32:21,360 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:32:21,380 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b11a7b9d1917581825382f HTTP/1.1" 200 None 2026-03-13 07:32:21,388 - lava_callback - ERROR - Error processing callback for node 69b11a7b9d1917581825382f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:35190 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34172 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56992 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36326 - "GET / HTTP/1.0" 200 OK 2026-03-13 07:33:01,218 - lava_callback - INFO - Received callback for node 69b11aa09d1917581825383e INFO: 172.20.0.1:49550 - "POST /node/69b11aa09d1917581825383e HTTP/1.0" 202 Accepted 2026-03-13 07:33:01,224 - lava_callback - INFO - Starting async job submit for node 69b11aa09d1917581825383e 2026-03-13 07:33:01,225 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:33:01,239 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b11aa09d1917581825383e HTTP/1.1" 200 None 2026-03-13 07:33:01,240 - lava_callback - DEBUG - Processing node 69b11aa09d1917581825383e: baseline-arm-clabbe 2026-03-13 07:33:01,240 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b11aa09d1917581825383e 2026-03-13 07:33:01,241 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:33:01,254 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b11aa09d1917581825383e HTTP/1.1" 200 None 2026-03-13 07:33:01,255 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:33:01,277 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e34efa461f55e59cdb7 HTTP/1.1" 200 None 2026-03-13 07:33:01,279 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:33:01,311 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b11aa09d1917581825383e HTTP/1.1" 200 None 2026-03-13 07:33:01,313 - lava_callback - ERROR - Error processing callback for node 69b11aa09d1917581825383e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:49552 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33178 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33192 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33194 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33834 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60038 - "GET / HTTP/1.0" 200 OK 2026-03-13 07:33:41,253 - lava_callback - INFO - Received callback for node 69b11ac49d1917581825384e INFO: 172.20.0.1:58202 - "POST /node/69b11ac49d1917581825384e HTTP/1.0" 202 Accepted 2026-03-13 07:33:41,258 - lava_callback - INFO - Starting async job submit for node 69b11ac49d1917581825384e 2026-03-13 07:33:41,259 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:33:41,274 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b11ac49d1917581825384e HTTP/1.1" 200 None 2026-03-13 07:33:41,274 - lava_callback - DEBUG - Processing node 69b11ac49d1917581825384e: baseline-arm-clabbe 2026-03-13 07:33:41,275 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b11ac49d1917581825384e 2026-03-13 07:33:41,275 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:33:41,289 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b11ac49d1917581825384e HTTP/1.1" 200 None 2026-03-13 07:33:41,290 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:33:41,304 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e34efa461f55e59cdb7 HTTP/1.1" 200 None 2026-03-13 07:33:41,306 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:33:41,326 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b11ac49d1917581825384e HTTP/1.1" 200 None 2026-03-13 07:33:41,333 - lava_callback - ERROR - Error processing callback for node 69b11ac49d1917581825384e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:58212 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46034 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60392 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48758 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48768 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48784 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58304 - "GET / HTTP/1.0" 200 OK 2026-03-13 07:34:28,205 - lava_callback - INFO - Received callback for node 69b11aef9d19175818253865 INFO: 172.20.0.1:58310 - "POST /node/69b11aef9d19175818253865 HTTP/1.0" 202 Accepted 2026-03-13 07:34:28,210 - lava_callback - INFO - Starting async job submit for node 69b11aef9d19175818253865 2026-03-13 07:34:28,211 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:34:28,225 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b11aef9d19175818253865 HTTP/1.1" 200 None 2026-03-13 07:34:28,226 - lava_callback - DEBUG - Processing node 69b11aef9d19175818253865: baseline-arm64-baylibre 2026-03-13 07:34:28,227 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b11aef9d19175818253865 2026-03-13 07:34:28,227 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:34:28,240 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b11aef9d19175818253865 HTTP/1.1" 200 None 2026-03-13 07:34:28,242 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:34:28,255 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11758ef4424e487b2eb1 HTTP/1.1" 200 None 2026-03-13 07:34:28,257 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:34:28,278 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b11aef9d19175818253865 HTTP/1.1" 200 None 2026-03-13 07:34:28,283 - lava_callback - ERROR - Error processing callback for node 69b11aef9d19175818253865: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:45924 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46702 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38494 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35882 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35616 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35632 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35638 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59700 - "GET / HTTP/1.0" 200 OK 2026-03-13 07:35:28,245 - lava_callback - INFO - Received callback for node 69b11b3a9d1917581825386d INFO: 172.20.0.1:59712 - "POST /node/69b11b3a9d1917581825386d HTTP/1.0" 202 Accepted 2026-03-13 07:35:28,251 - lava_callback - INFO - Starting async job submit for node 69b11b3a9d1917581825386d 2026-03-13 07:35:28,252 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:35:28,267 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b11b3a9d1917581825386d HTTP/1.1" 200 None 2026-03-13 07:35:28,268 - lava_callback - DEBUG - Processing node 69b11b3a9d1917581825386d: baseline-arm64-baylibre 2026-03-13 07:35:28,268 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b11b3a9d1917581825386d 2026-03-13 07:35:28,268 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:35:28,281 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b11b3a9d1917581825386d HTTP/1.1" 200 None 2026-03-13 07:35:28,283 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:35:28,296 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11758ef4424e487b2eb1 HTTP/1.1" 200 None 2026-03-13 07:35:28,298 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:35:28,324 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b11b3a9d1917581825386d HTTP/1.1" 200 None 2026-03-13 07:35:28,325 - lava_callback - ERROR - Error processing callback for node 69b11b3a9d1917581825386d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:36354 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40654 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36028 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58110 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47070 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47078 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47094 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55206 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44066 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42456 - "GET / HTTP/1.0" 200 OK 2026-03-13 07:36:48,362 - lava_callback - INFO - Received callback for node 69b11b849d1917581825387a INFO: 172.20.0.1:42466 - "POST /node/69b11b849d1917581825387a HTTP/1.0" 202 Accepted 2026-03-13 07:36:48,368 - lava_callback - INFO - Starting async job submit for node 69b11b849d1917581825387a 2026-03-13 07:36:48,369 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:36:48,385 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b11b849d1917581825387a HTTP/1.1" 200 None 2026-03-13 07:36:48,385 - lava_callback - DEBUG - Processing node 69b11b849d1917581825387a: baseline-arm64-baylibre 2026-03-13 07:36:48,386 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b11b849d1917581825387a 2026-03-13 07:36:48,386 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:36:48,405 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b11b849d1917581825387a HTTP/1.1" 200 None 2026-03-13 07:36:48,407 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:36:48,421 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11758ef4424e487b2eb1 HTTP/1.1" 200 None 2026-03-13 07:36:48,422 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:36:48,443 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b11b849d1917581825387a HTTP/1.1" 200 None 2026-03-13 07:36:48,444 - lava_callback - ERROR - Error processing callback for node 69b11b849d1917581825387a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:41940 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41664 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47556 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47570 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47586 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38860 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33936 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36998 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52134 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44834 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50828 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50838 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50852 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46708 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44996 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60440 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60074 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55384 - "GET / HTTP/1.0" 200 OK 2026-03-13 07:39:08,528 - lava_callback - INFO - Received callback for node 69b11c199d1917581825389e INFO: 172.20.0.1:55398 - "POST /node/69b11c199d1917581825389e HTTP/1.0" 202 Accepted 2026-03-13 07:39:08,534 - lava_callback - INFO - Starting async job submit for node 69b11c199d1917581825389e 2026-03-13 07:39:08,535 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:39:08,551 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b11c199d1917581825389e HTTP/1.1" 200 None 2026-03-13 07:39:08,552 - lava_callback - DEBUG - Processing node 69b11c199d1917581825389e: baseline-arm64-baylibre 2026-03-13 07:39:08,553 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b11c199d1917581825389e 2026-03-13 07:39:08,553 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:39:08,568 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b11c199d1917581825389e HTTP/1.1" 200 None 2026-03-13 07:39:08,570 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:39:08,586 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11758ef4424e487b2eb1 HTTP/1.1" 200 None 2026-03-13 07:39:08,588 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:39:08,616 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b11c199d1917581825389e HTTP/1.1" 200 None 2026-03-13 07:39:08,618 - lava_callback - ERROR - Error processing callback for node 69b11c199d1917581825389e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:52384 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52388 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52398 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41278 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38606 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39548 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37198 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50176 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58730 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58738 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58750 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55346 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60066 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37170 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48344 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54892 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39368 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39384 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39394 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38718 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56962 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38186 - "GET / HTTP/1.0" 200 OK 2026-03-13 07:41:48,626 - lava_callback - INFO - Received callback for node 69b11caf9d191758182538b2 INFO: 172.20.0.1:38192 - "POST /node/69b11caf9d191758182538b2 HTTP/1.0" 202 Accepted 2026-03-13 07:41:48,631 - lava_callback - INFO - Starting async job submit for node 69b11caf9d191758182538b2 2026-03-13 07:41:48,632 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:41:48,646 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b11caf9d191758182538b2 HTTP/1.1" 200 None 2026-03-13 07:41:48,647 - lava_callback - DEBUG - Processing node 69b11caf9d191758182538b2: baseline-arm64-baylibre 2026-03-13 07:41:48,648 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b11caf9d191758182538b2 2026-03-13 07:41:48,648 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:41:48,663 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b11caf9d191758182538b2 HTTP/1.1" 200 None 2026-03-13 07:41:48,664 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:41:48,684 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11758ef4424e487b2eb1 HTTP/1.1" 200 None 2026-03-13 07:41:48,686 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:41:48,707 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b11caf9d191758182538b2 HTTP/1.1" 200 None 2026-03-13 07:41:48,718 - lava_callback - ERROR - Error processing callback for node 69b11caf9d191758182538b2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:44310 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54538 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56318 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56328 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56344 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50658 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37842 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50718 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48154 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33834 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35302 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35312 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35318 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54964 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56926 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40866 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60680 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35734 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40076 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40080 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40090 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60224 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34336 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60870 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40794 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43706 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39488 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39494 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39504 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40606 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39190 - "GET / HTTP/1.0" 200 OK 2026-03-13 07:45:41,916 - lava_callback - INFO - Received callback for node 69b11d929d191758182538e2 INFO: 172.20.0.1:32846 - "POST /node/69b11d929d191758182538e2 HTTP/1.0" 202 Accepted 2026-03-13 07:45:41,922 - lava_callback - INFO - Starting async job submit for node 69b11d929d191758182538e2 2026-03-13 07:45:41,923 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:45:41,942 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b11d929d191758182538e2 HTTP/1.1" 200 None 2026-03-13 07:45:41,943 - lava_callback - DEBUG - Processing node 69b11d929d191758182538e2: baseline-arm64-clabbe 2026-03-13 07:45:41,944 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b11d929d191758182538e2 2026-03-13 07:45:41,944 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:45:41,959 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b11d929d191758182538e2 HTTP/1.1" 200 None 2026-03-13 07:45:41,961 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:45:41,974 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11758ef4424e487b2eb1 HTTP/1.1" 200 None 2026-03-13 07:45:41,976 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:45:41,998 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b11d929d191758182538e2 HTTP/1.1" 200 None 2026-03-13 07:45:42,000 - lava_callback - ERROR - Error processing callback for node 69b11d929d191758182538e2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:32852 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58182 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52210 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58370 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58384 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58388 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50384 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57376 - "GET / HTTP/1.0" 200 OK 2026-03-13 07:46:41,884 - lava_callback - INFO - Received callback for node 69b11ddd9d191758182538f7 INFO: 172.20.0.1:54650 - "POST /node/69b11ddd9d191758182538f7 HTTP/1.0" 202 Accepted 2026-03-13 07:46:41,889 - lava_callback - INFO - Starting async job submit for node 69b11ddd9d191758182538f7 2026-03-13 07:46:41,890 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:46:41,921 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b11ddd9d191758182538f7 HTTP/1.1" 200 None 2026-03-13 07:46:41,922 - lava_callback - DEBUG - Processing node 69b11ddd9d191758182538f7: baseline-arm64-clabbe 2026-03-13 07:46:41,922 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b11ddd9d191758182538f7 2026-03-13 07:46:41,923 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:46:41,936 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b11ddd9d191758182538f7 HTTP/1.1" 200 None 2026-03-13 07:46:41,937 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:46:41,950 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11758ef4424e487b2eb1 HTTP/1.1" 200 None 2026-03-13 07:46:41,951 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:46:41,974 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b11ddd9d191758182538f7 HTTP/1.1" 200 None 2026-03-13 07:46:41,981 - lava_callback - ERROR - Error processing callback for node 69b11ddd9d191758182538f7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:54664 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36442 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36764 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46532 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46536 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46550 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44370 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36560 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40476 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49494 - "GET / HTTP/1.0" 200 OK 2026-03-13 07:48:01,879 - lava_callback - INFO - Received callback for node 69b11e289d19175818253900 INFO: 172.20.0.1:37944 - "POST /node/69b11e289d19175818253900 HTTP/1.0" 202 Accepted 2026-03-13 07:48:01,885 - lava_callback - INFO - Starting async job submit for node 69b11e289d19175818253900 2026-03-13 07:48:01,886 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:48:01,901 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b11e289d19175818253900 HTTP/1.1" 200 None 2026-03-13 07:48:01,902 - lava_callback - DEBUG - Processing node 69b11e289d19175818253900: baseline-arm64-clabbe 2026-03-13 07:48:01,902 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b11e289d19175818253900 2026-03-13 07:48:01,903 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:48:01,919 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b11e289d19175818253900 HTTP/1.1" 200 None 2026-03-13 07:48:01,921 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:48:01,935 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11758ef4424e487b2eb1 HTTP/1.1" 200 None 2026-03-13 07:48:01,937 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:48:01,960 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b11e289d19175818253900 HTTP/1.1" 200 None 2026-03-13 07:48:01,962 - lava_callback - ERROR - Error processing callback for node 69b11e289d19175818253900: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:37958 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37952 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37960 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37972 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33558 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42068 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33688 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60880 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54002 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34554 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34558 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34570 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48416 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35238 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34226 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51414 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47708 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55462 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55470 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55474 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57100 - "GET / HTTP/1.0" 200 OK 2026-03-13 07:50:29,120 - lava_callback - INFO - Received callback for node 69b11eb89d19175818253915 INFO: 172.20.0.1:57114 - "POST /node/69b11eb89d19175818253915 HTTP/1.0" 202 Accepted 2026-03-13 07:50:29,126 - lava_callback - INFO - Starting async job submit for node 69b11eb89d19175818253915 2026-03-13 07:50:29,127 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:50:29,143 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b11eb89d19175818253915 HTTP/1.1" 200 None 2026-03-13 07:50:29,144 - lava_callback - DEBUG - Processing node 69b11eb89d19175818253915: baseline-arm64-baylibre 2026-03-13 07:50:29,144 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b11eb89d19175818253915 2026-03-13 07:50:29,145 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:50:29,159 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b11eb89d19175818253915 HTTP/1.1" 200 None 2026-03-13 07:50:29,160 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:50:29,174 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11758ef4424e487b2eb1 HTTP/1.1" 200 None 2026-03-13 07:50:29,175 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:50:29,202 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b11eb89d19175818253915 HTTP/1.1" 200 None 2026-03-13 07:50:29,205 - lava_callback - ERROR - Error processing callback for node 69b11eb89d19175818253915: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:51262 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56384 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51132 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53850 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45242 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45258 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45272 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55884 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35056 - "GET / HTTP/1.0" 200 OK 2026-03-13 07:51:42,071 - lava_callback - INFO - Received callback for node 69b11f089d19175818253926 INFO: 172.20.0.1:39132 - "POST /node/69b11f089d19175818253926 HTTP/1.0" 202 Accepted 2026-03-13 07:51:42,077 - lava_callback - INFO - Starting async job submit for node 69b11f089d19175818253926 2026-03-13 07:51:42,077 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:51:42,092 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b11f089d19175818253926 HTTP/1.1" 200 None 2026-03-13 07:51:42,092 - lava_callback - DEBUG - Processing node 69b11f089d19175818253926: baseline-arm64-clabbe 2026-03-13 07:51:42,093 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b11f089d19175818253926 2026-03-13 07:51:42,093 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:51:42,106 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b11f089d19175818253926 HTTP/1.1" 200 None 2026-03-13 07:51:42,108 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:51:42,148 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11758ef4424e487b2eb1 HTTP/1.1" 200 None 2026-03-13 07:51:42,149 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:51:42,175 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b11f089d19175818253926 HTTP/1.1" 200 None 2026-03-13 07:51:42,177 - lava_callback - ERROR - Error processing callback for node 69b11f089d19175818253926: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:39146 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51948 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43766 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35692 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35706 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35714 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45858 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58086 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54176 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50760 - "GET / HTTP/1.0" 200 OK 2026-03-13 07:53:02,160 - lava_callback - INFO - Received callback for node 69b11f539d19175818253931 INFO: 172.20.0.1:52568 - "POST /node/69b11f539d19175818253931 HTTP/1.0" 202 Accepted 2026-03-13 07:53:02,165 - lava_callback - INFO - Starting async job submit for node 69b11f539d19175818253931 2026-03-13 07:53:02,166 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:53:02,180 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b11f539d19175818253931 HTTP/1.1" 200 None 2026-03-13 07:53:02,181 - lava_callback - DEBUG - Processing node 69b11f539d19175818253931: baseline-arm64-clabbe 2026-03-13 07:53:02,181 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b11f539d19175818253931 2026-03-13 07:53:02,182 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:53:02,196 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b11f539d19175818253931 HTTP/1.1" 200 None 2026-03-13 07:53:02,197 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:53:02,221 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11758ef4424e487b2eb1 HTTP/1.1" 200 None 2026-03-13 07:53:02,222 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:53:02,243 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b11f539d19175818253931 HTTP/1.1" 200 None 2026-03-13 07:53:02,245 - lava_callback - ERROR - Error processing callback for node 69b11f539d19175818253931: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:52580 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53966 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53980 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53990 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50376 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40816 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41092 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47392 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36802 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42370 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42386 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42396 - "GET / HTTP/1.0" 200 OK 2026-03-13 07:54:22,243 - lava_callback - INFO - Received callback for node 69b11f9e9d19175818253942 INFO: 172.20.0.1:38974 - "POST /node/69b11f9e9d19175818253942 HTTP/1.0" 202 Accepted 2026-03-13 07:54:22,249 - lava_callback - INFO - Starting async job submit for node 69b11f9e9d19175818253942 2026-03-13 07:54:22,250 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:54:22,265 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b11f9e9d19175818253942 HTTP/1.1" 200 None 2026-03-13 07:54:22,265 - lava_callback - DEBUG - Processing node 69b11f9e9d19175818253942: baseline-arm64-clabbe 2026-03-13 07:54:22,266 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b11f9e9d19175818253942 2026-03-13 07:54:22,266 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:54:22,283 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b11f9e9d19175818253942 HTTP/1.1" 200 None 2026-03-13 07:54:22,285 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:54:22,298 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11758ef4424e487b2eb1 HTTP/1.1" 200 None 2026-03-13 07:54:22,300 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:54:22,323 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b11f9e9d19175818253942 HTTP/1.1" 200 None 2026-03-13 07:54:22,329 - lava_callback - ERROR - Error processing callback for node 69b11f9e9d19175818253942: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:38990 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47966 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34820 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42838 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44342 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48588 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48596 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48598 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57102 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45138 - "GET / HTTP/1.0" 200 OK 2026-03-13 07:55:42,265 - lava_callback - INFO - Received callback for node 69b11fea9d1917581825394b INFO: 172.20.0.1:40114 - "POST /node/69b11fea9d1917581825394b HTTP/1.0" 202 Accepted 2026-03-13 07:55:42,271 - lava_callback - INFO - Starting async job submit for node 69b11fea9d1917581825394b 2026-03-13 07:55:42,272 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:55:42,287 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b11fea9d1917581825394b HTTP/1.1" 200 None 2026-03-13 07:55:42,287 - lava_callback - DEBUG - Processing node 69b11fea9d1917581825394b: baseline-arm64-clabbe 2026-03-13 07:55:42,288 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b11fea9d1917581825394b 2026-03-13 07:55:42,288 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:55:42,302 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b11fea9d1917581825394b HTTP/1.1" 200 None 2026-03-13 07:55:42,303 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:55:42,316 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11758ef4424e487b2eb1 HTTP/1.1" 200 None 2026-03-13 07:55:42,317 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:55:42,339 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b11fea9d1917581825394b HTTP/1.1" 200 None 2026-03-13 07:55:42,340 - lava_callback - ERROR - Error processing callback for node 69b11fea9d1917581825394b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:40130 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38684 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50766 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50808 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50810 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50818 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36744 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35706 - "GET / HTTP/1.0" 200 OK 2026-03-13 07:56:42,264 - lava_callback - INFO - Received callback for node 69b120369d19175818253959 INFO: 172.20.0.1:39020 - "POST /node/69b120369d19175818253959 HTTP/1.0" 202 Accepted 2026-03-13 07:56:42,270 - lava_callback - INFO - Starting async job submit for node 69b120369d19175818253959 2026-03-13 07:56:42,271 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:56:42,285 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b120369d19175818253959 HTTP/1.1" 200 None 2026-03-13 07:56:42,285 - lava_callback - DEBUG - Processing node 69b120369d19175818253959: baseline-arm64-clabbe 2026-03-13 07:56:42,286 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b120369d19175818253959 2026-03-13 07:56:42,286 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:56:42,299 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b120369d19175818253959 HTTP/1.1" 200 None 2026-03-13 07:56:42,300 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:56:42,313 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11758ef4424e487b2eb1 HTTP/1.1" 200 None 2026-03-13 07:56:42,314 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:56:42,334 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b120369d19175818253959 HTTP/1.1" 200 None 2026-03-13 07:56:42,340 - lava_callback - ERROR - Error processing callback for node 69b120369d19175818253959: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:39022 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56558 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47550 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36548 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36558 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36568 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45834 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40346 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52186 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49904 - "GET / HTTP/1.0" 200 OK 2026-03-13 07:58:02,377 - lava_callback - INFO - Received callback for node 69b120839d19175818253963 INFO: 172.20.0.1:52186 - "POST /node/69b120839d19175818253963 HTTP/1.0" 202 Accepted 2026-03-13 07:58:02,382 - lava_callback - INFO - Starting async job submit for node 69b120839d19175818253963 2026-03-13 07:58:02,383 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:58:02,398 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b120839d19175818253963 HTTP/1.1" 200 None 2026-03-13 07:58:02,399 - lava_callback - DEBUG - Processing node 69b120839d19175818253963: baseline-arm64-clabbe 2026-03-13 07:58:02,399 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b120839d19175818253963 2026-03-13 07:58:02,400 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:58:02,416 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b120839d19175818253963 HTTP/1.1" 200 None 2026-03-13 07:58:02,417 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:58:02,432 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11758ef4424e487b2eb1 HTTP/1.1" 200 None 2026-03-13 07:58:02,434 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:58:02,460 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b120839d19175818253963 HTTP/1.1" 200 None 2026-03-13 07:58:02,461 - lava_callback - ERROR - Error processing callback for node 69b120839d19175818253963: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:52200 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46854 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46868 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46872 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40508 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44854 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48364 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51748 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36108 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50520 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50528 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50534 - "GET / HTTP/1.0" 200 OK 2026-03-13 07:59:22,454 - lava_callback - INFO - Received callback for node 69b120cd9d19175818253973 INFO: 172.20.0.1:34340 - "POST /node/69b120cd9d19175818253973 HTTP/1.0" 202 Accepted 2026-03-13 07:59:22,463 - lava_callback - INFO - Starting async job submit for node 69b120cd9d19175818253973 2026-03-13 07:59:22,464 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:59:22,478 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b120cd9d19175818253973 HTTP/1.1" 200 None 2026-03-13 07:59:22,479 - lava_callback - DEBUG - Processing node 69b120cd9d19175818253973: baseline-arm64-clabbe 2026-03-13 07:59:22,480 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b120cd9d19175818253973 2026-03-13 07:59:22,480 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:59:22,494 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b120cd9d19175818253973 HTTP/1.1" 200 None 2026-03-13 07:59:22,495 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:59:22,513 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11758ef4424e487b2eb1 HTTP/1.1" 200 None 2026-03-13 07:59:22,514 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 07:59:22,540 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b120cd9d19175818253973 HTTP/1.1" 200 None 2026-03-13 07:59:22,546 - lava_callback - ERROR - Error processing callback for node 69b120cd9d19175818253973: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:34354 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55734 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44356 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40436 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57410 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55556 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55558 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55570 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58594 - "GET / HTTP/1.0" 200 OK 2026-03-13 08:00:29,690 - lava_callback - INFO - Received callback for node 69b121109d1917581825397e INFO: 172.20.0.1:58604 - "POST /node/69b121109d1917581825397e HTTP/1.0" 202 Accepted 2026-03-13 08:00:29,695 - lava_callback - INFO - Starting async job submit for node 69b121109d1917581825397e 2026-03-13 08:00:29,696 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 08:00:29,714 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b121109d1917581825397e HTTP/1.1" 200 None 2026-03-13 08:00:29,715 - lava_callback - DEBUG - Processing node 69b121109d1917581825397e: baseline-arm64-baylibre 2026-03-13 08:00:29,716 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b121109d1917581825397e 2026-03-13 08:00:29,716 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 08:00:29,729 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b121109d1917581825397e HTTP/1.1" 200 None 2026-03-13 08:00:29,731 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 08:00:29,743 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e44efa461f55e59cdc8 HTTP/1.1" 200 None 2026-03-13 08:00:29,745 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 08:00:29,768 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b121109d1917581825397e HTTP/1.1" 200 None 2026-03-13 08:00:29,769 - lava_callback - ERROR - Error processing callback for node 69b121109d1917581825397e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:43834 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42392 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47024 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43196 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47550 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47556 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47572 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59226 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36472 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56440 - "GET / HTTP/1.0" 200 OK 2026-03-13 08:01:49,777 - lava_callback - INFO - Received callback for node 69b1215d9d19175818253986 INFO: 172.20.0.1:57094 - "POST /node/69b1215d9d19175818253986 HTTP/1.0" 202 Accepted 2026-03-13 08:01:49,784 - lava_callback - INFO - Starting async job submit for node 69b1215d9d19175818253986 2026-03-13 08:01:49,785 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 08:01:49,799 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b1215d9d19175818253986 HTTP/1.1" 200 None 2026-03-13 08:01:49,799 - lava_callback - DEBUG - Processing node 69b1215d9d19175818253986: baseline-arm64-baylibre 2026-03-13 08:01:49,800 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b1215d9d19175818253986 2026-03-13 08:01:49,800 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 08:01:49,816 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b1215d9d19175818253986 HTTP/1.1" 200 None 2026-03-13 08:01:49,817 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 08:01:49,838 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e44efa461f55e59cdc8 HTTP/1.1" 200 None 2026-03-13 08:01:49,840 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 08:01:49,861 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b1215d9d19175818253986 HTTP/1.1" 200 None 2026-03-13 08:01:49,868 - lava_callback - ERROR - Error processing callback for node 69b1215d9d19175818253986: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:57098 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43266 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38040 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38056 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38062 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35696 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44296 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39050 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45210 - "GET / HTTP/1.0" 200 OK 2026-03-13 08:03:02,622 - lava_callback - INFO - Received callback for node 69b121b39d19175818253995 INFO: 172.20.0.1:40294 - "POST /node/69b121b39d19175818253995 HTTP/1.0" 202 Accepted 2026-03-13 08:03:02,628 - lava_callback - INFO - Starting async job submit for node 69b121b39d19175818253995 2026-03-13 08:03:02,629 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 08:03:02,642 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b121b39d19175818253995 HTTP/1.1" 200 None 2026-03-13 08:03:02,643 - lava_callback - DEBUG - Processing node 69b121b39d19175818253995: baseline-arm64-clabbe 2026-03-13 08:03:02,643 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b121b39d19175818253995 2026-03-13 08:03:02,644 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 08:03:02,656 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b121b39d19175818253995 HTTP/1.1" 200 None 2026-03-13 08:03:02,658 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 08:03:02,671 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11758ef4424e487b2eb1 HTTP/1.1" 200 None 2026-03-13 08:03:02,672 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 08:03:02,692 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b121b39d19175818253995 HTTP/1.1" 200 None 2026-03-13 08:03:02,694 - lava_callback - ERROR - Error processing callback for node 69b121b39d19175818253995: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:40304 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58068 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58080 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58094 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40384 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38130 - "GET /node/6994465f5ae459b9df3d9f7b%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:38136 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58552 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36870 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38134 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41640 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41644 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41650 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57890 - "GET / HTTP/1.0" 200 OK 2026-03-13 08:04:29,913 - lava_callback - INFO - Received callback for node 69b121fa9d191758182539a1 INFO: 172.20.0.1:50866 - "POST /node/69b121fa9d191758182539a1 HTTP/1.0" 202 Accepted 2026-03-13 08:04:29,919 - lava_callback - INFO - Starting async job submit for node 69b121fa9d191758182539a1 2026-03-13 08:04:29,919 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 08:04:29,935 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b121fa9d191758182539a1 HTTP/1.1" 200 None 2026-03-13 08:04:29,936 - lava_callback - DEBUG - Processing node 69b121fa9d191758182539a1: baseline-arm64-baylibre 2026-03-13 08:04:29,936 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b121fa9d191758182539a1 2026-03-13 08:04:29,937 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 08:04:29,950 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b121fa9d191758182539a1 HTTP/1.1" 200 None 2026-03-13 08:04:29,951 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 08:04:29,978 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e44efa461f55e59cdc8 HTTP/1.1" 200 None 2026-03-13 08:04:29,980 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 08:04:30,048 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b121fa9d191758182539a1 HTTP/1.1" 200 None 2026-03-13 08:04:30,055 - lava_callback - ERROR - Error processing callback for node 69b121fa9d191758182539a1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:50872 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36776 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50798 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39422 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48522 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48528 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48540 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38062 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46660 - "GET / HTTP/1.0" 200 OK 2026-03-13 08:05:42,700 - lava_callback - INFO - Received callback for node 69b122509d191758182539a8 INFO: 172.20.0.1:36152 - "POST /node/69b122509d191758182539a8 HTTP/1.0" 202 Accepted 2026-03-13 08:05:42,705 - lava_callback - INFO - Starting async job submit for node 69b122509d191758182539a8 2026-03-13 08:05:42,706 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 08:05:42,720 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b122509d191758182539a8 HTTP/1.1" 200 None 2026-03-13 08:05:42,721 - lava_callback - DEBUG - Processing node 69b122509d191758182539a8: baseline-arm64-clabbe 2026-03-13 08:05:42,721 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b122509d191758182539a8 2026-03-13 08:05:42,721 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 08:05:42,754 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b122509d191758182539a8 HTTP/1.1" 200 None 2026-03-13 08:05:42,756 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 08:05:42,769 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11758ef4424e487b2eb1 HTTP/1.1" 200 None 2026-03-13 08:05:42,770 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 08:05:42,790 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b122509d191758182539a8 HTTP/1.1" 200 None 2026-03-13 08:05:42,795 - lava_callback - ERROR - Error processing callback for node 69b122509d191758182539a8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:36168 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55140 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42304 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52066 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52068 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52082 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37934 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39098 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55718 - "GET / HTTP/1.0" 200 OK 2026-03-13 08:06:50,083 - lava_callback - INFO - Received callback for node 69b122959d191758182539b3 INFO: 172.20.0.1:39982 - "POST /node/69b122959d191758182539b3 HTTP/1.0" 202 Accepted 2026-03-13 08:06:50,088 - lava_callback - INFO - Starting async job submit for node 69b122959d191758182539b3 2026-03-13 08:06:50,089 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 08:06:50,108 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b122959d191758182539b3 HTTP/1.1" 200 None 2026-03-13 08:06:50,109 - lava_callback - DEBUG - Processing node 69b122959d191758182539b3: baseline-arm64-baylibre 2026-03-13 08:06:50,109 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b122959d191758182539b3 2026-03-13 08:06:50,110 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 08:06:50,124 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b122959d191758182539b3 HTTP/1.1" 200 None 2026-03-13 08:06:50,126 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 08:06:50,139 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e44efa461f55e59cdc8 HTTP/1.1" 200 None 2026-03-13 08:06:50,141 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 08:06:50,198 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b122959d191758182539b3 HTTP/1.1" 200 None 2026-03-13 08:06:50,204 - lava_callback - ERROR - Error processing callback for node 69b122959d191758182539b3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:39988 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34118 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58178 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58184 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58188 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39556 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40616 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57718 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50294 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55130 - "GET / HTTP/1.0" 200 OK 2026-03-13 08:08:10,171 - lava_callback - INFO - Received callback for node 69b122e19d191758182539b9 INFO: 172.20.0.1:38598 - "POST /node/69b122e19d191758182539b9 HTTP/1.0" 202 Accepted 2026-03-13 08:08:10,177 - lava_callback - INFO - Starting async job submit for node 69b122e19d191758182539b9 2026-03-13 08:08:10,177 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 08:08:10,191 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b122e19d191758182539b9 HTTP/1.1" 200 None 2026-03-13 08:08:10,192 - lava_callback - DEBUG - Processing node 69b122e19d191758182539b9: baseline-arm64-baylibre 2026-03-13 08:08:10,192 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b122e19d191758182539b9 2026-03-13 08:08:10,193 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 08:08:10,206 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b122e19d191758182539b9 HTTP/1.1" 200 None 2026-03-13 08:08:10,207 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 08:08:10,221 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e44efa461f55e59cdc8 HTTP/1.1" 200 None 2026-03-13 08:08:10,223 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 08:08:10,243 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b122e19d191758182539b9 HTTP/1.1" 200 None 2026-03-13 08:08:10,244 - lava_callback - ERROR - Error processing callback for node 69b122e19d191758182539b9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:38610 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38616 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38632 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46890 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55520 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39390 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55234 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32948 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52616 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52620 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52636 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51670 - "GET / HTTP/1.0" 200 OK 2026-03-13 08:09:30,219 - lava_callback - INFO - Received callback for node 69b1232d9d191758182539c5 INFO: 172.20.0.1:38004 - "POST /node/69b1232d9d191758182539c5 HTTP/1.0" 202 Accepted 2026-03-13 08:09:30,225 - lava_callback - INFO - Starting async job submit for node 69b1232d9d191758182539c5 2026-03-13 08:09:30,225 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 08:09:30,242 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b1232d9d191758182539c5 HTTP/1.1" 200 None 2026-03-13 08:09:30,243 - lava_callback - DEBUG - Processing node 69b1232d9d191758182539c5: baseline-arm64-baylibre 2026-03-13 08:09:30,244 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b1232d9d191758182539c5 2026-03-13 08:09:30,244 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 08:09:30,257 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b1232d9d191758182539c5 HTTP/1.1" 200 None 2026-03-13 08:09:30,258 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 08:09:30,271 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e44efa461f55e59cdc8 HTTP/1.1" 200 None 2026-03-13 08:09:30,273 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 08:09:30,294 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b1232d9d191758182539c5 HTTP/1.1" 200 None 2026-03-13 08:09:30,295 - lava_callback - ERROR - Error processing callback for node 69b1232d9d191758182539c5: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:38014 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57542 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36016 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48398 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39392 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39402 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39408 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56602 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43440 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56662 - "GET / HTTP/1.0" 200 OK 2026-03-13 08:10:50,234 - lava_callback - INFO - Received callback for node 69b123799d191758182539cd INFO: 172.20.0.1:43312 - "POST /node/69b123799d191758182539cd HTTP/1.0" 202 Accepted 2026-03-13 08:10:50,239 - lava_callback - INFO - Starting async job submit for node 69b123799d191758182539cd 2026-03-13 08:10:50,240 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 08:10:50,255 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b123799d191758182539cd HTTP/1.1" 200 None 2026-03-13 08:10:50,256 - lava_callback - DEBUG - Processing node 69b123799d191758182539cd: baseline-arm64-baylibre 2026-03-13 08:10:50,256 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b123799d191758182539cd 2026-03-13 08:10:50,257 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 08:10:50,299 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b123799d191758182539cd HTTP/1.1" 200 None 2026-03-13 08:10:50,301 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 08:10:50,315 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11758ef4424e487b2eb1 HTTP/1.1" 200 None 2026-03-13 08:10:50,317 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-13 08:10:50,339 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b123799d191758182539cd HTTP/1.1" 200 None 2026-03-13 08:10:50,349 - lava_callback - ERROR - Error processing callback for node 69b123799d191758182539cd: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:43316 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46220 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50318 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50322 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50330 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42352 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45020 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42382 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50070 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34510 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48964 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48970 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48984 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56510 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50126 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38390 - "GET / HTTP/1.0" 200 OK INFO: Shutting down INFO: Waiting for application shutdown. INFO: Application shutdown complete. INFO: Finished server process [1]