INFO: Started server process [1] INFO: Waiting for application startup. INFO: Application startup complete. INFO: Uvicorn running on http://0.0.0.0:8000 (Press CTRL+C to quit) INFO: 172.21.0.1:59864 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47942 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43428 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42850 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42866 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34324 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50720 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50724 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33990 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57030 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40046 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35438 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35442 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57436 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41300 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41316 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44700 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32958 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36674 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49688 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49704 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33870 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48556 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48566 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42310 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55148 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58702 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41606 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41620 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45058 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48614 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48630 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60342 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58752 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42542 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47702 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47710 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40082 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49480 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49494 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42902 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43774 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52502 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35422 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35436 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42846 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46722 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46734 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37490 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39374 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40486 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33634 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33638 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55584 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47734 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47736 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49394 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48420 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58420 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32912 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32920 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55566 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41002 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41004 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59864 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33440 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47376 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35822 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35824 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36414 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44936 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44948 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47382 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41978 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39210 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37854 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37858 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49210 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50328 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50338 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43490 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60194 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60968 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51762 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51770 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54006 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44084 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44088 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52756 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37052 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35298 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36058 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36074 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54690 - "GET / HTTP/1.0" 200 OK 2026-03-26 08:26:14,843 - lava_callback - INFO - Received callback for node 69c24a9e4a2c5046eb87f8da 2026-03-26 08:26:14,850 - lava_callback - INFO - Starting async job submit for node 69c24a9e4a2c5046eb87f8da INFO: 172.21.0.1:32936 - "POST /node/69c24a9e4a2c5046eb87f8da HTTP/1.0" 202 Accepted 2026-03-26 08:26:14,852 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:26:14,877 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c24a9e4a2c5046eb87f8da HTTP/1.1" 200 None 2026-03-26 08:26:14,878 - lava_callback - DEBUG - Processing node 69c24a9e4a2c5046eb87f8da: baseline-arm64-baylibre 2026-03-26 08:26:14,879 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c24a9e4a2c5046eb87f8da 2026-03-26 08:26:14,880 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:26:14,897 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c24a9e4a2c5046eb87f8da HTTP/1.1" 200 None 2026-03-26 08:26:14,899 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:26:15,096 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc92e6b4744551b27f9fd3 HTTP/1.1" 200 None 2026-03-26 08:26:15,098 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:26:15,121 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c24a9e4a2c5046eb87f8da HTTP/1.1" 200 None 2026-03-26 08:26:15,123 - lava_callback - ERROR - Error processing callback for node 69c24a9e4a2c5046eb87f8da: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:32950 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:32956 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44932 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48868 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36864 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47566 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47580 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56868 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35766 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35776 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54562 - "GET / HTTP/1.0" 200 OK 2026-03-26 08:27:34,766 - lava_callback - INFO - Received callback for node 69c24af34a2c5046eb87f8ee INFO: 172.21.0.1:55290 - "POST /node/69c24af34a2c5046eb87f8ee HTTP/1.0" 202 Accepted 2026-03-26 08:27:34,772 - lava_callback - INFO - Starting async job submit for node 69c24af34a2c5046eb87f8ee 2026-03-26 08:27:34,773 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:27:34,788 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c24af34a2c5046eb87f8ee HTTP/1.1" 200 None 2026-03-26 08:27:34,788 - lava_callback - DEBUG - Processing node 69c24af34a2c5046eb87f8ee: baseline-arm64-baylibre 2026-03-26 08:27:34,789 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c24af34a2c5046eb87f8ee 2026-03-26 08:27:34,789 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:27:34,802 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c24af34a2c5046eb87f8ee HTTP/1.1" 200 None 2026-03-26 08:27:34,804 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:27:34,816 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc92e6b4744551b27f9fd3 HTTP/1.1" 200 None 2026-03-26 08:27:34,818 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:27:34,838 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c24af34a2c5046eb87f8ee HTTP/1.1" 200 None 2026-03-26 08:27:34,839 - lava_callback - ERROR - Error processing callback for node 69c24af34a2c5046eb87f8ee: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:55302 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35990 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46954 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46966 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39964 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47862 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47870 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45912 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54426 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34864 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52634 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52636 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48380 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41990 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41998 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41794 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60120 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60168 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37980 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37984 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52212 - "GET / HTTP/1.0" 200 OK 2026-03-26 08:30:14,944 - lava_callback - INFO - Received callback for node 69c24b8f4a2c5046eb87f8fb INFO: 172.21.0.1:48564 - "POST /node/69c24b8f4a2c5046eb87f8fb HTTP/1.0" 202 Accepted 2026-03-26 08:30:14,950 - lava_callback - INFO - Starting async job submit for node 69c24b8f4a2c5046eb87f8fb 2026-03-26 08:30:14,951 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:30:14,966 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c24b8f4a2c5046eb87f8fb HTTP/1.1" 200 None 2026-03-26 08:30:14,967 - lava_callback - DEBUG - Processing node 69c24b8f4a2c5046eb87f8fb: baseline-arm64-baylibre 2026-03-26 08:30:14,967 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c24b8f4a2c5046eb87f8fb 2026-03-26 08:30:14,968 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:30:14,982 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c24b8f4a2c5046eb87f8fb HTTP/1.1" 200 None 2026-03-26 08:30:14,984 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:30:14,999 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc92e6b4744551b27f9fd3 HTTP/1.1" 200 None 2026-03-26 08:30:15,001 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:30:15,022 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c24b8f4a2c5046eb87f8fb HTTP/1.1" 200 None 2026-03-26 08:30:15,027 - lava_callback - ERROR - Error processing callback for node 69c24b8f4a2c5046eb87f8fb: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:48568 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48584 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41194 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53658 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42780 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34934 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34944 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59622 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51840 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51844 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54536 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51658 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46910 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54778 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54794 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33370 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40982 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-26 08:32:15,278 - lava_callback - INFO - Received callback for node 69c24c074a2c5046eb87f911 INFO: 172.21.0.1:40988 - "POST /node/69c24c074a2c5046eb87f911 HTTP/1.0" 202 Accepted 2026-03-26 08:32:15,283 - lava_callback - INFO - Starting async job submit for node 69c24c074a2c5046eb87f911 2026-03-26 08:32:15,284 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:32:15,299 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c24c074a2c5046eb87f911 HTTP/1.1" 200 None 2026-03-26 08:32:15,300 - lava_callback - DEBUG - Processing node 69c24c074a2c5046eb87f911: baseline-arm64-baylibre 2026-03-26 08:32:15,300 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c24c074a2c5046eb87f911 2026-03-26 08:32:15,301 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:32:15,326 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c24c074a2c5046eb87f911 HTTP/1.1" 200 None 2026-03-26 08:32:15,328 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:32:15,341 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc92e6b4744551b27f9fd3 HTTP/1.1" 200 None 2026-03-26 08:32:15,342 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:32:15,363 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c24c074a2c5046eb87f911 HTTP/1.1" 200 None 2026-03-26 08:32:15,364 - lava_callback - ERROR - Error processing callback for node 69c24c074a2c5046eb87f911: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-26 08:32:16,432 - lava_callback - INFO - Received callback for node 69c4ee96f63ce05331e2aa82 INFO: 172.21.0.1:40996 - "POST /node/69c4ee96f63ce05331e2aa82 HTTP/1.0" 202 Accepted 2026-03-26 08:32:16,438 - lava_callback - INFO - Starting async job submit for node 69c4ee96f63ce05331e2aa82 2026-03-26 08:32:16,461 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:32:16,475 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ee96f63ce05331e2aa82 HTTP/1.1" 200 None 2026-03-26 08:32:16,476 - lava_callback - DEBUG - Processing node 69c4ee96f63ce05331e2aa82: baseline-arm INFO: 172.21.0.1:40998 - "GET / HTTP/1.0" 200 OK 2026-03-26 08:32:16,668 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:32:16,885 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:32:16,886 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69c4ee96f63ce05331e2aa82/log.txt.gz 2026-03-26 08:32:16,995 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:32:17,211 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:32:17,213 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c4ee96f63ce05331e2aa82/lava-logs.yaml 2026-03-26 08:32:17,218 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:32:17,375 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:32:17,377 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c4ee96f63ce05331e2aa82/lava_callback.json.gz 2026-03-26 08:32:17,401 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:32:17,415 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ee96f63ce05331e2aa82 HTTP/1.1" 200 None 2026-03-26 08:32:17,416 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:32:17,430 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebd0f63ce05331e2a489 HTTP/1.1" 200 None 2026-03-26 08:32:17,431 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:32:17,458 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4ee96f63ce05331e2aa82 HTTP/1.1" 200 None 2026-03-26 08:32:17,463 - lava_callback - INFO - Completed processing callback for node 69c4ee96f63ce05331e2aa82 INFO: 172.21.0.1:56886 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36088 - "GET / HTTP/1.0" 200 OK 2026-03-26 08:32:45,125 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:32:45,176 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:43238 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38980 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38986 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56100 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41880 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41886 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40244 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39038 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57082 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33516 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33530 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54226 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59136 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59144 - "GET / HTTP/1.0" 200 OK 2026-03-26 08:34:18,854 - lava_callback - INFO - Received callback for node 69c24c874a2c5046eb87fed9 INFO: 172.21.0.1:59156 - "POST /node/69c24c874a2c5046eb87fed9 HTTP/1.0" 202 Accepted 2026-03-26 08:34:18,860 - lava_callback - INFO - Starting async job submit for node 69c24c874a2c5046eb87fed9 2026-03-26 08:34:18,861 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:34:18,886 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c24c874a2c5046eb87fed9 HTTP/1.1" 200 None 2026-03-26 08:34:18,887 - lava_callback - DEBUG - Processing node 69c24c874a2c5046eb87fed9: baseline-arm64-clabbe 2026-03-26 08:34:18,887 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c24c874a2c5046eb87fed9 2026-03-26 08:34:18,888 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:34:18,915 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c24c874a2c5046eb87fed9 HTTP/1.1" 200 None 2026-03-26 08:34:18,916 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:34:18,930 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc92e6b4744551b27f9fd3 HTTP/1.1" 200 None 2026-03-26 08:34:18,932 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:34:18,982 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c24c874a2c5046eb87fed9 HTTP/1.1" 200 None 2026-03-26 08:34:18,988 - lava_callback - ERROR - Error processing callback for node 69c24c874a2c5046eb87fed9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:38700 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32848 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49478 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58616 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58626 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50768 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52742 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52754 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52184 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42006 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40856 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52154 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52160 - "GET / HTTP/1.0" 200 OK 2026-03-26 08:35:58,910 - lava_callback - INFO - Received callback for node 69c24cde4a2c5046eb87feea INFO: 172.21.0.1:52162 - "POST /node/69c24cde4a2c5046eb87feea HTTP/1.0" 202 Accepted 2026-03-26 08:35:58,917 - lava_callback - INFO - Starting async job submit for node 69c24cde4a2c5046eb87feea 2026-03-26 08:35:58,918 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:35:58,944 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c24cde4a2c5046eb87feea HTTP/1.1" 200 None 2026-03-26 08:35:58,946 - lava_callback - DEBUG - Processing node 69c24cde4a2c5046eb87feea: baseline-arm64-clabbe 2026-03-26 08:35:58,946 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c24cde4a2c5046eb87feea 2026-03-26 08:35:58,947 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:35:58,982 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c24cde4a2c5046eb87feea HTTP/1.1" 200 None 2026-03-26 08:35:58,984 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:35:58,998 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc92e6b4744551b27f9fd3 HTTP/1.1" 200 None 2026-03-26 08:35:59,001 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:35:59,032 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c24cde4a2c5046eb87feea HTTP/1.1" 200 None 2026-03-26 08:35:59,033 - lava_callback - ERROR - Error processing callback for node 69c24cde4a2c5046eb87feea: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:56638 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38826 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38828 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43282 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55466 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54116 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43208 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43224 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41026 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52678 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52688 - "GET / HTTP/1.0" 200 OK 2026-03-26 08:37:18,950 - lava_callback - INFO - Received callback for node 69c24d304a2c5046eb87fefe INFO: 172.21.0.1:52696 - "POST /node/69c24d304a2c5046eb87fefe HTTP/1.0" 202 Accepted 2026-03-26 08:37:18,956 - lava_callback - INFO - Starting async job submit for node 69c24d304a2c5046eb87fefe 2026-03-26 08:37:18,957 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:37:18,974 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c24d304a2c5046eb87fefe HTTP/1.1" 200 None 2026-03-26 08:37:18,975 - lava_callback - DEBUG - Processing node 69c24d304a2c5046eb87fefe: baseline-arm64-clabbe 2026-03-26 08:37:18,976 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c24d304a2c5046eb87fefe 2026-03-26 08:37:18,977 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:37:18,992 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c24d304a2c5046eb87fefe HTTP/1.1" 200 None 2026-03-26 08:37:18,993 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:37:19,025 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc92e6b4744551b27f9fd3 HTTP/1.1" 200 None 2026-03-26 08:37:19,028 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:37:19,060 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c24d304a2c5046eb87fefe HTTP/1.1" 200 None 2026-03-26 08:37:19,062 - lava_callback - ERROR - Error processing callback for node 69c24d304a2c5046eb87fefe: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:48176 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50632 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36578 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44776 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44782 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58228 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51484 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51496 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46396 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58492 - "GET / HTTP/1.0" 200 OK 2026-03-26 08:38:38,971 - lava_callback - INFO - Received callback for node 69c24d834a2c5046eb87ff32 INFO: 172.21.0.1:58508 - "POST /node/69c24d834a2c5046eb87ff32 HTTP/1.0" 202 Accepted 2026-03-26 08:38:38,980 - lava_callback - INFO - Starting async job submit for node 69c24d834a2c5046eb87ff32 2026-03-26 08:38:38,983 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:38:39,041 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c24d834a2c5046eb87ff32 HTTP/1.1" 200 None 2026-03-26 08:38:39,042 - lava_callback - DEBUG - Processing node 69c24d834a2c5046eb87ff32: baseline-arm64-clabbe 2026-03-26 08:38:39,042 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c24d834a2c5046eb87ff32 2026-03-26 08:38:39,043 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:38:39,068 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c24d834a2c5046eb87ff32 HTTP/1.1" 200 None 2026-03-26 08:38:39,071 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:38:39,099 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc92e6b4744551b27f9fd3 HTTP/1.1" 200 None 2026-03-26 08:38:39,102 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:38:39,143 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c24d834a2c5046eb87ff32 HTTP/1.1" 200 None 2026-03-26 08:38:39,145 - lava_callback - ERROR - Error processing callback for node 69c24d834a2c5046eb87ff32: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:53436 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55306 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55320 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50118 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51804 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51810 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59920 - "GET / HTTP/1.0" 200 OK 2026-03-26 08:39:35,383 - lava_callback - INFO - Received callback for node 69c24dbf4a2c5046eb87ff48 INFO: 172.21.0.1:58912 - "POST /node/69c24dbf4a2c5046eb87ff48 HTTP/1.0" 202 Accepted 2026-03-26 08:39:35,389 - lava_callback - INFO - Starting async job submit for node 69c24dbf4a2c5046eb87ff48 2026-03-26 08:39:35,390 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:39:35,404 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c24dbf4a2c5046eb87ff48 HTTP/1.1" 200 None 2026-03-26 08:39:35,405 - lava_callback - DEBUG - Processing node 69c24dbf4a2c5046eb87ff48: baseline-arm-baylibre 2026-03-26 08:39:35,405 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c24dbf4a2c5046eb87ff48 2026-03-26 08:39:35,406 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:39:35,420 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c24dbf4a2c5046eb87ff48 HTTP/1.1" 200 None 2026-03-26 08:39:35,421 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:39:35,436 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ba5f6d537ae4d3c8236e46 HTTP/1.1" 200 None 2026-03-26 08:39:35,437 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:39:35,470 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c24dbf4a2c5046eb87ff48 HTTP/1.1" 200 None 2026-03-26 08:39:35,476 - lava_callback - ERROR - Error processing callback for node 69c24dbf4a2c5046eb87ff48: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:58924 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52662 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48632 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48644 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46858 - "GET / HTTP/1.0" 200 OK 2026-03-26 08:40:15,105 - lava_callback - INFO - Received callback for node 69c4f0dbf63ce05331e2aad2 INFO: 172.21.0.1:49786 - "POST /node/69c4f0dbf63ce05331e2aad2 HTTP/1.0" 202 Accepted 2026-03-26 08:40:15,114 - lava_callback - INFO - Starting async job submit for node 69c4f0dbf63ce05331e2aad2 2026-03-26 08:40:15,119 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:40:15,142 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f0dbf63ce05331e2aad2 HTTP/1.1" 200 None 2026-03-26 08:40:15,144 - lava_callback - DEBUG - Processing node 69c4f0dbf63ce05331e2aad2: baseline-arm 2026-03-26 08:40:15,152 - lava_callback - WARNING - Failed to upload log 2026-03-26 08:40:15,157 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.21.0.1:49788 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-26 08:40:15,401 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:40:15,403 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c4f0dbf63ce05331e2aad2/lava-logs.yaml 2026-03-26 08:40:15,405 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:40:15,601 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:40:15,602 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c4f0dbf63ce05331e2aad2/lava_callback.json.gz 2026-03-26 08:40:15,605 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:40:15,620 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f0dbf63ce05331e2aad2 HTTP/1.1" 200 None 2026-03-26 08:40:15,622 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:40:15,643 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebcbf63ce05331e2a47f HTTP/1.1" 200 None 2026-03-26 08:40:15,645 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:40:15,672 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4f0dbf63ce05331e2aad2 HTTP/1.1" 200 None 2026-03-26 08:40:15,677 - lava_callback - INFO - Completed processing callback for node 69c4f0dbf63ce05331e2aad2 2026-03-26 08:40:16,272 - lava_callback - INFO - Received callback for node 69c4f0d4f63ce05331e2aabd INFO: 172.21.0.1:49792 - "POST /node/69c4f0d4f63ce05331e2aabd HTTP/1.0" 202 Accepted 2026-03-26 08:40:16,278 - lava_callback - INFO - Starting async job submit for node 69c4f0d4f63ce05331e2aabd 2026-03-26 08:40:16,295 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:40:16,310 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f0d4f63ce05331e2aabd HTTP/1.1" 200 None 2026-03-26 08:40:16,311 - lava_callback - DEBUG - Processing node 69c4f0d4f63ce05331e2aabd: baseline-x86 INFO: 172.21.0.1:49804 - "GET / HTTP/1.0" 200 OK 2026-03-26 08:40:16,636 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:40:16,760 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:40:16,761 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69c4f0d4f63ce05331e2aabd/log.txt.gz 2026-03-26 08:40:16,865 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:40:17,045 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:40:17,047 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69c4f0d4f63ce05331e2aabd/lava-logs.yaml 2026-03-26 08:40:17,053 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:40:17,195 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:40:17,197 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69c4f0d4f63ce05331e2aabd/lava_callback.json.gz 2026-03-26 08:40:17,215 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:40:17,229 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f0d4f63ce05331e2aabd HTTP/1.1" 200 None 2026-03-26 08:40:17,230 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:40:17,243 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebecf63ce05331e2a4ae HTTP/1.1" 200 None 2026-03-26 08:40:17,244 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:40:17,289 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4f0d4f63ce05331e2aabd HTTP/1.1" 200 None 2026-03-26 08:40:17,309 - lava_callback - INFO - Completed processing callback for node 69c4f0d4f63ce05331e2aabd 2026-03-26 08:40:25,253 - lava_callback - INFO - Received callback for node 69c4f0d2f63ce05331e2aab4 INFO: 172.21.0.1:48990 - "POST /node/69c4f0d2f63ce05331e2aab4 HTTP/1.0" 202 Accepted 2026-03-26 08:40:25,263 - lava_callback - INFO - Starting async job submit for node 69c4f0d2f63ce05331e2aab4 2026-03-26 08:40:25,288 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:40:25,302 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f0d2f63ce05331e2aab4 HTTP/1.1" 200 None 2026-03-26 08:40:25,303 - lava_callback - DEBUG - Processing node 69c4f0d2f63ce05331e2aab4: baseline-x86 2026-03-26 08:40:25,506 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:40:25,723 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:40:25,725 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69c4f0d2f63ce05331e2aab4/log.txt.gz 2026-03-26 08:40:25,827 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:40:26,011 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:40:26,013 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69c4f0d2f63ce05331e2aab4/lava-logs.yaml 2026-03-26 08:40:26,018 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:40:26,179 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:40:26,180 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69c4f0d2f63ce05331e2aab4/lava_callback.json.gz 2026-03-26 08:40:26,200 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:40:26,216 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f0d2f63ce05331e2aab4 HTTP/1.1" 200 None 2026-03-26 08:40:26,217 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:40:26,239 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebecf63ce05331e2a4ae HTTP/1.1" 200 None 2026-03-26 08:40:26,241 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:40:26,267 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4f0d2f63ce05331e2aab4 HTTP/1.1" 200 None 2026-03-26 08:40:26,269 - lava_callback - INFO - Completed processing callback for node 69c4f0d2f63ce05331e2aab4 INFO: 172.21.0.1:48996 - "GET / HTTP/1.0" 200 OK 2026-03-26 08:40:35,231 - lava_callback - INFO - Received callback for node 69c4f0dcf63ce05331e2aad5 INFO: 172.21.0.1:55084 - "POST /node/69c4f0dcf63ce05331e2aad5 HTTP/1.0" 202 Accepted 2026-03-26 08:40:35,277 - lava_callback - INFO - Starting async job submit for node 69c4f0dcf63ce05331e2aad5 2026-03-26 08:40:35,296 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:40:35,311 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f0dcf63ce05331e2aad5 HTTP/1.1" 200 None 2026-03-26 08:40:35,312 - lava_callback - DEBUG - Processing node 69c4f0dcf63ce05331e2aad5: baseline-arm 2026-03-26 08:40:35,457 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:40:35,606 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:40:35,607 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69c4f0dcf63ce05331e2aad5/log.txt.gz 2026-03-26 08:40:35,684 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:40:35,846 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:40:35,847 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c4f0dcf63ce05331e2aad5/lava-logs.yaml 2026-03-26 08:40:35,851 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:40:35,997 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:40:35,999 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c4f0dcf63ce05331e2aad5/lava_callback.json.gz 2026-03-26 08:40:36,020 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:40:36,034 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f0dcf63ce05331e2aad5 HTTP/1.1" 200 None 2026-03-26 08:40:36,036 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:40:36,049 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebcbf63ce05331e2a47f HTTP/1.1" 200 None 2026-03-26 08:40:36,051 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:40:36,086 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4f0dcf63ce05331e2aad5 HTTP/1.1" 200 None 2026-03-26 08:40:36,087 - lava_callback - INFO - Completed processing callback for node 69c4f0dcf63ce05331e2aad5 INFO: 172.21.0.1:55090 - "GET / HTTP/1.0" 200 OK 2026-03-26 08:40:39,000 - lava_callback - INFO - Received callback for node 69c24df94a2c5046eb87ff5e INFO: 172.21.0.1:55098 - "POST /node/69c24df94a2c5046eb87ff5e HTTP/1.0" 202 Accepted 2026-03-26 08:40:39,007 - lava_callback - INFO - Starting async job submit for node 69c24df94a2c5046eb87ff5e 2026-03-26 08:40:39,008 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:40:39,025 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c24df94a2c5046eb87ff5e HTTP/1.1" 200 None 2026-03-26 08:40:39,026 - lava_callback - DEBUG - Processing node 69c24df94a2c5046eb87ff5e: baseline-arm64-clabbe 2026-03-26 08:40:39,026 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c24df94a2c5046eb87ff5e 2026-03-26 08:40:39,027 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:40:39,043 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c24df94a2c5046eb87ff5e HTTP/1.1" 200 None 2026-03-26 08:40:39,045 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:40:39,062 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc92e6b4744551b27f9fd3 HTTP/1.1" 200 None 2026-03-26 08:40:39,064 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:40:39,085 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c24df94a2c5046eb87ff5e HTTP/1.1" 200 None 2026-03-26 08:40:39,091 - lava_callback - ERROR - Error processing callback for node 69c24df94a2c5046eb87ff5e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-26 08:40:45,179 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:40:45,204 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 INFO: 172.21.0.1:43912 - "GET / HTTP/1.0" 200 OK 2026-03-26 08:40:56,036 - lava_callback - INFO - Received callback for node 69c4f0dff63ce05331e2aade INFO: 172.21.0.1:60912 - "POST /node/69c4f0dff63ce05331e2aade HTTP/1.0" 202 Accepted 2026-03-26 08:40:56,086 - lava_callback - INFO - Starting async job submit for node 69c4f0dff63ce05331e2aade 2026-03-26 08:40:56,103 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:40:56,117 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f0dff63ce05331e2aade HTTP/1.1" 200 None 2026-03-26 08:40:56,118 - lava_callback - DEBUG - Processing node 69c4f0dff63ce05331e2aade: baseline-arm 2026-03-26 08:40:56,305 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:40:56,437 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:40:56,439 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69c4f0dff63ce05331e2aade/log.txt.gz 2026-03-26 08:40:56,513 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.21.0.1:60916 - "GET / HTTP/1.0" 200 OK 2026-03-26 08:40:56,687 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:40:56,688 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c4f0dff63ce05331e2aade/lava-logs.yaml 2026-03-26 08:40:56,692 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:40:56,882 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:40:56,883 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c4f0dff63ce05331e2aade/lava_callback.json.gz 2026-03-26 08:40:56,901 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:40:56,916 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f0dff63ce05331e2aade HTTP/1.1" 200 None 2026-03-26 08:40:56,917 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:40:56,931 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebcbf63ce05331e2a47f HTTP/1.1" 200 None 2026-03-26 08:40:56,932 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:40:56,956 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4f0dff63ce05331e2aade HTTP/1.1" 200 None 2026-03-26 08:40:56,961 - lava_callback - INFO - Completed processing callback for node 69c4f0dff63ce05331e2aade INFO: 172.21.0.1:60924 - "GET / HTTP/1.0" 200 OK 2026-03-26 08:41:00,719 - lava_callback - INFO - Received callback for node 69c4f0def63ce05331e2aadb INFO: 172.21.0.1:46158 - "POST /node/69c4f0def63ce05331e2aadb HTTP/1.0" 202 Accepted 2026-03-26 08:41:00,725 - lava_callback - INFO - Starting async job submit for node 69c4f0def63ce05331e2aadb 2026-03-26 08:41:00,789 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:41:00,818 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f0def63ce05331e2aadb HTTP/1.1" 200 None 2026-03-26 08:41:00,820 - lava_callback - DEBUG - Processing node 69c4f0def63ce05331e2aadb: baseline-arm 2026-03-26 08:41:01,044 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:41:01,171 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:41:01,172 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69c4f0def63ce05331e2aadb/log.txt.gz 2026-03-26 08:41:01,318 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:41:01,570 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:41:01,572 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c4f0def63ce05331e2aadb/lava-logs.yaml 2026-03-26 08:41:01,577 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:41:01,708 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:41:01,710 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c4f0def63ce05331e2aadb/lava_callback.json.gz 2026-03-26 08:41:01,744 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:41:01,773 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f0def63ce05331e2aadb HTTP/1.1" 200 None 2026-03-26 08:41:01,775 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:41:01,800 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebcbf63ce05331e2a47f HTTP/1.1" 200 None 2026-03-26 08:41:01,803 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:41:01,844 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4f0def63ce05331e2aadb HTTP/1.1" 200 None 2026-03-26 08:41:01,846 - lava_callback - INFO - Completed processing callback for node 69c4f0def63ce05331e2aadb INFO: 172.21.0.1:46166 - "GET / HTTP/1.0" 200 OK 2026-03-26 08:41:06,748 - lava_callback - INFO - Received callback for node 69c4f0d3f63ce05331e2aab7 INFO: 172.21.0.1:46170 - "POST /node/69c4f0d3f63ce05331e2aab7 HTTP/1.0" 202 Accepted 2026-03-26 08:41:06,765 - lava_callback - INFO - Starting async job submit for node 69c4f0d3f63ce05331e2aab7 2026-03-26 08:41:06,802 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:41:06,817 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f0d3f63ce05331e2aab7 HTTP/1.1" 200 None 2026-03-26 08:41:06,818 - lava_callback - DEBUG - Processing node 69c4f0d3f63ce05331e2aab7: baseline-x86 2026-03-26 08:41:07,104 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:41:07,228 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:41:07,230 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69c4f0d3f63ce05331e2aab7/log.txt.gz 2026-03-26 08:41:07,471 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:41:07,681 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:41:07,683 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69c4f0d3f63ce05331e2aab7/lava-logs.yaml 2026-03-26 08:41:07,691 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:41:07,850 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:41:07,852 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69c4f0d3f63ce05331e2aab7/lava_callback.json.gz 2026-03-26 08:41:07,871 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:41:07,887 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f0d3f63ce05331e2aab7 HTTP/1.1" 200 None 2026-03-26 08:41:07,888 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:41:07,901 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebecf63ce05331e2a4ae HTTP/1.1" 200 None 2026-03-26 08:41:07,903 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:41:07,928 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4f0d3f63ce05331e2aab7 HTTP/1.1" 200 None 2026-03-26 08:41:07,929 - lava_callback - INFO - Completed processing callback for node 69c4f0d3f63ce05331e2aab7 INFO: 172.21.0.1:40772 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-26 08:41:15,206 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:41:15,222 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:40782 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38224 - "GET / HTTP/1.0" 200 OK 2026-03-26 08:41:29,351 - lava_callback - INFO - Received callback for node 69c4f0dbf63ce05331e2aacf INFO: 172.21.0.1:38238 - "POST /node/69c4f0dbf63ce05331e2aacf HTTP/1.0" 202 Accepted 2026-03-26 08:41:29,359 - lava_callback - INFO - Starting async job submit for node 69c4f0dbf63ce05331e2aacf 2026-03-26 08:41:29,380 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:41:29,404 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f0dbf63ce05331e2aacf HTTP/1.1" 200 None 2026-03-26 08:41:29,405 - lava_callback - DEBUG - Processing node 69c4f0dbf63ce05331e2aacf: baseline-arm 2026-03-26 08:41:29,601 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:41:29,714 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:41:29,716 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69c4f0dbf63ce05331e2aacf/log.txt.gz 2026-03-26 08:41:29,836 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:41:30,188 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:41:30,190 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c4f0dbf63ce05331e2aacf/lava-logs.yaml 2026-03-26 08:41:30,195 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:41:30,329 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:41:30,331 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c4f0dbf63ce05331e2aacf/lava_callback.json.gz 2026-03-26 08:41:30,354 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:41:30,374 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f0dbf63ce05331e2aacf HTTP/1.1" 200 None 2026-03-26 08:41:30,375 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:41:30,388 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebcbf63ce05331e2a47f HTTP/1.1" 200 None 2026-03-26 08:41:30,390 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:41:30,414 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4f0dbf63ce05331e2aacf HTTP/1.1" 200 None 2026-03-26 08:41:30,420 - lava_callback - INFO - Completed processing callback for node 69c4f0dbf63ce05331e2aacf 2026-03-26 08:41:35,470 - lava_callback - INFO - Received callback for node 69c24e3a4a2c5046eb87ff68 INFO: 172.21.0.1:38954 - "POST /node/69c24e3a4a2c5046eb87ff68 HTTP/1.0" 202 Accepted 2026-03-26 08:41:35,476 - lava_callback - INFO - Starting async job submit for node 69c24e3a4a2c5046eb87ff68 2026-03-26 08:41:35,476 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:41:35,490 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c24e3a4a2c5046eb87ff68 HTTP/1.1" 200 None 2026-03-26 08:41:35,491 - lava_callback - DEBUG - Processing node 69c24e3a4a2c5046eb87ff68: baseline-arm-baylibre 2026-03-26 08:41:35,491 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c24e3a4a2c5046eb87ff68 2026-03-26 08:41:35,492 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:41:35,505 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c24e3a4a2c5046eb87ff68 HTTP/1.1" 200 None 2026-03-26 08:41:35,507 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:41:35,524 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ba5f6d537ae4d3c8236e46 HTTP/1.1" 200 None 2026-03-26 08:41:35,525 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:41:35,546 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c24e3a4a2c5046eb87ff68 HTTP/1.1" 200 None 2026-03-26 08:41:35,548 - lava_callback - ERROR - Error processing callback for node 69c24e3a4a2c5046eb87ff68: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:38966 - "GET / HTTP/1.0" 200 OK 2026-03-26 08:41:36,740 - lava_callback - INFO - Received callback for node 69c4f0e8f63ce05331e2aaef INFO: 172.21.0.1:38980 - "POST /node/69c4f0e8f63ce05331e2aaef HTTP/1.0" 202 Accepted 2026-03-26 08:41:36,746 - lava_callback - INFO - Starting async job submit for node 69c4f0e8f63ce05331e2aaef 2026-03-26 08:41:36,777 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:41:36,777 - lava_callback - INFO - Received callback for node 69c4f0eff63ce05331e2ab09 2026-03-26 08:41:36,783 - lava_callback - INFO - Starting async job submit for node 69c4f0eff63ce05331e2ab09 INFO: 172.21.0.1:38996 - "POST /node/69c4f0eff63ce05331e2ab09 HTTP/1.0" 202 Accepted 2026-03-26 08:41:36,811 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:41:36,817 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f0e8f63ce05331e2aaef HTTP/1.1" 200 None 2026-03-26 08:41:36,817 - lava_callback - DEBUG - Processing node 69c4f0e8f63ce05331e2aaef: baseline-arm-broonie 2026-03-26 08:41:36,859 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f0eff63ce05331e2ab09 HTTP/1.1" 200 None 2026-03-26 08:41:36,876 - lava_callback - DEBUG - Processing node 69c4f0eff63ce05331e2ab09: baseline-arm-clabbe 2026-03-26 08:41:37,255 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:41:37,257 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:41:37,387 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:41:37,389 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69c4f0e8f63ce05331e2aaef/log.txt.gz 2026-03-26 08:41:37,412 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:41:37,445 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69c4f0eff63ce05331e2ab09/log.txt.gz 2026-03-26 08:41:37,666 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:41:37,676 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:41:37,882 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:41:37,884 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69c4f0e8f63ce05331e2aaef/lava-logs.yaml 2026-03-26 08:41:37,885 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:41:37,887 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69c4f0eff63ce05331e2ab09/lava-logs.yaml 2026-03-26 08:41:37,890 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:41:37,892 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:41:38,037 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:41:38,039 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69c4f0eff63ce05331e2ab09/lava_callback.json.gz 2026-03-26 08:41:38,064 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:41:38,078 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f0eff63ce05331e2ab09 HTTP/1.1" 200 None 2026-03-26 08:41:38,080 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:41:38,091 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:41:38,093 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69c4f0e8f63ce05331e2aaef/lava_callback.json.gz 2026-03-26 08:41:38,104 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebcbf63ce05331e2a47f HTTP/1.1" 200 None 2026-03-26 08:41:38,118 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:41:38,118 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:41:38,134 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f0e8f63ce05331e2aaef HTTP/1.1" 200 None 2026-03-26 08:41:38,135 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:41:38,154 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4f0eff63ce05331e2ab09 HTTP/1.1" 200 None 2026-03-26 08:41:38,156 - lava_callback - INFO - Completed processing callback for node 69c4f0eff63ce05331e2ab09 2026-03-26 08:41:38,205 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebcbf63ce05331e2a47f HTTP/1.1" 200 None 2026-03-26 08:41:38,208 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:41:38,247 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4f0e8f63ce05331e2aaef HTTP/1.1" 200 None 2026-03-26 08:41:38,249 - lava_callback - INFO - Completed processing callback for node 69c4f0e8f63ce05331e2aaef 2026-03-26 08:41:38,281 - lava_callback - INFO - Received callback for node 69c4f0e9f63ce05331e2aaf5 INFO: 172.21.0.1:39002 - "POST /node/69c4f0e9f63ce05331e2aaf5 HTTP/1.0" 202 Accepted 2026-03-26 08:41:38,298 - lava_callback - INFO - Starting async job submit for node 69c4f0e9f63ce05331e2aaf5 2026-03-26 08:41:38,345 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:41:38,368 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f0e9f63ce05331e2aaf5 HTTP/1.1" 200 None 2026-03-26 08:41:38,368 - lava_callback - DEBUG - Processing node 69c4f0e9f63ce05331e2aaf5: baseline-arm-broonie 2026-03-26 08:41:38,666 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:41:38,778 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:41:38,780 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69c4f0e9f63ce05331e2aaf5/log.txt.gz 2026-03-26 08:41:38,912 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:41:39,096 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:41:39,098 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69c4f0e9f63ce05331e2aaf5/lava-logs.yaml 2026-03-26 08:41:39,103 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:41:39,235 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:41:39,237 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69c4f0e9f63ce05331e2aaf5/lava_callback.json.gz 2026-03-26 08:41:39,260 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:41:39,274 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f0e9f63ce05331e2aaf5 HTTP/1.1" 200 None 2026-03-26 08:41:39,275 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:41:39,288 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebcbf63ce05331e2a47f HTTP/1.1" 200 None 2026-03-26 08:41:39,290 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:41:39,321 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4f0e9f63ce05331e2aaf5 HTTP/1.1" 200 None 2026-03-26 08:41:39,326 - lava_callback - INFO - Completed processing callback for node 69c4f0e9f63ce05331e2aaf5 2026-03-26 08:41:44,418 - lava_callback - INFO - Received callback for node 69c4f0fdf63ce05331e2ab6d INFO: 172.21.0.1:37308 - "POST /node/69c4f0fdf63ce05331e2ab6d HTTP/1.0" 202 Accepted 2026-03-26 08:41:44,470 - lava_callback - INFO - Starting async job submit for node 69c4f0fdf63ce05331e2ab6d 2026-03-26 08:41:44,493 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:41:44,513 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f0fdf63ce05331e2ab6d HTTP/1.1" 200 None 2026-03-26 08:41:44,514 - lava_callback - DEBUG - Processing node 69c4f0fdf63ce05331e2ab6d: baseline-arm-pengutronix 2026-03-26 08:41:44,661 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:41:44,828 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:41:44,830 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c4f0fdf63ce05331e2ab6d/log.txt.gz 2026-03-26 08:41:44,932 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:41:45,217 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:41:45,219 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c4f0fdf63ce05331e2ab6d/lava-logs.yaml 2026-03-26 08:41:45,225 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:41:45,225 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:41:45,248 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 2026-03-26 08:41:45,359 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:41:45,362 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c4f0fdf63ce05331e2ab6d/lava_callback.json.gz 2026-03-26 08:41:45,389 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:41:45,425 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f0fdf63ce05331e2ab6d HTTP/1.1" 200 None 2026-03-26 08:41:45,427 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:41:45,441 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebcbf63ce05331e2a47f HTTP/1.1" 200 None 2026-03-26 08:41:45,443 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:41:45,477 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4f0fdf63ce05331e2ab6d HTTP/1.1" 200 None 2026-03-26 08:41:45,482 - lava_callback - INFO - Completed processing callback for node 69c4f0fdf63ce05331e2ab6d 2026-03-26 08:41:45,621 - lava_callback - INFO - Received callback for node 69c4f0f9f63ce05331e2ab60 INFO: 172.21.0.1:37312 - "POST /node/69c4f0f9f63ce05331e2ab60 HTTP/1.0" 202 Accepted 2026-03-26 08:41:45,630 - lava_callback - INFO - Starting async job submit for node 69c4f0f9f63ce05331e2ab60 2026-03-26 08:41:45,673 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:41:45,704 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f0f9f63ce05331e2ab60 HTTP/1.1" 200 None 2026-03-26 08:41:45,706 - lava_callback - DEBUG - Processing node 69c4f0f9f63ce05331e2ab60: baseline-arm-pengutronix 2026-03-26 08:41:46,001 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:41:46,171 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:41:46,173 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c4f0f9f63ce05331e2ab60/log.txt.gz 2026-03-26 08:41:46,294 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.21.0.1:37314 - "GET / HTTP/1.0" 200 OK 2026-03-26 08:41:46,706 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:41:46,708 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c4f0f9f63ce05331e2ab60/lava-logs.yaml 2026-03-26 08:41:46,714 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:41:46,784 - lava_callback - INFO - Received callback for node 69c4f0faf63ce05331e2ab67 INFO: 172.21.0.1:37324 - "POST /node/69c4f0faf63ce05331e2ab67 HTTP/1.0" 202 Accepted 2026-03-26 08:41:46,791 - lava_callback - INFO - Starting async job submit for node 69c4f0faf63ce05331e2ab67 2026-03-26 08:41:46,814 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:41:46,829 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f0faf63ce05331e2ab67 HTTP/1.1" 200 None 2026-03-26 08:41:46,830 - lava_callback - DEBUG - Processing node 69c4f0faf63ce05331e2ab67: baseline-arm-pengutronix 2026-03-26 08:41:46,866 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:41:46,914 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c4f0f9f63ce05331e2ab60/lava_callback.json.gz 2026-03-26 08:41:46,965 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:41:47,020 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:41:47,024 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f0f9f63ce05331e2ab60 HTTP/1.1" 200 None 2026-03-26 08:41:47,026 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:41:47,041 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebcbf63ce05331e2a47f HTTP/1.1" 200 None 2026-03-26 08:41:47,043 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:41:47,084 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4f0f9f63ce05331e2ab60 HTTP/1.1" 200 None 2026-03-26 08:41:47,086 - lava_callback - INFO - Completed processing callback for node 69c4f0f9f63ce05331e2ab60 2026-03-26 08:41:47,327 - lava_callback - INFO - Received callback for node 69c4f0ebf63ce05331e2aaf9 INFO: 172.21.0.1:37328 - "POST /node/69c4f0ebf63ce05331e2aaf9 HTTP/1.0" 202 Accepted 2026-03-26 08:41:47,337 - lava_callback - INFO - Starting async job submit for node 69c4f0ebf63ce05331e2aaf9 2026-03-26 08:41:47,398 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:41:47,427 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f0ebf63ce05331e2aaf9 HTTP/1.1" 200 None 2026-03-26 08:41:47,428 - lava_callback - DEBUG - Processing node 69c4f0ebf63ce05331e2aaf9: baseline-arm-broonie 2026-03-26 08:41:47,577 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:41:47,595 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c4f0faf63ce05331e2ab67/log.txt.gz 2026-03-26 08:41:47,916 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:41:47,981 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:41:48,372 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:41:48,374 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69c4f0ebf63ce05331e2aaf9/log.txt.gz 2026-03-26 08:41:48,475 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:41:48,505 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c4f0faf63ce05331e2ab67/lava-logs.yaml 2026-03-26 08:41:48,568 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:41:48,569 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:41:48,708 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:41:48,710 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c4f0faf63ce05331e2ab67/lava_callback.json.gz 2026-03-26 08:41:48,734 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:41:48,773 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f0faf63ce05331e2ab67 HTTP/1.1" 200 None 2026-03-26 08:41:48,774 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:41:48,791 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebcbf63ce05331e2a47f HTTP/1.1" 200 None 2026-03-26 08:41:48,794 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:41:48,794 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:41:48,796 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69c4f0ebf63ce05331e2aaf9/lava-logs.yaml 2026-03-26 08:41:48,815 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:41:48,860 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4f0faf63ce05331e2ab67 HTTP/1.1" 200 None 2026-03-26 08:41:48,862 - lava_callback - INFO - Completed processing callback for node 69c4f0faf63ce05331e2ab67 2026-03-26 08:41:49,321 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:41:49,323 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69c4f0ebf63ce05331e2aaf9/lava_callback.json.gz 2026-03-26 08:41:49,361 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:41:49,385 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f0ebf63ce05331e2aaf9 HTTP/1.1" 200 None 2026-03-26 08:41:49,387 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:41:49,406 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebcbf63ce05331e2a47f HTTP/1.1" 200 None 2026-03-26 08:41:49,407 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:41:49,435 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4f0ebf63ce05331e2aaf9 HTTP/1.1" 200 None 2026-03-26 08:41:49,436 - lava_callback - INFO - Completed processing callback for node 69c4f0ebf63ce05331e2aaf9 INFO: 172.21.0.1:56772 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56774 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42092 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38878 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-26 08:42:15,251 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:42:15,268 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 INFO: 172.21.0.1:38894 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51998 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47502 - "GET / HTTP/1.0" 200 OK 2026-03-26 08:42:39,156 - lava_callback - INFO - Received callback for node 69c24e734a2c5046eb87ff7c INFO: 172.21.0.1:47506 - "POST /node/69c24e734a2c5046eb87ff7c HTTP/1.0" 202 Accepted 2026-03-26 08:42:39,162 - lava_callback - INFO - Starting async job submit for node 69c24e734a2c5046eb87ff7c 2026-03-26 08:42:39,163 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:42:39,177 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c24e734a2c5046eb87ff7c HTTP/1.1" 200 None 2026-03-26 08:42:39,178 - lava_callback - DEBUG - Processing node 69c24e734a2c5046eb87ff7c: baseline-arm64-clabbe 2026-03-26 08:42:39,178 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c24e734a2c5046eb87ff7c 2026-03-26 08:42:39,179 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:42:39,192 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c24e734a2c5046eb87ff7c HTTP/1.1" 200 None 2026-03-26 08:42:39,193 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:42:39,207 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc92e6b4744551b27f9fd3 HTTP/1.1" 200 None 2026-03-26 08:42:39,209 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:42:39,229 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c24e734a2c5046eb87ff7c HTTP/1.1" 200 None 2026-03-26 08:42:39,235 - lava_callback - ERROR - Error processing callback for node 69c24e734a2c5046eb87ff7c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:56372 - "GET / HTTP/1.0" 200 OK 2026-03-26 08:42:50,814 - lava_callback - INFO - Received callback for node 69c4f17cf63ce05331e2ac50 INFO: 172.21.0.1:34196 - "POST /node/69c4f17cf63ce05331e2ac50 HTTP/1.0" 202 Accepted 2026-03-26 08:42:50,824 - lava_callback - INFO - Starting async job submit for node 69c4f17cf63ce05331e2ac50 2026-03-26 08:42:50,829 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:42:50,857 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f17cf63ce05331e2ac50 HTTP/1.1" 200 None 2026-03-26 08:42:50,858 - lava_callback - DEBUG - Processing node 69c4f17cf63ce05331e2ac50: baseline-arm 2026-03-26 08:42:50,868 - lava_callback - WARNING - Failed to upload log 2026-03-26 08:42:50,873 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:42:51,098 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:42:51,100 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c4f17cf63ce05331e2ac50/lava-logs.yaml 2026-03-26 08:42:51,101 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:42:51,544 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:42:51,547 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c4f17cf63ce05331e2ac50/lava_callback.json.gz 2026-03-26 08:42:51,552 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:42:51,574 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f17cf63ce05331e2ac50 HTTP/1.1" 200 None 2026-03-26 08:42:51,577 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:42:51,598 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebcbf63ce05331e2a47f HTTP/1.1" 200 None 2026-03-26 08:42:51,600 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:42:51,627 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4f17cf63ce05331e2ac50 HTTP/1.1" 200 None 2026-03-26 08:42:51,635 - lava_callback - INFO - Completed processing callback for node 69c4f17cf63ce05331e2ac50 INFO: 172.21.0.1:34204 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34216 - "GET / HTTP/1.0" 200 OK 2026-03-26 08:43:03,804 - lava_callback - INFO - Received callback for node 69c4f0f0f63ce05331e2ab0e INFO: 172.21.0.1:39324 - "POST /node/69c4f0f0f63ce05331e2ab0e HTTP/1.0" 202 Accepted 2026-03-26 08:43:03,814 - lava_callback - INFO - Starting async job submit for node 69c4f0f0f63ce05331e2ab0e 2026-03-26 08:43:03,857 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:43:03,887 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f0f0f63ce05331e2ab0e HTTP/1.1" 200 None 2026-03-26 08:43:03,888 - lava_callback - DEBUG - Processing node 69c4f0f0f63ce05331e2ab0e: baseline-arm-clabbe 2026-03-26 08:43:04,276 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:43:04,955 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:43:04,956 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69c4f0f0f63ce05331e2ab0e/log.txt.gz 2026-03-26 08:43:05,194 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:43:05,968 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:43:05,971 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69c4f0f0f63ce05331e2ab0e/lava-logs.yaml 2026-03-26 08:43:05,977 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.21.0.1:39326 - "GET / HTTP/1.0" 200 OK 2026-03-26 08:43:06,655 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:43:06,657 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69c4f0f0f63ce05331e2ab0e/lava_callback.json.gz 2026-03-26 08:43:06,703 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:43:06,735 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f0f0f63ce05331e2ab0e HTTP/1.1" 200 None 2026-03-26 08:43:06,738 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:43:06,765 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebcbf63ce05331e2a47f HTTP/1.1" 200 None 2026-03-26 08:43:06,767 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:43:06,807 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4f0f0f63ce05331e2ab0e HTTP/1.1" 200 None 2026-03-26 08:43:06,809 - lava_callback - INFO - Completed processing callback for node 69c4f0f0f63ce05331e2ab0e 2026-03-26 08:43:07,401 - lava_callback - INFO - Received callback for node 69c4f0e7f63ce05331e2aaec INFO: 172.21.0.1:39334 - "POST /node/69c4f0e7f63ce05331e2aaec HTTP/1.0" 202 Accepted 2026-03-26 08:43:07,411 - lava_callback - INFO - Starting async job submit for node 69c4f0e7f63ce05331e2aaec 2026-03-26 08:43:07,461 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:43:07,487 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f0e7f63ce05331e2aaec HTTP/1.1" 200 None 2026-03-26 08:43:07,488 - lava_callback - DEBUG - Processing node 69c4f0e7f63ce05331e2aaec: baseline-arm-broonie 2026-03-26 08:43:07,930 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:43:08,359 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:43:08,361 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69c4f0e7f63ce05331e2aaec/log.txt.gz 2026-03-26 08:43:08,574 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:43:09,429 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:43:09,431 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69c4f0e7f63ce05331e2aaec/lava-logs.yaml 2026-03-26 08:43:09,438 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:43:10,040 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:43:10,043 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69c4f0e7f63ce05331e2aaec/lava_callback.json.gz 2026-03-26 08:43:10,092 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:43:10,115 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f0e7f63ce05331e2aaec HTTP/1.1" 200 None 2026-03-26 08:43:10,118 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:43:10,141 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebcbf63ce05331e2a47f HTTP/1.1" 200 None 2026-03-26 08:43:10,143 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:43:10,196 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4f0e7f63ce05331e2aaec HTTP/1.1" 200 None 2026-03-26 08:43:10,198 - lava_callback - INFO - Completed processing callback for node 69c4f0e7f63ce05331e2aaec INFO: 172.21.0.1:53192 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-26 08:43:15,270 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:43:15,306 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:53206 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37180 - "GET / HTTP/1.0" 200 OK 2026-03-26 08:43:35,544 - lava_callback - INFO - Received callback for node 69c24eab4a2c5046eb87ffa5 INFO: 172.21.0.1:53078 - "POST /node/69c24eab4a2c5046eb87ffa5 HTTP/1.0" 202 Accepted 2026-03-26 08:43:35,553 - lava_callback - INFO - Starting async job submit for node 69c24eab4a2c5046eb87ffa5 2026-03-26 08:43:35,554 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:43:35,589 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c24eab4a2c5046eb87ffa5 HTTP/1.1" 200 None 2026-03-26 08:43:35,591 - lava_callback - DEBUG - Processing node 69c24eab4a2c5046eb87ffa5: baseline-arm-baylibre 2026-03-26 08:43:35,591 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c24eab4a2c5046eb87ffa5 2026-03-26 08:43:35,592 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:43:35,643 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c24eab4a2c5046eb87ffa5 HTTP/1.1" 200 None 2026-03-26 08:43:35,646 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:43:35,674 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248d74a2c5046eb87f890 HTTP/1.1" 200 None 2026-03-26 08:43:35,676 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:43:35,725 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c24eab4a2c5046eb87ffa5 HTTP/1.1" 200 None 2026-03-26 08:43:35,727 - lava_callback - ERROR - Error processing callback for node 69c24eab4a2c5046eb87ffa5: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-26 08:43:36,223 - lava_callback - INFO - Received callback for node 69c24ead4a2c5046eb87ffa8 INFO: 172.21.0.1:53084 - "POST /node/69c24ead4a2c5046eb87ffa8 HTTP/1.0" 202 Accepted 2026-03-26 08:43:36,232 - lava_callback - INFO - Starting async job submit for node 69c24ead4a2c5046eb87ffa8 2026-03-26 08:43:36,234 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:43:36,262 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c24ead4a2c5046eb87ffa8 HTTP/1.1" 200 None 2026-03-26 08:43:36,263 - lava_callback - DEBUG - Processing node 69c24ead4a2c5046eb87ffa8: baseline-arm-baylibre 2026-03-26 08:43:36,263 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c24ead4a2c5046eb87ffa8 2026-03-26 08:43:36,264 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:43:36,289 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c24ead4a2c5046eb87ffa8 HTTP/1.1" 200 None 2026-03-26 08:43:36,291 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:43:36,318 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248d74a2c5046eb87f890 HTTP/1.1" 200 None 2026-03-26 08:43:36,320 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:43:36,360 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c24ead4a2c5046eb87ffa8 HTTP/1.1" 200 None 2026-03-26 08:43:36,362 - lava_callback - ERROR - Error processing callback for node 69c24ead4a2c5046eb87ffa8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:53088 - "GET / HTTP/1.0" 200 OK 2026-03-26 08:43:36,904 - lava_callback - INFO - Received callback for node 69c24eb04a2c5046eb87ffad INFO: 172.21.0.1:53096 - "POST /node/69c24eb04a2c5046eb87ffad HTTP/1.0" 202 Accepted 2026-03-26 08:43:36,912 - lava_callback - INFO - Starting async job submit for node 69c24eb04a2c5046eb87ffad 2026-03-26 08:43:36,913 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:43:36,941 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c24eb04a2c5046eb87ffad HTTP/1.1" 200 None 2026-03-26 08:43:36,942 - lava_callback - DEBUG - Processing node 69c24eb04a2c5046eb87ffad: baseline-arm-baylibre 2026-03-26 08:43:36,943 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c24eb04a2c5046eb87ffad 2026-03-26 08:43:36,944 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:43:36,969 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c24eb04a2c5046eb87ffad HTTP/1.1" 200 None 2026-03-26 08:43:36,971 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:43:36,996 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248d74a2c5046eb87f890 HTTP/1.1" 200 None 2026-03-26 08:43:36,999 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:43:37,035 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c24eb04a2c5046eb87ffad HTTP/1.1" 200 None 2026-03-26 08:43:37,036 - lava_callback - ERROR - Error processing callback for node 69c24eb04a2c5046eb87ffad: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-26 08:43:37,593 - lava_callback - INFO - Received callback for node 69c24eb24a2c5046eb87ffb0 INFO: 172.21.0.1:53106 - "POST /node/69c24eb24a2c5046eb87ffb0 HTTP/1.0" 202 Accepted 2026-03-26 08:43:37,601 - lava_callback - INFO - Starting async job submit for node 69c24eb24a2c5046eb87ffb0 2026-03-26 08:43:37,607 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:43:37,635 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c24eb24a2c5046eb87ffb0 HTTP/1.1" 200 None 2026-03-26 08:43:37,636 - lava_callback - DEBUG - Processing node 69c24eb24a2c5046eb87ffb0: baseline-arm-baylibre 2026-03-26 08:43:37,637 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c24eb24a2c5046eb87ffb0 2026-03-26 08:43:37,637 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:43:37,661 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c24eb24a2c5046eb87ffb0 HTTP/1.1" 200 None 2026-03-26 08:43:37,663 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:43:37,688 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248d74a2c5046eb87f890 HTTP/1.1" 200 None 2026-03-26 08:43:37,691 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:43:37,744 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c24eb24a2c5046eb87ffb0 HTTP/1.1" 200 None 2026-03-26 08:43:37,745 - lava_callback - ERROR - Error processing callback for node 69c24eb24a2c5046eb87ffb0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-26 08:43:38,283 - lava_callback - INFO - Received callback for node 69c24eb54a2c5046eb87ffbd INFO: 172.21.0.1:53108 - "POST /node/69c24eb54a2c5046eb87ffbd HTTP/1.0" 202 Accepted 2026-03-26 08:43:38,293 - lava_callback - INFO - Starting async job submit for node 69c24eb54a2c5046eb87ffbd 2026-03-26 08:43:38,294 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:43:38,325 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c24eb54a2c5046eb87ffbd HTTP/1.1" 200 None 2026-03-26 08:43:38,326 - lava_callback - DEBUG - Processing node 69c24eb54a2c5046eb87ffbd: baseline-arm-baylibre 2026-03-26 08:43:38,326 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c24eb54a2c5046eb87ffbd 2026-03-26 08:43:38,327 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:43:38,352 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c24eb54a2c5046eb87ffbd HTTP/1.1" 200 None 2026-03-26 08:43:38,354 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:43:38,378 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248d74a2c5046eb87f890 HTTP/1.1" 200 None 2026-03-26 08:43:38,380 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:43:38,423 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c24eb54a2c5046eb87ffbd HTTP/1.1" 200 None 2026-03-26 08:43:38,425 - lava_callback - ERROR - Error processing callback for node 69c24eb54a2c5046eb87ffbd: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-26 08:43:38,949 - lava_callback - INFO - Received callback for node 69c24eb54a2c5046eb87ffbf INFO: 172.21.0.1:53110 - "POST /node/69c24eb54a2c5046eb87ffbf HTTP/1.0" 202 Accepted 2026-03-26 08:43:38,958 - lava_callback - INFO - Starting async job submit for node 69c24eb54a2c5046eb87ffbf 2026-03-26 08:43:38,961 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:43:38,986 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c24eb54a2c5046eb87ffbf HTTP/1.1" 200 None 2026-03-26 08:43:38,987 - lava_callback - DEBUG - Processing node 69c24eb54a2c5046eb87ffbf: baseline-arm-baylibre 2026-03-26 08:43:38,987 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c24eb54a2c5046eb87ffbf 2026-03-26 08:43:38,988 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:43:39,010 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c24eb54a2c5046eb87ffbf HTTP/1.1" 200 None 2026-03-26 08:43:39,013 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:43:39,037 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ba5f6d537ae4d3c8236e46 HTTP/1.1" 200 None 2026-03-26 08:43:39,040 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:43:39,089 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c24eb54a2c5046eb87ffbf HTTP/1.1" 200 None 2026-03-26 08:43:39,091 - lava_callback - ERROR - Error processing callback for node 69c24eb54a2c5046eb87ffbf: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-26 08:43:42,202 - lava_callback - INFO - Received callback for node 69c4f114f63ce05331e2abac INFO: 172.21.0.1:39284 - "POST /node/69c4f114f63ce05331e2abac HTTP/1.0" 202 Accepted 2026-03-26 08:43:42,213 - lava_callback - INFO - Starting async job submit for node 69c4f114f63ce05331e2abac 2026-03-26 08:43:42,254 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:43:42,279 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f114f63ce05331e2abac HTTP/1.1" 200 None 2026-03-26 08:43:42,281 - lava_callback - DEBUG - Processing node 69c4f114f63ce05331e2abac: ltp-dio 2026-03-26 08:43:42,951 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:43:43,386 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:43:43,389 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-dio-69c4f114f63ce05331e2abac/log.txt.gz 2026-03-26 08:43:43,834 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:43:44,354 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:43:44,357 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-dio-69c4f114f63ce05331e2abac/lava-logs.yaml 2026-03-26 08:43:44,368 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:43:44,839 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:43:44,842 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-dio-69c4f114f63ce05331e2abac/lava_callback.json.gz 2026-03-26 08:43:44,887 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:43:44,911 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f114f63ce05331e2abac HTTP/1.1" 200 None 2026-03-26 08:43:44,913 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:43:44,934 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebcbf63ce05331e2a47f HTTP/1.1" 200 None 2026-03-26 08:43:44,937 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:43:44,973 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4f114f63ce05331e2abac HTTP/1.1" 200 None 2026-03-26 08:43:44,974 - lava_callback - INFO - Completed processing callback for node 69c4f114f63ce05331e2abac 2026-03-26 08:43:45,309 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:43:45,345 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:39294 - "GET / HTTP/1.0" 200 OK 2026-03-26 08:43:55,534 - lava_callback - INFO - Received callback for node 69c24eb74a2c5046eb87ffc3 INFO: 172.21.0.1:36568 - "POST /node/69c24eb74a2c5046eb87ffc3 HTTP/1.0" 202 Accepted 2026-03-26 08:43:55,545 - lava_callback - INFO - Starting async job submit for node 69c24eb74a2c5046eb87ffc3 2026-03-26 08:43:55,546 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:43:55,578 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c24eb74a2c5046eb87ffc3 HTTP/1.1" 200 None 2026-03-26 08:43:55,579 - lava_callback - DEBUG - Processing node 69c24eb74a2c5046eb87ffc3: baseline-arm-baylibre 2026-03-26 08:43:55,579 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c24eb74a2c5046eb87ffc3 2026-03-26 08:43:55,580 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:43:55,613 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c24eb74a2c5046eb87ffc3 HTTP/1.1" 200 None 2026-03-26 08:43:55,615 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:43:55,653 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248d74a2c5046eb87f890 HTTP/1.1" 200 None 2026-03-26 08:43:55,655 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:43:55,712 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c24eb74a2c5046eb87ffc3 HTTP/1.1" 200 None 2026-03-26 08:43:55,714 - lava_callback - ERROR - Error processing callback for node 69c24eb74a2c5046eb87ffc3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:36580 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36584 - "GET / HTTP/1.0" 200 OK 2026-03-26 08:43:59,163 - lava_callback - INFO - Received callback for node 69c24ebf4a2c5046eb87ffda INFO: 172.21.0.1:36590 - "POST /node/69c24ebf4a2c5046eb87ffda HTTP/1.0" 202 Accepted 2026-03-26 08:43:59,172 - lava_callback - INFO - Starting async job submit for node 69c24ebf4a2c5046eb87ffda 2026-03-26 08:43:59,173 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:43:59,196 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c24ebf4a2c5046eb87ffda HTTP/1.1" 200 None 2026-03-26 08:43:59,198 - lava_callback - DEBUG - Processing node 69c24ebf4a2c5046eb87ffda: baseline-arm-clabbe 2026-03-26 08:43:59,198 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c24ebf4a2c5046eb87ffda 2026-03-26 08:43:59,199 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:43:59,222 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c24ebf4a2c5046eb87ffda HTTP/1.1" 200 None 2026-03-26 08:43:59,224 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:43:59,247 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248d74a2c5046eb87f890 HTTP/1.1" 200 None 2026-03-26 08:43:59,249 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:43:59,295 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c24ebf4a2c5046eb87ffda HTTP/1.1" 200 None 2026-03-26 08:43:59,297 - lava_callback - ERROR - Error processing callback for node 69c24ebf4a2c5046eb87ffda: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-26 08:43:59,819 - lava_callback - INFO - Received callback for node 69c24ec44a2c5046eb87ffe7 INFO: 172.21.0.1:48416 - "POST /node/69c24ec44a2c5046eb87ffe7 HTTP/1.0" 202 Accepted 2026-03-26 08:43:59,828 - lava_callback - INFO - Starting async job submit for node 69c24ec44a2c5046eb87ffe7 2026-03-26 08:43:59,829 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:43:59,862 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c24ec44a2c5046eb87ffe7 HTTP/1.1" 200 None 2026-03-26 08:43:59,863 - lava_callback - DEBUG - Processing node 69c24ec44a2c5046eb87ffe7: baseline-arm-clabbe 2026-03-26 08:43:59,864 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c24ec44a2c5046eb87ffe7 2026-03-26 08:43:59,864 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:43:59,889 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c24ec44a2c5046eb87ffe7 HTTP/1.1" 200 None 2026-03-26 08:43:59,892 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:43:59,919 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248d74a2c5046eb87f890 HTTP/1.1" 200 None 2026-03-26 08:43:59,922 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:43:59,957 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c24ec44a2c5046eb87ffe7 HTTP/1.1" 200 None 2026-03-26 08:43:59,958 - lava_callback - ERROR - Error processing callback for node 69c24ec44a2c5046eb87ffe7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-26 08:44:00,488 - lava_callback - INFO - Received callback for node 69c24ec54a2c5046eb87ffea INFO: 172.21.0.1:48422 - "POST /node/69c24ec54a2c5046eb87ffea HTTP/1.0" 202 Accepted 2026-03-26 08:44:00,497 - lava_callback - INFO - Starting async job submit for node 69c24ec54a2c5046eb87ffea 2026-03-26 08:44:00,498 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:44:00,529 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c24ec54a2c5046eb87ffea HTTP/1.1" 200 None 2026-03-26 08:44:00,530 - lava_callback - DEBUG - Processing node 69c24ec54a2c5046eb87ffea: baseline-arm-clabbe 2026-03-26 08:44:00,530 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c24ec54a2c5046eb87ffea 2026-03-26 08:44:00,531 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:44:00,556 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c24ec54a2c5046eb87ffea HTTP/1.1" 200 None 2026-03-26 08:44:00,558 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:44:00,581 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248d74a2c5046eb87f890 HTTP/1.1" 200 None 2026-03-26 08:44:00,583 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:44:00,618 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c24ec54a2c5046eb87ffea HTTP/1.1" 200 None 2026-03-26 08:44:00,620 - lava_callback - ERROR - Error processing callback for node 69c24ec54a2c5046eb87ffea: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-26 08:44:01,152 - lava_callback - INFO - Received callback for node 69c24ec74a2c5046eb88000b INFO: 172.21.0.1:48434 - "POST /node/69c24ec74a2c5046eb88000b HTTP/1.0" 202 Accepted 2026-03-26 08:44:01,163 - lava_callback - INFO - Starting async job submit for node 69c24ec74a2c5046eb88000b 2026-03-26 08:44:01,165 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:44:01,189 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c24ec74a2c5046eb88000b HTTP/1.1" 200 None 2026-03-26 08:44:01,190 - lava_callback - DEBUG - Processing node 69c24ec74a2c5046eb88000b: baseline-arm-clabbe 2026-03-26 08:44:01,199 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c24ec74a2c5046eb88000b 2026-03-26 08:44:01,200 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:44:01,229 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c24ec74a2c5046eb88000b HTTP/1.1" 200 None 2026-03-26 08:44:01,232 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:44:01,259 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248d74a2c5046eb87f890 HTTP/1.1" 200 None 2026-03-26 08:44:01,261 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:44:01,317 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c24ec74a2c5046eb88000b HTTP/1.1" 200 None 2026-03-26 08:44:01,318 - lava_callback - ERROR - Error processing callback for node 69c24ec74a2c5046eb88000b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-26 08:44:01,798 - lava_callback - INFO - Received callback for node 69c24eca4a2c5046eb880016 INFO: 172.21.0.1:48440 - "POST /node/69c24eca4a2c5046eb880016 HTTP/1.0" 202 Accepted 2026-03-26 08:44:01,819 - lava_callback - INFO - Starting async job submit for node 69c24eca4a2c5046eb880016 2026-03-26 08:44:01,824 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:44:01,851 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c24eca4a2c5046eb880016 HTTP/1.1" 200 None 2026-03-26 08:44:01,853 - lava_callback - DEBUG - Processing node 69c24eca4a2c5046eb880016: baseline-arm-clabbe 2026-03-26 08:44:01,853 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c24eca4a2c5046eb880016 2026-03-26 08:44:01,854 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:44:01,878 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c24eca4a2c5046eb880016 HTTP/1.1" 200 None 2026-03-26 08:44:01,881 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:44:01,905 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248d74a2c5046eb87f890 HTTP/1.1" 200 None 2026-03-26 08:44:01,907 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:44:01,950 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c24eca4a2c5046eb880016 HTTP/1.1" 200 None 2026-03-26 08:44:01,951 - lava_callback - ERROR - Error processing callback for node 69c24eca4a2c5046eb880016: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:48452 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59960 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-26 08:44:15,534 - lava_callback - INFO - Received callback for node 69c24edc4a2c5046eb88003b INFO: 172.21.0.1:59964 - "POST /node/69c24edc4a2c5046eb88003b HTTP/1.0" 202 Accepted 2026-03-26 08:44:15,540 - lava_callback - INFO - Starting async job submit for node 69c24edc4a2c5046eb88003b 2026-03-26 08:44:15,541 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:44:15,555 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c24edc4a2c5046eb88003b HTTP/1.1" 200 None 2026-03-26 08:44:15,556 - lava_callback - DEBUG - Processing node 69c24edc4a2c5046eb88003b: baseline-arm-baylibre 2026-03-26 08:44:15,556 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c24edc4a2c5046eb88003b 2026-03-26 08:44:15,557 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:44:15,570 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c24edc4a2c5046eb88003b HTTP/1.1" 200 None 2026-03-26 08:44:15,571 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:44:15,585 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ba5f6d537ae4d3c8236e46 HTTP/1.1" 200 None 2026-03-26 08:44:15,587 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:44:15,607 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c24edc4a2c5046eb88003b HTTP/1.1" 200 None 2026-03-26 08:44:15,613 - lava_callback - ERROR - Error processing callback for node 69c24edc4a2c5046eb88003b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:59976 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56456 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37326 - "GET / HTTP/1.0" 200 OK 2026-03-26 08:44:42,891 - lava_callback - INFO - Received callback for node 69c4f0f3f63ce05331e2ab1e INFO: 172.21.0.1:60888 - "POST /node/69c4f0f3f63ce05331e2ab1e HTTP/1.0" 202 Accepted 2026-03-26 08:44:42,902 - lava_callback - INFO - Starting async job submit for node 69c4f0f3f63ce05331e2ab1e 2026-03-26 08:44:42,946 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:44:42,977 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f0f3f63ce05331e2ab1e HTTP/1.1" 200 None 2026-03-26 08:44:42,978 - lava_callback - DEBUG - Processing node 69c4f0f3f63ce05331e2ab1e: baseline-arm-clabbe 2026-03-26 08:44:43,186 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:44:43,482 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:44:43,484 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69c4f0f3f63ce05331e2ab1e/log.txt.gz 2026-03-26 08:44:43,610 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:44:44,986 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:44:44,988 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69c4f0f3f63ce05331e2ab1e/lava-logs.yaml 2026-03-26 08:44:44,994 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:44:45,156 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:44:45,158 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69c4f0f3f63ce05331e2ab1e/lava_callback.json.gz 2026-03-26 08:44:45,182 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:44:45,197 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f0f3f63ce05331e2ab1e HTTP/1.1" 200 None 2026-03-26 08:44:45,199 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:44:45,218 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebcbf63ce05331e2a47f HTTP/1.1" 200 None 2026-03-26 08:44:45,220 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:44:45,245 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4f0f3f63ce05331e2ab1e HTTP/1.1" 200 None 2026-03-26 08:44:45,246 - lava_callback - INFO - Completed processing callback for node 69c4f0f3f63ce05331e2ab1e 2026-03-26 08:44:45,348 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:44:45,366 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:60890 - "GET / HTTP/1.0" 200 OK 2026-03-26 08:44:47,560 - lava_callback - INFO - Received callback for node 69c4f110f63ce05331e2ab8d INFO: 172.21.0.1:60896 - "POST /node/69c4f110f63ce05331e2ab8d HTTP/1.0" 202 Accepted 2026-03-26 08:44:47,567 - lava_callback - INFO - Starting async job submit for node 69c4f110f63ce05331e2ab8d 2026-03-26 08:44:47,592 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:44:47,606 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f110f63ce05331e2ab8d HTTP/1.1" 200 None 2026-03-26 08:44:47,607 - lava_callback - DEBUG - Processing node 69c4f110f63ce05331e2ab8d: ltp-controllers 2026-03-26 08:44:48,037 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:44:48,206 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:44:48,208 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-controllers-69c4f110f63ce05331e2ab8d/log.txt.gz 2026-03-26 08:44:48,502 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:44:48,728 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:44:48,730 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-controllers-69c4f110f63ce05331e2ab8d/lava-logs.yaml 2026-03-26 08:44:48,740 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:44:48,892 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:44:48,894 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-controllers-69c4f110f63ce05331e2ab8d/lava_callback.json.gz 2026-03-26 08:44:48,923 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:44:48,937 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f110f63ce05331e2ab8d HTTP/1.1" 200 None 2026-03-26 08:44:48,939 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:44:48,956 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebcbf63ce05331e2a47f HTTP/1.1" 200 None 2026-03-26 08:44:48,958 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:44:48,991 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4f110f63ce05331e2ab8d HTTP/1.1" 200 None 2026-03-26 08:44:49,000 - lava_callback - INFO - Completed processing callback for node 69c4f110f63ce05331e2ab8d 2026-03-26 08:44:49,169 - lava_callback - INFO - Received callback for node 69c4f112f63ce05331e2ab91 INFO: 172.21.0.1:60910 - "POST /node/69c4f112f63ce05331e2ab91 HTTP/1.0" 202 Accepted 2026-03-26 08:44:49,177 - lava_callback - INFO - Starting async job submit for node 69c4f112f63ce05331e2ab91 2026-03-26 08:44:49,219 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:44:49,233 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f112f63ce05331e2ab91 HTTP/1.1" 200 None 2026-03-26 08:44:49,234 - lava_callback - DEBUG - Processing node 69c4f112f63ce05331e2ab91: ltp-cve 2026-03-26 08:44:49,612 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:44:49,767 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:44:49,769 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-cve-69c4f112f63ce05331e2ab91/log.txt.gz 2026-03-26 08:44:50,000 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:44:50,201 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:44:50,203 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-cve-69c4f112f63ce05331e2ab91/lava-logs.yaml 2026-03-26 08:44:50,212 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:44:50,358 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:44:50,360 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-cve-69c4f112f63ce05331e2ab91/lava_callback.json.gz 2026-03-26 08:44:50,385 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:44:50,402 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f112f63ce05331e2ab91 HTTP/1.1" 200 None 2026-03-26 08:44:50,404 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:44:50,417 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebcbf63ce05331e2a47f HTTP/1.1" 200 None 2026-03-26 08:44:50,419 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:44:50,442 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4f112f63ce05331e2ab91 HTTP/1.1" 200 None 2026-03-26 08:44:50,444 - lava_callback - INFO - Completed processing callback for node 69c4f112f63ce05331e2ab91 2026-03-26 08:44:50,805 - lava_callback - INFO - Received callback for node 69c4f120f63ce05331e2abcd INFO: 172.21.0.1:51976 - "POST /node/69c4f120f63ce05331e2abcd HTTP/1.0" 202 Accepted 2026-03-26 08:44:50,813 - lava_callback - INFO - Starting async job submit for node 69c4f120f63ce05331e2abcd 2026-03-26 08:44:50,853 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:44:50,868 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f120f63ce05331e2abcd HTTP/1.1" 200 None 2026-03-26 08:44:50,869 - lava_callback - DEBUG - Processing node 69c4f120f63ce05331e2abcd: ltp-syscalls 2026-03-26 08:44:51,608 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:44:51,747 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:44:51,749 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-syscalls-69c4f120f63ce05331e2abcd/log.txt.gz 2026-03-26 08:44:52,200 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:44:52,420 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:44:52,423 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-syscalls-69c4f120f63ce05331e2abcd/lava-logs.yaml 2026-03-26 08:44:52,440 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:44:52,609 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:44:52,611 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-syscalls-69c4f120f63ce05331e2abcd/lava_callback.json.gz 2026-03-26 08:44:52,638 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:44:52,663 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f120f63ce05331e2abcd HTTP/1.1" 200 None 2026-03-26 08:44:52,665 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:44:52,683 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebcbf63ce05331e2a47f HTTP/1.1" 200 None 2026-03-26 08:44:52,684 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:44:52,709 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4f120f63ce05331e2abcd HTTP/1.1" 200 None 2026-03-26 08:44:52,714 - lava_callback - INFO - Completed processing callback for node 69c4f120f63ce05331e2abcd 2026-03-26 08:44:55,717 - lava_callback - INFO - Received callback for node 69c24f044a2c5046eb8800ae INFO: 172.21.0.1:51990 - "POST /node/69c24f044a2c5046eb8800ae HTTP/1.0" 202 Accepted 2026-03-26 08:44:55,722 - lava_callback - INFO - Starting async job submit for node 69c24f044a2c5046eb8800ae 2026-03-26 08:44:55,723 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:44:55,737 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c24f044a2c5046eb8800ae HTTP/1.1" 200 None 2026-03-26 08:44:55,738 - lava_callback - DEBUG - Processing node 69c24f044a2c5046eb8800ae: baseline-arm-baylibre 2026-03-26 08:44:55,738 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c24f044a2c5046eb8800ae 2026-03-26 08:44:55,738 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:44:55,772 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c24f044a2c5046eb8800ae HTTP/1.1" 200 None 2026-03-26 08:44:55,773 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:44:55,788 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ba5f6d537ae4d3c8236e46 HTTP/1.1" 200 None 2026-03-26 08:44:55,789 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:44:55,811 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c24f044a2c5046eb8800ae HTTP/1.1" 200 None 2026-03-26 08:44:55,816 - lava_callback - ERROR - Error processing callback for node 69c24f044a2c5046eb8800ae: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:51996 - "GET / HTTP/1.0" 200 OK 2026-03-26 08:44:57,065 - lava_callback - INFO - Received callback for node 69c4f10ef63ce05331e2ab86 INFO: 172.21.0.1:52008 - "POST /node/69c4f10ef63ce05331e2ab86 HTTP/1.0" 202 Accepted 2026-03-26 08:44:57,071 - lava_callback - INFO - Starting async job submit for node 69c4f10ef63ce05331e2ab86 2026-03-26 08:44:57,099 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:44:57,125 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f10ef63ce05331e2ab86 HTTP/1.1" 200 None 2026-03-26 08:44:57,126 - lava_callback - DEBUG - Processing node 69c4f10ef63ce05331e2ab86: ltp-containers 2026-03-26 08:44:57,488 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:44:57,612 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:44:57,614 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-containers-69c4f10ef63ce05331e2ab86/log.txt.gz 2026-03-26 08:44:57,827 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:44:58,024 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:44:58,027 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-containers-69c4f10ef63ce05331e2ab86/lava-logs.yaml 2026-03-26 08:44:58,035 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:44:58,209 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:44:58,211 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-containers-69c4f10ef63ce05331e2ab86/lava_callback.json.gz 2026-03-26 08:44:58,237 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:44:58,252 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f10ef63ce05331e2ab86 HTTP/1.1" 200 None 2026-03-26 08:44:58,254 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:44:58,267 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebcbf63ce05331e2a47f HTTP/1.1" 200 None 2026-03-26 08:44:58,269 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:44:58,293 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4f10ef63ce05331e2ab86 HTTP/1.1" 200 None 2026-03-26 08:44:58,302 - lava_callback - INFO - Completed processing callback for node 69c4f10ef63ce05331e2ab86 INFO: 172.21.0.1:52010 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59230 - "GET / HTTP/1.0" 200 OK 2026-03-26 08:45:07,980 - lava_callback - INFO - Received callback for node 69c4f10df63ce05331e2ab83 INFO: 172.21.0.1:59240 - "POST /node/69c4f10df63ce05331e2ab83 HTTP/1.0" 202 Accepted 2026-03-26 08:45:07,987 - lava_callback - INFO - Starting async job submit for node 69c4f10df63ce05331e2ab83 2026-03-26 08:45:08,016 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:45:08,030 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f10df63ce05331e2ab83 HTTP/1.1" 200 None 2026-03-26 08:45:08,031 - lava_callback - DEBUG - Processing node 69c4f10df63ce05331e2ab83: ltp-cap_bounds 2026-03-26 08:45:08,410 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:45:08,533 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:45:08,535 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-cap_bounds-69c4f10df63ce05331e2ab83/log.txt.gz 2026-03-26 08:45:08,757 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:45:08,966 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:45:08,968 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-cap_bounds-69c4f10df63ce05331e2ab83/lava-logs.yaml 2026-03-26 08:45:08,977 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:45:09,135 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:45:09,136 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-cap_bounds-69c4f10df63ce05331e2ab83/lava_callback.json.gz 2026-03-26 08:45:09,162 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:45:09,177 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f10df63ce05331e2ab83 HTTP/1.1" 200 None 2026-03-26 08:45:09,179 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:45:09,193 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebcbf63ce05331e2a47f HTTP/1.1" 200 None 2026-03-26 08:45:09,194 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:45:09,219 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4f10df63ce05331e2ab83 HTTP/1.1" 200 None 2026-03-26 08:45:09,221 - lava_callback - INFO - Completed processing callback for node 69c4f10df63ce05331e2ab83 INFO: 172.21.0.1:57512 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-26 08:45:15,369 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:45:15,391 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 INFO: 172.21.0.1:57528 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44452 - "GET / HTTP/1.0" 200 OK 2026-03-26 08:45:35,650 - lava_callback - INFO - Received callback for node 69c24f2d4a2c5046eb8800e0 INFO: 172.21.0.1:34038 - "POST /node/69c24f2d4a2c5046eb8800e0 HTTP/1.0" 202 Accepted 2026-03-26 08:45:35,656 - lava_callback - INFO - Starting async job submit for node 69c24f2d4a2c5046eb8800e0 2026-03-26 08:45:35,657 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:45:35,672 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c24f2d4a2c5046eb8800e0 HTTP/1.1" 200 None 2026-03-26 08:45:35,672 - lava_callback - DEBUG - Processing node 69c24f2d4a2c5046eb8800e0: baseline-arm-baylibre 2026-03-26 08:45:35,673 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c24f2d4a2c5046eb8800e0 2026-03-26 08:45:35,673 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:45:35,686 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c24f2d4a2c5046eb8800e0 HTTP/1.1" 200 None 2026-03-26 08:45:35,688 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:45:35,701 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd026d05e4bbd6ba634c57 HTTP/1.1" 200 None 2026-03-26 08:45:35,703 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:45:35,723 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c24f2d4a2c5046eb8800e0 HTTP/1.1" 200 None 2026-03-26 08:45:35,725 - lava_callback - ERROR - Error processing callback for node 69c24f2d4a2c5046eb8800e0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:34044 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55098 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36598 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36604 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41030 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44494 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44510 - "GET / HTTP/1.0" 200 OK 2026-03-26 08:46:25,965 - lava_callback - INFO - Received callback for node 69c4f0f4f63ce05331e2ab21 INFO: 172.21.0.1:35428 - "POST /node/69c4f0f4f63ce05331e2ab21 HTTP/1.0" 202 Accepted 2026-03-26 08:46:25,971 - lava_callback - INFO - Starting async job submit for node 69c4f0f4f63ce05331e2ab21 2026-03-26 08:46:25,992 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:46:26,007 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f0f4f63ce05331e2ab21 HTTP/1.1" 200 None 2026-03-26 08:46:26,008 - lava_callback - DEBUG - Processing node 69c4f0f4f63ce05331e2ab21: baseline-arm-clabbe 2026-03-26 08:46:26,216 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:46:26,363 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:46:26,365 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69c4f0f4f63ce05331e2ab21/log.txt.gz 2026-03-26 08:46:26,496 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.21.0.1:35440 - "GET / HTTP/1.0" 200 OK 2026-03-26 08:46:26,709 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:46:26,711 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69c4f0f4f63ce05331e2ab21/lava-logs.yaml 2026-03-26 08:46:26,717 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:46:26,880 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:46:26,882 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69c4f0f4f63ce05331e2ab21/lava_callback.json.gz 2026-03-26 08:46:26,904 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:46:26,918 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f0f4f63ce05331e2ab21 HTTP/1.1" 200 None 2026-03-26 08:46:26,920 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:46:26,933 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebcbf63ce05331e2a47f HTTP/1.1" 200 None 2026-03-26 08:46:26,934 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:46:26,958 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4f0f4f63ce05331e2ab21 HTTP/1.1" 200 None 2026-03-26 08:46:26,966 - lava_callback - INFO - Completed processing callback for node 69c4f0f4f63ce05331e2ab21 INFO: 172.21.0.1:49302 - "GET / HTTP/1.0" 200 OK 2026-03-26 08:46:45,394 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:46:45,410 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:51964 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47880 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47896 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52708 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51866 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51872 - "GET / HTTP/1.0" 200 OK 2026-03-26 08:47:19,425 - lava_callback - INFO - Received callback for node 69c4f280f63ce05331e2ad1a INFO: 172.21.0.1:51874 - "POST /node/69c4f280f63ce05331e2ad1a HTTP/1.0" 202 Accepted 2026-03-26 08:47:19,431 - lava_callback - INFO - Starting async job submit for node 69c4f280f63ce05331e2ad1a 2026-03-26 08:47:19,434 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:47:19,449 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f280f63ce05331e2ad1a HTTP/1.1" 200 None 2026-03-26 08:47:19,450 - lava_callback - DEBUG - Processing node 69c4f280f63ce05331e2ad1a: baseline-arm 2026-03-26 08:47:19,455 - lava_callback - WARNING - Failed to upload log 2026-03-26 08:47:19,458 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:47:19,593 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:47:19,594 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c4f280f63ce05331e2ad1a/lava-logs.yaml 2026-03-26 08:47:19,596 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:47:19,717 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:47:19,719 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c4f280f63ce05331e2ad1a/lava_callback.json.gz 2026-03-26 08:47:19,722 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:47:19,737 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f280f63ce05331e2ad1a HTTP/1.1" 200 None 2026-03-26 08:47:19,738 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:47:19,752 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebcbf63ce05331e2a47f HTTP/1.1" 200 None 2026-03-26 08:47:19,754 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:47:19,777 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4f280f63ce05331e2ad1a HTTP/1.1" 200 None 2026-03-26 08:47:19,787 - lava_callback - INFO - Completed processing callback for node 69c4f280f63ce05331e2ad1a INFO: 172.21.0.1:47962 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36726 - "GET / HTTP/1.0" 200 OK 2026-03-26 08:47:45,412 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:47:45,429 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:60598 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33566 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33574 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33076 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36180 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36192 - "GET / HTTP/1.0" 200 OK 2026-03-26 08:48:19,486 - lava_callback - INFO - Received callback for node 69c24fc64a2c5046eb880135 INFO: 172.21.0.1:36206 - "POST /node/69c24fc64a2c5046eb880135 HTTP/1.0" 202 Accepted 2026-03-26 08:48:19,491 - lava_callback - INFO - Starting async job submit for node 69c24fc64a2c5046eb880135 2026-03-26 08:48:19,492 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:48:19,506 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c24fc64a2c5046eb880135 HTTP/1.1" 200 None 2026-03-26 08:48:19,507 - lava_callback - DEBUG - Processing node 69c24fc64a2c5046eb880135: baseline-arm-clabbe 2026-03-26 08:48:19,507 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c24fc64a2c5046eb880135 2026-03-26 08:48:19,508 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:48:19,520 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c24fc64a2c5046eb880135 HTTP/1.1" 200 None 2026-03-26 08:48:19,522 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:48:19,547 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd026d05e4bbd6ba634c57 HTTP/1.1" 200 None 2026-03-26 08:48:19,548 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:48:19,569 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c24fc64a2c5046eb880135 HTTP/1.1" 200 None 2026-03-26 08:48:19,596 - lava_callback - ERROR - Error processing callback for node 69c24fc64a2c5046eb880135: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:37604 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44444 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56162 - "GET / HTTP/1.0" 200 OK 2026-03-26 08:48:47,763 - lava_callback - INFO - Received callback for node 69c4f118f63ce05331e2abb5 INFO: 172.21.0.1:56176 - "POST /node/69c4f118f63ce05331e2abb5 HTTP/1.0" 202 Accepted 2026-03-26 08:48:47,769 - lava_callback - INFO - Starting async job submit for node 69c4f118f63ce05331e2abb5 2026-03-26 08:48:47,794 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:48:47,809 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f118f63ce05331e2abb5 HTTP/1.1" 200 None 2026-03-26 08:48:47,809 - lava_callback - DEBUG - Processing node 69c4f118f63ce05331e2abb5: ltp-input 2026-03-26 08:48:48,178 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:48:48,329 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:48:48,331 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-input-69c4f118f63ce05331e2abb5/log.txt.gz 2026-03-26 08:48:48,540 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:48:48,772 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:48:48,774 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-input-69c4f118f63ce05331e2abb5/lava-logs.yaml 2026-03-26 08:48:48,782 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:48:49,136 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:48:49,138 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-input-69c4f118f63ce05331e2abb5/lava_callback.json.gz 2026-03-26 08:48:49,164 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:48:49,178 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f118f63ce05331e2abb5 HTTP/1.1" 200 None 2026-03-26 08:48:49,179 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:48:49,192 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebcbf63ce05331e2a47f HTTP/1.1" 200 None 2026-03-26 08:48:49,193 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:48:49,217 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4f118f63ce05331e2abb5 HTTP/1.1" 200 None 2026-03-26 08:48:49,218 - lava_callback - INFO - Completed processing callback for node 69c4f118f63ce05331e2abb5 2026-03-26 08:48:49,369 - lava_callback - INFO - Received callback for node 69c4f11af63ce05331e2abbd INFO: 172.21.0.1:56188 - "POST /node/69c4f11af63ce05331e2abbd HTTP/1.0" 202 Accepted 2026-03-26 08:48:49,379 - lava_callback - INFO - Starting async job submit for node 69c4f11af63ce05331e2abbd 2026-03-26 08:48:49,421 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:48:49,440 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f11af63ce05331e2abbd HTTP/1.1" 200 None 2026-03-26 08:48:49,442 - lava_callback - DEBUG - Processing node 69c4f11af63ce05331e2abbd: ltp-ipc 2026-03-26 08:48:49,866 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:48:50,168 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:48:50,170 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-ipc-69c4f11af63ce05331e2abbd/log.txt.gz 2026-03-26 08:48:50,382 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:48:50,588 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:48:50,590 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-ipc-69c4f11af63ce05331e2abbd/lava-logs.yaml 2026-03-26 08:48:50,598 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:48:50,749 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:48:50,751 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-ipc-69c4f11af63ce05331e2abbd/lava_callback.json.gz 2026-03-26 08:48:50,777 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:48:50,795 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f11af63ce05331e2abbd HTTP/1.1" 200 None 2026-03-26 08:48:50,796 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:48:50,809 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebcbf63ce05331e2a47f HTTP/1.1" 200 None 2026-03-26 08:48:50,811 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:48:50,834 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4f11af63ce05331e2abbd HTTP/1.1" 200 None 2026-03-26 08:48:50,836 - lava_callback - INFO - Completed processing callback for node 69c4f11af63ce05331e2abbd INFO: 172.21.0.1:42458 - "GET / HTTP/1.0" 200 OK 2026-03-26 08:48:57,142 - lava_callback - INFO - Received callback for node 69c4f117f63ce05331e2abb2 INFO: 172.21.0.1:42462 - "POST /node/69c4f117f63ce05331e2abb2 HTTP/1.0" 202 Accepted 2026-03-26 08:48:57,149 - lava_callback - INFO - Starting async job submit for node 69c4f117f63ce05331e2abb2 2026-03-26 08:48:57,174 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:48:57,197 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f117f63ce05331e2abb2 HTTP/1.1" 200 None 2026-03-26 08:48:57,198 - lava_callback - DEBUG - Processing node 69c4f117f63ce05331e2abb2: ltp-fsx 2026-03-26 08:48:57,557 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:48:57,671 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:48:57,672 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-fsx-69c4f117f63ce05331e2abb2/log.txt.gz 2026-03-26 08:48:57,885 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:48:58,076 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:48:58,078 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-fsx-69c4f117f63ce05331e2abb2/lava-logs.yaml 2026-03-26 08:48:58,087 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:48:58,260 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:48:58,261 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-fsx-69c4f117f63ce05331e2abb2/lava_callback.json.gz 2026-03-26 08:48:58,287 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:48:58,310 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f117f63ce05331e2abb2 HTTP/1.1" 200 None 2026-03-26 08:48:58,312 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:48:58,325 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebcbf63ce05331e2a47f HTTP/1.1" 200 None 2026-03-26 08:48:58,327 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:48:58,350 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4f117f63ce05331e2abb2 HTTP/1.1" 200 None 2026-03-26 08:48:58,352 - lava_callback - INFO - Completed processing callback for node 69c4f117f63ce05331e2abb2 INFO: 172.21.0.1:42474 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56054 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49210 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-26 08:49:15,431 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:49:15,448 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:49222 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38828 - "GET / HTTP/1.0" 200 OK 2026-03-26 08:49:27,771 - lava_callback - INFO - Received callback for node 69c4f11bf63ce05331e2abc0 INFO: 172.21.0.1:38840 - "POST /node/69c4f11bf63ce05331e2abc0 HTTP/1.0" 202 Accepted 2026-03-26 08:49:27,777 - lava_callback - INFO - Starting async job submit for node 69c4f11bf63ce05331e2abc0 2026-03-26 08:49:27,802 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:49:27,816 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f11bf63ce05331e2abc0 HTTP/1.1" 200 None 2026-03-26 08:49:27,817 - lava_callback - DEBUG - Processing node 69c4f11bf63ce05331e2abc0: ltp-pty 2026-03-26 08:49:28,168 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:49:28,288 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:49:28,289 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-pty-69c4f11bf63ce05331e2abc0/log.txt.gz 2026-03-26 08:49:28,520 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:49:28,712 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:49:28,715 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-pty-69c4f11bf63ce05331e2abc0/lava-logs.yaml 2026-03-26 08:49:28,723 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:49:28,882 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:49:28,884 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-pty-69c4f11bf63ce05331e2abc0/lava_callback.json.gz 2026-03-26 08:49:28,909 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:49:28,924 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f11bf63ce05331e2abc0 HTTP/1.1" 200 None 2026-03-26 08:49:28,925 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:49:28,940 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebcbf63ce05331e2a47f HTTP/1.1" 200 None 2026-03-26 08:49:28,941 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:49:28,974 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4f11bf63ce05331e2abc0 HTTP/1.1" 200 None 2026-03-26 08:49:28,976 - lava_callback - INFO - Completed processing callback for node 69c4f11bf63ce05331e2abc0 INFO: 172.21.0.1:44248 - "GET / HTTP/1.0" 200 OK 2026-03-26 08:49:39,588 - lava_callback - INFO - Received callback for node 69c250124a2c5046eb88017a INFO: 172.21.0.1:44258 - "POST /node/69c250124a2c5046eb88017a HTTP/1.0" 202 Accepted 2026-03-26 08:49:39,597 - lava_callback - INFO - Starting async job submit for node 69c250124a2c5046eb88017a 2026-03-26 08:49:39,599 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:49:39,636 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c250124a2c5046eb88017a HTTP/1.1" 200 None 2026-03-26 08:49:39,638 - lava_callback - DEBUG - Processing node 69c250124a2c5046eb88017a: baseline-arm-clabbe 2026-03-26 08:49:39,638 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c250124a2c5046eb88017a 2026-03-26 08:49:39,639 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:49:39,668 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c250124a2c5046eb88017a HTTP/1.1" 200 None 2026-03-26 08:49:39,670 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:49:39,699 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd026d05e4bbd6ba634c57 HTTP/1.1" 200 None 2026-03-26 08:49:39,702 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:49:39,748 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c250124a2c5046eb88017a HTTP/1.1" 200 None 2026-03-26 08:49:39,750 - lava_callback - ERROR - Error processing callback for node 69c250124a2c5046eb88017a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-26 08:49:45,450 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:49:45,474 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:60102 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35982 - "GET / HTTP/1.0" 200 OK 2026-03-26 08:49:57,535 - lava_callback - INFO - Received callback for node 69c4ee97f63ce05331e2aa86 INFO: 172.21.0.1:35990 - "POST /node/69c4ee97f63ce05331e2aa86 HTTP/1.0" 202 Accepted 2026-03-26 08:49:57,546 - lava_callback - INFO - Starting async job submit for node 69c4ee97f63ce05331e2aa86 2026-03-26 08:49:57,662 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:49:57,688 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ee97f63ce05331e2aa86 HTTP/1.1" 200 None 2026-03-26 08:49:57,690 - lava_callback - DEBUG - Processing node 69c4ee97f63ce05331e2aa86: ltp-smoke 2026-03-26 08:49:58,431 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:49:58,801 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:49:58,803 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoke-69c4ee97f63ce05331e2aa86/log.txt.gz INFO: 172.21.0.1:35998 - "GET / HTTP/1.0" 200 OK 2026-03-26 08:49:59,150 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:49:59,877 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:49:59,880 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-smoke-69c4ee97f63ce05331e2aa86/lava-logs.yaml 2026-03-26 08:49:59,896 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:50:00,363 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:50:00,365 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoke-69c4ee97f63ce05331e2aa86/lava_callback.json.gz 2026-03-26 08:50:00,565 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:50:00,598 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ee97f63ce05331e2aa86 HTTP/1.1" 200 None 2026-03-26 08:50:00,600 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:50:00,617 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebd0f63ce05331e2a489 HTTP/1.1" 200 None 2026-03-26 08:50:00,619 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:50:00,647 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4ee97f63ce05331e2aa86 HTTP/1.1" 200 None 2026-03-26 08:50:00,905 - lava_callback - INFO - Completed processing callback for node 69c4ee97f63ce05331e2aa86 INFO: 172.21.0.1:52192 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41396 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-26 08:50:15,477 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:50:15,502 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:41398 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47982 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45846 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43170 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33476 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33480 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57828 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33568 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33584 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46204 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54310 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46468 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33356 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33360 - "GET / HTTP/1.0" 200 OK 2026-03-26 08:51:59,681 - lava_callback - INFO - Received callback for node 69c250a94a2c5046eb8801be INFO: 172.21.0.1:33370 - "POST /node/69c250a94a2c5046eb8801be HTTP/1.0" 202 Accepted 2026-03-26 08:51:59,690 - lava_callback - INFO - Starting async job submit for node 69c250a94a2c5046eb8801be 2026-03-26 08:51:59,691 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:51:59,734 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c250a94a2c5046eb8801be HTTP/1.1" 200 None 2026-03-26 08:51:59,735 - lava_callback - DEBUG - Processing node 69c250a94a2c5046eb8801be: baseline-arm-clabbe 2026-03-26 08:51:59,735 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c250a94a2c5046eb8801be 2026-03-26 08:51:59,736 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:51:59,757 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c250a94a2c5046eb8801be HTTP/1.1" 200 None 2026-03-26 08:51:59,760 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:51:59,787 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd026d05e4bbd6ba634c57 HTTP/1.1" 200 None 2026-03-26 08:51:59,790 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:51:59,833 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c250a94a2c5046eb8801be HTTP/1.1" 200 None 2026-03-26 08:51:59,834 - lava_callback - ERROR - Error processing callback for node 69c250a94a2c5046eb8801be: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:34478 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50738 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50748 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42274 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53196 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53206 - "GET / HTTP/1.0" 200 OK 2026-03-26 08:52:39,604 - lava_callback - INFO - Received callback for node 69c250d04a2c5046eb8801d3 INFO: 172.21.0.1:53216 - "POST /node/69c250d04a2c5046eb8801d3 HTTP/1.0" 202 Accepted 2026-03-26 08:52:39,614 - lava_callback - INFO - Starting async job submit for node 69c250d04a2c5046eb8801d3 2026-03-26 08:52:39,615 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:52:39,645 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c250d04a2c5046eb8801d3 HTTP/1.1" 200 None 2026-03-26 08:52:39,647 - lava_callback - DEBUG - Processing node 69c250d04a2c5046eb8801d3: baseline-arm-clabbe 2026-03-26 08:52:39,647 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c250d04a2c5046eb8801d3 2026-03-26 08:52:39,648 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:52:39,672 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c250d04a2c5046eb8801d3 HTTP/1.1" 200 None 2026-03-26 08:52:39,675 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:52:39,700 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd026d05e4bbd6ba634c57 HTTP/1.1" 200 None 2026-03-26 08:52:39,703 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:52:39,736 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c250d04a2c5046eb8801d3 HTTP/1.1" 200 None 2026-03-26 08:52:39,738 - lava_callback - ERROR - Error processing callback for node 69c250d04a2c5046eb8801d3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:40930 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40936 - "GET /favicon.ico HTTP/1.0" 404 Not Found INFO: 172.21.0.1:40938 - "GET / HTTP/1.0" 200 OK 2026-03-26 08:52:47,854 - lava_callback - INFO - Received callback for node 69c4f11ef63ce05331e2abca INFO: 172.21.0.1:40944 - "POST /node/69c4f11ef63ce05331e2abca HTTP/1.0" 202 Accepted 2026-03-26 08:52:47,864 - lava_callback - INFO - Starting async job submit for node 69c4f11ef63ce05331e2abca 2026-03-26 08:52:47,912 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:52:47,939 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f11ef63ce05331e2abca HTTP/1.1" 200 None 2026-03-26 08:52:47,941 - lava_callback - DEBUG - Processing node 69c4f11ef63ce05331e2abca: ltp-smoke 2026-03-26 08:52:48,422 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:52:48,655 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:52:48,657 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoke-69c4f11ef63ce05331e2abca/log.txt.gz 2026-03-26 08:52:48,873 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:52:49,350 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:52:49,353 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-smoke-69c4f11ef63ce05331e2abca/lava-logs.yaml 2026-03-26 08:52:49,362 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:52:49,470 - lava_callback - INFO - Received callback for node 69c4f121f63ce05331e2abd0 INFO: 172.21.0.1:40954 - "POST /node/69c4f121f63ce05331e2abd0 HTTP/1.0" 202 Accepted 2026-03-26 08:52:49,477 - lava_callback - INFO - Starting async job submit for node 69c4f121f63ce05331e2abd0 2026-03-26 08:52:49,513 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:52:49,528 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f121f63ce05331e2abd0 HTTP/1.1" 200 None 2026-03-26 08:52:49,528 - lava_callback - DEBUG - Processing node 69c4f121f63ce05331e2abd0: ltp-syscalls-ipc 2026-03-26 08:52:49,629 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:52:49,631 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoke-69c4f11ef63ce05331e2abca/lava_callback.json.gz 2026-03-26 08:52:49,714 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:52:49,766 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f11ef63ce05331e2abca HTTP/1.1" 200 None 2026-03-26 08:52:49,769 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:52:49,829 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebcbf63ce05331e2a47f HTTP/1.1" 200 None 2026-03-26 08:52:49,832 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:52:49,898 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4f11ef63ce05331e2abca HTTP/1.1" 200 None 2026-03-26 08:52:49,900 - lava_callback - INFO - Completed processing callback for node 69c4f11ef63ce05331e2abca 2026-03-26 08:52:50,296 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:52:50,804 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:52:50,806 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-syscalls-ipc-69c4f121f63ce05331e2abd0/log.txt.gz 2026-03-26 08:52:51,020 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:52:51,522 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:52:51,524 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-syscalls-ipc-69c4f121f63ce05331e2abd0/lava-logs.yaml 2026-03-26 08:52:51,533 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:52:51,674 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:52:51,675 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-syscalls-ipc-69c4f121f63ce05331e2abd0/lava_callback.json.gz 2026-03-26 08:52:51,701 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:52:51,725 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f121f63ce05331e2abd0 HTTP/1.1" 200 None 2026-03-26 08:52:51,727 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:52:51,748 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebcbf63ce05331e2a47f HTTP/1.1" 200 None 2026-03-26 08:52:51,750 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:52:51,787 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4f121f63ce05331e2abd0 HTTP/1.1" 200 None 2026-03-26 08:52:51,797 - lava_callback - INFO - Completed processing callback for node 69c4f121f63ce05331e2abd0 INFO: 172.21.0.1:37314 - "GET / HTTP/1.0" 200 OK 2026-03-26 08:52:57,374 - lava_callback - INFO - Received callback for node 69c4f11df63ce05331e2abc3 INFO: 172.21.0.1:37316 - "POST /node/69c4f11df63ce05331e2abc3 HTTP/1.0" 202 Accepted 2026-03-26 08:52:57,381 - lava_callback - INFO - Starting async job submit for node 69c4f11df63ce05331e2abc3 2026-03-26 08:52:57,423 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:52:57,448 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f11df63ce05331e2abc3 HTTP/1.1" 200 None 2026-03-26 08:52:57,449 - lava_callback - DEBUG - Processing node 69c4f11df63ce05331e2abc3: ltp-sched 2026-03-26 08:52:57,991 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:52:58,100 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:52:58,102 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-sched-69c4f11df63ce05331e2abc3/log.txt.gz 2026-03-26 08:52:58,464 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:52:58,695 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:52:58,697 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-sched-69c4f11df63ce05331e2abc3/lava-logs.yaml 2026-03-26 08:52:58,706 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.21.0.1:37328 - "GET / HTTP/1.0" 200 OK 2026-03-26 08:52:59,150 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:52:59,152 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-sched-69c4f11df63ce05331e2abc3/lava_callback.json.gz 2026-03-26 08:52:59,197 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:52:59,222 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f11df63ce05331e2abc3 HTTP/1.1" 200 None 2026-03-26 08:52:59,224 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:52:59,247 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebcbf63ce05331e2a47f HTTP/1.1" 200 None 2026-03-26 08:52:59,249 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:52:59,289 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4f11df63ce05331e2abc3 HTTP/1.1" 200 None 2026-03-26 08:52:59,291 - lava_callback - INFO - Completed processing callback for node 69c4f11df63ce05331e2abc3 INFO: 172.21.0.1:42226 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51884 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-26 08:53:15,505 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:53:15,522 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:51892 - "GET / HTTP/1.0" 200 OK 2026-03-26 08:53:19,767 - lava_callback - INFO - Received callback for node 69c250f64a2c5046eb8801e6 INFO: 172.21.0.1:38024 - "POST /node/69c250f64a2c5046eb8801e6 HTTP/1.0" 202 Accepted 2026-03-26 08:53:19,772 - lava_callback - INFO - Starting async job submit for node 69c250f64a2c5046eb8801e6 2026-03-26 08:53:19,773 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:53:19,788 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c250f64a2c5046eb8801e6 HTTP/1.1" 200 None 2026-03-26 08:53:19,789 - lava_callback - DEBUG - Processing node 69c250f64a2c5046eb8801e6: baseline-arm-clabbe 2026-03-26 08:53:19,789 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c250f64a2c5046eb8801e6 2026-03-26 08:53:19,790 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:53:19,803 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c250f64a2c5046eb8801e6 HTTP/1.1" 200 None 2026-03-26 08:53:19,804 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:53:19,823 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd026d05e4bbd6ba634c57 HTTP/1.1" 200 None 2026-03-26 08:53:19,825 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:53:19,846 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c250f64a2c5046eb8801e6 HTTP/1.1" 200 None 2026-03-26 08:53:19,851 - lava_callback - ERROR - Error processing callback for node 69c250f64a2c5046eb8801e6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:38038 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39008 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44262 - "GET / HTTP/1.0" 200 OK 2026-03-26 08:53:47,956 - lava_callback - INFO - Received callback for node 69c4f123f63ce05331e2abd3 INFO: 172.21.0.1:44278 - "POST /node/69c4f123f63ce05331e2abd3 HTTP/1.0" 202 Accepted 2026-03-26 08:53:47,963 - lava_callback - INFO - Starting async job submit for node 69c4f123f63ce05331e2abd3 2026-03-26 08:53:48,006 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:53:48,020 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f123f63ce05331e2abd3 HTTP/1.1" 200 None 2026-03-26 08:53:48,021 - lava_callback - DEBUG - Processing node 69c4f123f63ce05331e2abd3: ltp-timers 2026-03-26 08:53:48,375 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:53:48,484 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:53:48,486 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-timers-69c4f123f63ce05331e2abd3/log.txt.gz 2026-03-26 08:53:48,697 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:53:48,920 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:53:48,922 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-timers-69c4f123f63ce05331e2abd3/lava-logs.yaml 2026-03-26 08:53:48,931 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:53:49,099 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:53:49,101 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-timers-69c4f123f63ce05331e2abd3/lava_callback.json.gz 2026-03-26 08:53:49,128 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:53:49,143 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f123f63ce05331e2abd3 HTTP/1.1" 200 None 2026-03-26 08:53:49,145 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:53:49,158 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebcbf63ce05331e2a47f HTTP/1.1" 200 None 2026-03-26 08:53:49,160 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:53:49,193 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4f123f63ce05331e2abd3 HTTP/1.1" 200 None 2026-03-26 08:53:49,195 - lava_callback - INFO - Completed processing callback for node 69c4f123f63ce05331e2abd3 INFO: 172.21.0.1:36714 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36724 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59840 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41208 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-26 08:54:15,524 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:54:15,543 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:41216 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42230 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37822 - "GET / HTTP/1.0" 200 OK 2026-03-26 08:54:39,708 - lava_callback - INFO - Received callback for node 69c251464a2c5046eb880208 INFO: 172.21.0.1:59574 - "POST /node/69c251464a2c5046eb880208 HTTP/1.0" 202 Accepted 2026-03-26 08:54:39,714 - lava_callback - INFO - Starting async job submit for node 69c251464a2c5046eb880208 2026-03-26 08:54:39,715 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:54:39,736 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c251464a2c5046eb880208 HTTP/1.1" 200 None 2026-03-26 08:54:39,737 - lava_callback - DEBUG - Processing node 69c251464a2c5046eb880208: baseline-arm-clabbe 2026-03-26 08:54:39,738 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c251464a2c5046eb880208 2026-03-26 08:54:39,738 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:54:39,753 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c251464a2c5046eb880208 HTTP/1.1" 200 None 2026-03-26 08:54:39,755 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:54:39,769 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ba5f6d537ae4d3c8236e46 HTTP/1.1" 200 None 2026-03-26 08:54:39,771 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:54:39,792 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c251464a2c5046eb880208 HTTP/1.1" 200 None 2026-03-26 08:54:39,798 - lava_callback - ERROR - Error processing callback for node 69c251464a2c5046eb880208: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:59576 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37144 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37152 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39672 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46600 - "GET /security.txt HTTP/1.0" 404 Not Found INFO: 172.21.0.1:46602 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46606 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52594 - "GET / HTTP/1.0" 200 OK 2026-03-26 08:55:36,174 - lava_callback - INFO - Received callback for node 69c251794a2c5046eb8802c6 INFO: 172.21.0.1:48136 - "POST /node/69c251794a2c5046eb8802c6 HTTP/1.0" 202 Accepted 2026-03-26 08:55:36,181 - lava_callback - INFO - Starting async job submit for node 69c251794a2c5046eb8802c6 2026-03-26 08:55:36,182 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:55:36,201 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c251794a2c5046eb8802c6 HTTP/1.1" 200 None 2026-03-26 08:55:36,203 - lava_callback - DEBUG - Processing node 69c251794a2c5046eb8802c6: baseline-arm64-baylibre 2026-03-26 08:55:36,203 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c251794a2c5046eb8802c6 2026-03-26 08:55:36,204 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:55:36,232 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c251794a2c5046eb8802c6 HTTP/1.1" 200 None 2026-03-26 08:55:36,234 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:55:36,255 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248e54a2c5046eb87f89f HTTP/1.1" 200 None 2026-03-26 08:55:36,257 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:55:36,287 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c251794a2c5046eb8802c6 HTTP/1.1" 200 None 2026-03-26 08:55:36,299 - lava_callback - ERROR - Error processing callback for node 69c251794a2c5046eb8802c6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:48142 - "GET / HTTP/1.0" 200 OK 2026-03-26 08:55:36,864 - lava_callback - INFO - Received callback for node 69c2517a4a2c5046eb8802c9 INFO: 172.21.0.1:48154 - "POST /node/69c2517a4a2c5046eb8802c9 HTTP/1.0" 202 Accepted 2026-03-26 08:55:36,870 - lava_callback - INFO - Starting async job submit for node 69c2517a4a2c5046eb8802c9 2026-03-26 08:55:36,871 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:55:36,885 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2517a4a2c5046eb8802c9 HTTP/1.1" 200 None 2026-03-26 08:55:36,886 - lava_callback - DEBUG - Processing node 69c2517a4a2c5046eb8802c9: baseline-arm64-baylibre 2026-03-26 08:55:36,887 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2517a4a2c5046eb8802c9 2026-03-26 08:55:36,887 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:55:36,901 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2517a4a2c5046eb8802c9 HTTP/1.1" 200 None 2026-03-26 08:55:36,903 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:55:36,915 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248e54a2c5046eb87f89f HTTP/1.1" 200 None 2026-03-26 08:55:36,917 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:55:36,937 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2517a4a2c5046eb8802c9 HTTP/1.1" 200 None 2026-03-26 08:55:36,943 - lava_callback - ERROR - Error processing callback for node 69c2517a4a2c5046eb8802c9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-26 08:55:37,541 - lava_callback - INFO - Received callback for node 69c2517d4a2c5046eb8802d3 INFO: 172.21.0.1:48168 - "POST /node/69c2517d4a2c5046eb8802d3 HTTP/1.0" 202 Accepted 2026-03-26 08:55:37,550 - lava_callback - INFO - Starting async job submit for node 69c2517d4a2c5046eb8802d3 2026-03-26 08:55:37,552 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:55:37,573 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2517d4a2c5046eb8802d3 HTTP/1.1" 200 None 2026-03-26 08:55:37,575 - lava_callback - DEBUG - Processing node 69c2517d4a2c5046eb8802d3: baseline-arm64-baylibre 2026-03-26 08:55:37,575 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2517d4a2c5046eb8802d3 2026-03-26 08:55:37,576 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:55:37,598 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2517d4a2c5046eb8802d3 HTTP/1.1" 200 None 2026-03-26 08:55:37,600 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:55:37,626 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248e54a2c5046eb87f89f HTTP/1.1" 200 None 2026-03-26 08:55:37,628 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:55:37,664 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2517d4a2c5046eb8802d3 HTTP/1.1" 200 None 2026-03-26 08:55:37,666 - lava_callback - ERROR - Error processing callback for node 69c2517d4a2c5046eb8802d3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-26 08:55:38,201 - lava_callback - INFO - Received callback for node 69c2517f4a2c5046eb8802da INFO: 172.21.0.1:48184 - "POST /node/69c2517f4a2c5046eb8802da HTTP/1.0" 202 Accepted 2026-03-26 08:55:38,207 - lava_callback - INFO - Starting async job submit for node 69c2517f4a2c5046eb8802da 2026-03-26 08:55:38,208 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:55:38,233 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2517f4a2c5046eb8802da HTTP/1.1" 200 None 2026-03-26 08:55:38,234 - lava_callback - DEBUG - Processing node 69c2517f4a2c5046eb8802da: baseline-arm64-baylibre 2026-03-26 08:55:38,234 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2517f4a2c5046eb8802da 2026-03-26 08:55:38,235 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:55:38,250 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2517f4a2c5046eb8802da HTTP/1.1" 200 None 2026-03-26 08:55:38,251 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:55:38,266 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248e54a2c5046eb87f89f HTTP/1.1" 200 None 2026-03-26 08:55:38,267 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:55:38,291 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2517f4a2c5046eb8802da HTTP/1.1" 200 None 2026-03-26 08:55:38,297 - lava_callback - ERROR - Error processing callback for node 69c2517f4a2c5046eb8802da: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-26 08:55:38,878 - lava_callback - INFO - Received callback for node 69c251804a2c5046eb8802dd INFO: 172.21.0.1:48196 - "POST /node/69c251804a2c5046eb8802dd HTTP/1.0" 202 Accepted 2026-03-26 08:55:38,886 - lava_callback - INFO - Starting async job submit for node 69c251804a2c5046eb8802dd 2026-03-26 08:55:38,887 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:55:38,902 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c251804a2c5046eb8802dd HTTP/1.1" 200 None 2026-03-26 08:55:38,904 - lava_callback - DEBUG - Processing node 69c251804a2c5046eb8802dd: baseline-arm64-baylibre 2026-03-26 08:55:38,904 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c251804a2c5046eb8802dd 2026-03-26 08:55:38,905 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:55:38,919 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c251804a2c5046eb8802dd HTTP/1.1" 200 None 2026-03-26 08:55:38,920 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:55:38,935 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248e54a2c5046eb87f89f HTTP/1.1" 200 None 2026-03-26 08:55:38,937 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:55:38,972 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c251804a2c5046eb8802dd HTTP/1.1" 200 None 2026-03-26 08:55:38,974 - lava_callback - ERROR - Error processing callback for node 69c251804a2c5046eb8802dd: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-26 08:55:41,242 - lava_callback - INFO - Received callback for node 69c4f0d4f63ce05331e2aaba INFO: 172.21.0.1:54384 - "POST /node/69c4f0d4f63ce05331e2aaba HTTP/1.0" 202 Accepted 2026-03-26 08:55:41,250 - lava_callback - INFO - Starting async job submit for node 69c4f0d4f63ce05331e2aaba 2026-03-26 08:55:41,301 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:55:41,322 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f0d4f63ce05331e2aaba HTTP/1.1" 200 None 2026-03-26 08:55:41,323 - lava_callback - DEBUG - Processing node 69c4f0d4f63ce05331e2aaba: baseline-x86 2026-03-26 08:55:42,253 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:55:42,581 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:55:42,583 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69c4f0d4f63ce05331e2aaba/log.txt.gz 2026-03-26 08:55:43,232 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:55:43,503 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:55:43,506 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69c4f0d4f63ce05331e2aaba/lava-logs.yaml 2026-03-26 08:55:43,524 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:55:43,826 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:55:43,829 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69c4f0d4f63ce05331e2aaba/lava_callback.json.gz DEBUG: Setup failed for 69c4f0d4f63ce05331e2aaba. Transitting job node result: incomplete -> fail 2026-03-26 08:55:43,935 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:55:43,950 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f0d4f63ce05331e2aaba HTTP/1.1" 200 None 2026-03-26 08:55:43,952 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:55:43,971 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebecf63ce05331e2a4ae HTTP/1.1" 200 None 2026-03-26 08:55:43,973 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:55:44,011 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4f0d4f63ce05331e2aaba HTTP/1.1" 200 None 2026-03-26 08:55:44,115 - lava_callback - INFO - Completed processing callback for node 69c4f0d4f63ce05331e2aaba 2026-03-26 08:55:45,545 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:55:45,563 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:54396 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46704 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46710 - "GET / HTTP/1.0" 200 OK 2026-03-26 08:55:59,744 - lava_callback - INFO - Received callback for node 69c251924a2c5046eb8802fc INFO: 172.21.0.1:57068 - "POST /node/69c251924a2c5046eb8802fc HTTP/1.0" 202 Accepted 2026-03-26 08:55:59,751 - lava_callback - INFO - Starting async job submit for node 69c251924a2c5046eb8802fc 2026-03-26 08:55:59,752 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:55:59,769 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c251924a2c5046eb8802fc HTTP/1.1" 200 None 2026-03-26 08:55:59,770 - lava_callback - DEBUG - Processing node 69c251924a2c5046eb8802fc: baseline-arm-clabbe 2026-03-26 08:55:59,770 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c251924a2c5046eb8802fc 2026-03-26 08:55:59,771 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:55:59,792 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c251924a2c5046eb8802fc HTTP/1.1" 200 None 2026-03-26 08:55:59,795 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:55:59,815 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ba5f6d537ae4d3c8236e46 HTTP/1.1" 200 None 2026-03-26 08:55:59,817 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:55:59,849 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c251924a2c5046eb8802fc HTTP/1.1" 200 None 2026-03-26 08:55:59,850 - lava_callback - ERROR - Error processing callback for node 69c251924a2c5046eb8802fc: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-26 08:56:00,412 - lava_callback - INFO - Received callback for node 69c251954a2c5046eb880303 INFO: 172.21.0.1:57070 - "POST /node/69c251954a2c5046eb880303 HTTP/1.0" 202 Accepted 2026-03-26 08:56:00,420 - lava_callback - INFO - Starting async job submit for node 69c251954a2c5046eb880303 2026-03-26 08:56:00,421 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:56:00,443 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c251954a2c5046eb880303 HTTP/1.1" 200 None 2026-03-26 08:56:00,445 - lava_callback - DEBUG - Processing node 69c251954a2c5046eb880303: baseline-arm64-clabbe 2026-03-26 08:56:00,445 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c251954a2c5046eb880303 2026-03-26 08:56:00,446 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:56:00,469 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c251954a2c5046eb880303 HTTP/1.1" 200 None 2026-03-26 08:56:00,471 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:56:00,497 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248e54a2c5046eb87f89f HTTP/1.1" 200 None 2026-03-26 08:56:00,499 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:56:00,533 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c251954a2c5046eb880303 HTTP/1.1" 200 None 2026-03-26 08:56:00,534 - lava_callback - ERROR - Error processing callback for node 69c251954a2c5046eb880303: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-26 08:56:01,134 - lava_callback - INFO - Received callback for node 69c251954a2c5046eb880306 INFO: 172.21.0.1:57084 - "POST /node/69c251954a2c5046eb880306 HTTP/1.0" 202 Accepted 2026-03-26 08:56:01,140 - lava_callback - INFO - Starting async job submit for node 69c251954a2c5046eb880306 2026-03-26 08:56:01,141 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:56:01,156 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c251954a2c5046eb880306 HTTP/1.1" 200 None 2026-03-26 08:56:01,157 - lava_callback - DEBUG - Processing node 69c251954a2c5046eb880306: baseline-arm64-clabbe 2026-03-26 08:56:01,158 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c251954a2c5046eb880306 2026-03-26 08:56:01,158 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:56:01,181 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c251954a2c5046eb880306 HTTP/1.1" 200 None 2026-03-26 08:56:01,183 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:56:01,197 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248e54a2c5046eb87f89f HTTP/1.1" 200 None 2026-03-26 08:56:01,199 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:56:01,229 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c251954a2c5046eb880306 HTTP/1.1" 200 None 2026-03-26 08:56:01,231 - lava_callback - ERROR - Error processing callback for node 69c251954a2c5046eb880306: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-26 08:56:01,819 - lava_callback - INFO - Received callback for node 69c251964a2c5046eb880309 INFO: 172.21.0.1:57096 - "POST /node/69c251964a2c5046eb880309 HTTP/1.0" 202 Accepted 2026-03-26 08:56:01,825 - lava_callback - INFO - Starting async job submit for node 69c251964a2c5046eb880309 2026-03-26 08:56:01,826 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:56:01,841 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c251964a2c5046eb880309 HTTP/1.1" 200 None 2026-03-26 08:56:01,842 - lava_callback - DEBUG - Processing node 69c251964a2c5046eb880309: baseline-arm64-clabbe 2026-03-26 08:56:01,843 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c251964a2c5046eb880309 2026-03-26 08:56:01,843 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:56:01,859 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c251964a2c5046eb880309 HTTP/1.1" 200 None 2026-03-26 08:56:01,861 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:56:01,875 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248e54a2c5046eb87f89f HTTP/1.1" 200 None 2026-03-26 08:56:01,877 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:56:01,906 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c251964a2c5046eb880309 HTTP/1.1" 200 None 2026-03-26 08:56:01,907 - lava_callback - ERROR - Error processing callback for node 69c251964a2c5046eb880309: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-26 08:56:02,495 - lava_callback - INFO - Received callback for node 69c251974a2c5046eb880323 INFO: 172.21.0.1:57106 - "POST /node/69c251974a2c5046eb880323 HTTP/1.0" 202 Accepted 2026-03-26 08:56:02,504 - lava_callback - INFO - Starting async job submit for node 69c251974a2c5046eb880323 2026-03-26 08:56:02,506 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:56:02,527 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c251974a2c5046eb880323 HTTP/1.1" 200 None 2026-03-26 08:56:02,529 - lava_callback - DEBUG - Processing node 69c251974a2c5046eb880323: baseline-arm64-clabbe 2026-03-26 08:56:02,529 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c251974a2c5046eb880323 2026-03-26 08:56:02,530 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:56:02,556 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c251974a2c5046eb880323 HTTP/1.1" 200 None 2026-03-26 08:56:02,558 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:56:02,586 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248e54a2c5046eb87f89f HTTP/1.1" 200 None 2026-03-26 08:56:02,588 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:56:02,632 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c251974a2c5046eb880323 HTTP/1.1" 200 None 2026-03-26 08:56:02,633 - lava_callback - ERROR - Error processing callback for node 69c251974a2c5046eb880323: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-26 08:56:03,171 - lava_callback - INFO - Received callback for node 69c251994a2c5046eb88032a INFO: 172.21.0.1:57122 - "POST /node/69c251994a2c5046eb88032a HTTP/1.0" 202 Accepted 2026-03-26 08:56:03,180 - lava_callback - INFO - Starting async job submit for node 69c251994a2c5046eb88032a 2026-03-26 08:56:03,181 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:56:03,197 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c251994a2c5046eb88032a HTTP/1.1" 200 None 2026-03-26 08:56:03,199 - lava_callback - DEBUG - Processing node 69c251994a2c5046eb88032a: baseline-arm64-clabbe 2026-03-26 08:56:03,199 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c251994a2c5046eb88032a 2026-03-26 08:56:03,200 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:56:03,214 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c251994a2c5046eb88032a HTTP/1.1" 200 None 2026-03-26 08:56:03,216 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:56:03,230 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248e54a2c5046eb87f89f HTTP/1.1" 200 None 2026-03-26 08:56:03,232 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:56:03,255 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c251994a2c5046eb88032a HTTP/1.1" 200 None 2026-03-26 08:56:03,257 - lava_callback - ERROR - Error processing callback for node 69c251994a2c5046eb88032a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-26 08:56:03,866 - lava_callback - INFO - Received callback for node 69c2519a4a2c5046eb880331 INFO: 172.21.0.1:57132 - "POST /node/69c2519a4a2c5046eb880331 HTTP/1.0" 202 Accepted 2026-03-26 08:56:03,872 - lava_callback - INFO - Starting async job submit for node 69c2519a4a2c5046eb880331 2026-03-26 08:56:03,873 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:56:03,893 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2519a4a2c5046eb880331 HTTP/1.1" 200 None 2026-03-26 08:56:03,894 - lava_callback - DEBUG - Processing node 69c2519a4a2c5046eb880331: baseline-arm64-clabbe 2026-03-26 08:56:03,895 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2519a4a2c5046eb880331 2026-03-26 08:56:03,895 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:56:03,908 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2519a4a2c5046eb880331 HTTP/1.1" 200 None 2026-03-26 08:56:03,910 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:56:03,926 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248e54a2c5046eb87f89f HTTP/1.1" 200 None 2026-03-26 08:56:03,928 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:56:03,961 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2519a4a2c5046eb880331 HTTP/1.1" 200 None 2026-03-26 08:56:03,968 - lava_callback - ERROR - Error processing callback for node 69c2519a4a2c5046eb880331: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:57140 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41406 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41408 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51968 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44990 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56372 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56386 - "GET /favicon.ico HTTP/1.0" 404 Not Found INFO: 172.21.0.1:56392 - "GET / HTTP/1.0" 200 OK 2026-03-26 08:56:46,837 - lava_callback - INFO - Received callback for node 69c4f4bbf63ce05331e2ae5c INFO: 172.21.0.1:56404 - "POST /node/69c4f4bbf63ce05331e2ae5c HTTP/1.0" 202 Accepted 2026-03-26 08:56:46,846 - lava_callback - INFO - Starting async job submit for node 69c4f4bbf63ce05331e2ae5c 2026-03-26 08:56:46,851 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:56:46,901 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f4bbf63ce05331e2ae5c HTTP/1.1" 200 None 2026-03-26 08:56:46,902 - lava_callback - DEBUG - Processing node 69c4f4bbf63ce05331e2ae5c: baseline-arm64 2026-03-26 08:56:46,915 - lava_callback - WARNING - Failed to upload log 2026-03-26 08:56:46,930 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:56:47,304 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:56:47,306 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c4f4bbf63ce05331e2ae5c/lava-logs.yaml 2026-03-26 08:56:47,308 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:56:47,592 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:56:47,594 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c4f4bbf63ce05331e2ae5c/lava_callback.json.gz 2026-03-26 08:56:47,598 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:56:47,644 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f4bbf63ce05331e2ae5c HTTP/1.1" 200 None 2026-03-26 08:56:47,647 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:56:47,677 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebd8f63ce05331e2a492 HTTP/1.1" 200 None 2026-03-26 08:56:47,680 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:56:47,716 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4f4bbf63ce05331e2ae5c HTTP/1.1" 200 None 2026-03-26 08:56:47,721 - lava_callback - INFO - Completed processing callback for node 69c4f4bbf63ce05331e2ae5c 2026-03-26 08:56:48,059 - lava_callback - INFO - Received callback for node 69c4f124f63ce05331e2abda INFO: 172.21.0.1:56416 - "POST /node/69c4f124f63ce05331e2abda HTTP/1.0" 202 Accepted 2026-03-26 08:56:48,071 - lava_callback - INFO - Starting async job submit for node 69c4f124f63ce05331e2abda 2026-03-26 08:56:48,152 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:56:48,177 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f124f63ce05331e2abda HTTP/1.1" 200 None 2026-03-26 08:56:48,178 - lava_callback - DEBUG - Processing node 69c4f124f63ce05331e2abda: ltp-watchqueue 2026-03-26 08:56:48,821 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:56:49,211 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:56:49,213 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-watchqueue-69c4f124f63ce05331e2abda/log.txt.gz 2026-03-26 08:56:49,483 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:56:50,061 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:56:50,064 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-watchqueue-69c4f124f63ce05331e2abda/lava-logs.yaml 2026-03-26 08:56:50,074 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:56:50,306 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:56:50,309 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-watchqueue-69c4f124f63ce05331e2abda/lava_callback.json.gz 2026-03-26 08:56:50,350 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:56:50,382 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f124f63ce05331e2abda HTTP/1.1" 200 None 2026-03-26 08:56:50,384 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:56:50,410 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebcbf63ce05331e2a47f HTTP/1.1" 200 None 2026-03-26 08:56:50,412 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:56:50,464 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4f124f63ce05331e2abda HTTP/1.1" 200 None 2026-03-26 08:56:50,465 - lava_callback - INFO - Completed processing callback for node 69c4f124f63ce05331e2abda INFO: 172.21.0.1:46202 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46204 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35182 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40160 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-26 08:57:15,565 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:57:15,582 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:40172 - "GET / HTTP/1.0" 200 OK 2026-03-26 08:57:19,825 - lava_callback - INFO - Received callback for node 69c251df4a2c5046eb8803d3 INFO: 172.21.0.1:44366 - "POST /node/69c251df4a2c5046eb8803d3 HTTP/1.0" 202 Accepted 2026-03-26 08:57:19,834 - lava_callback - INFO - Starting async job submit for node 69c251df4a2c5046eb8803d3 2026-03-26 08:57:19,835 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:57:19,856 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c251df4a2c5046eb8803d3 HTTP/1.1" 200 None 2026-03-26 08:57:19,857 - lava_callback - DEBUG - Processing node 69c251df4a2c5046eb8803d3: baseline-arm-clabbe 2026-03-26 08:57:19,858 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c251df4a2c5046eb8803d3 2026-03-26 08:57:19,858 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:57:19,876 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c251df4a2c5046eb8803d3 HTTP/1.1" 200 None 2026-03-26 08:57:19,878 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:57:19,904 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ba5f6d537ae4d3c8236e46 HTTP/1.1" 200 None 2026-03-26 08:57:19,906 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:57:19,938 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c251df4a2c5046eb8803d3 HTTP/1.1" 200 None 2026-03-26 08:57:19,941 - lava_callback - ERROR - Error processing callback for node 69c251df4a2c5046eb8803d3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:44380 - "GET / HTTP/1.0" 200 OK 2026-03-26 08:57:31,571 - lava_callback - INFO - Received callback for node 69c4f4dbf63ce05331e2aecb INFO: 172.21.0.1:60014 - "POST /node/69c4f4dbf63ce05331e2aecb HTTP/1.0" 202 Accepted 2026-03-26 08:57:31,583 - lava_callback - INFO - Starting async job submit for node 69c4f4dbf63ce05331e2aecb 2026-03-26 08:57:31,585 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:57:31,639 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f4dbf63ce05331e2aecb HTTP/1.1" 200 None 2026-03-26 08:57:31,641 - lava_callback - DEBUG - Processing node 69c4f4dbf63ce05331e2aecb: baseline-nfs-arm64 2026-03-26 08:57:31,643 - lava_callback - WARNING - Failed to upload log 2026-03-26 08:57:31,645 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:57:32,091 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:57:32,093 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c4f4dbf63ce05331e2aecb/lava-logs.yaml 2026-03-26 08:57:32,095 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:57:32,324 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:57:32,326 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c4f4dbf63ce05331e2aecb/lava_callback.json.gz 2026-03-26 08:57:32,328 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:57:32,344 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f4dbf63ce05331e2aecb HTTP/1.1" 200 None 2026-03-26 08:57:32,346 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:57:32,377 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebd8f63ce05331e2a492 HTTP/1.1" 200 None 2026-03-26 08:57:32,386 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:57:32,438 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4f4dbf63ce05331e2aecb HTTP/1.1" 200 None 2026-03-26 08:57:32,449 - lava_callback - INFO - Completed processing callback for node 69c4f4dbf63ce05331e2aecb 2026-03-26 08:57:33,089 - lava_callback - INFO - Received callback for node 69c4f4bff63ce05331e2ae68 INFO: 172.21.0.1:60028 - "POST /node/69c4f4bff63ce05331e2ae68 HTTP/1.0" 202 Accepted 2026-03-26 08:57:33,100 - lava_callback - INFO - Starting async job submit for node 69c4f4bff63ce05331e2ae68 2026-03-26 08:57:33,132 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:57:33,147 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f4bff63ce05331e2ae68 HTTP/1.1" 200 None 2026-03-26 08:57:33,148 - lava_callback - DEBUG - Processing node 69c4f4bff63ce05331e2ae68: baseline-arm64 2026-03-26 08:57:33,312 - lava_callback - WARNING - Failed to upload log 2026-03-26 08:57:33,437 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:57:33,446 - lava_callback - INFO - Received callback for node 69c4f4bdf63ce05331e2ae62 INFO: 172.21.0.1:60034 - "POST /node/69c4f4bdf63ce05331e2ae62 HTTP/1.0" 202 Accepted 2026-03-26 08:57:33,457 - lava_callback - INFO - Starting async job submit for node 69c4f4bdf63ce05331e2ae62 2026-03-26 08:57:33,535 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:57:33,564 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f4bdf63ce05331e2ae62 HTTP/1.1" 200 None 2026-03-26 08:57:33,566 - lava_callback - DEBUG - Processing node 69c4f4bdf63ce05331e2ae62: baseline-arm64 2026-03-26 08:57:33,938 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:57:33,986 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c4f4bff63ce05331e2ae68/lava-logs.yaml 2026-03-26 08:57:34,049 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:57:34,050 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:57:34,133 - lava_callback - INFO - Received callback for node 69c4f4c9f63ce05331e2ae7d 2026-03-26 08:57:34,143 - lava_callback - INFO - Starting async job submit for node 69c4f4c9f63ce05331e2ae7d INFO: 172.21.0.1:60046 - "POST /node/69c4f4c9f63ce05331e2ae7d HTTP/1.0" 202 Accepted 2026-03-26 08:57:34,171 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:57:34,202 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f4c9f63ce05331e2ae7d HTTP/1.1" 200 None 2026-03-26 08:57:34,203 - lava_callback - DEBUG - Processing node 69c4f4c9f63ce05331e2ae7d: baseline-arm64 2026-03-26 08:57:34,468 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:57:34,468 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:57:34,470 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c4f4bff63ce05331e2ae68/lava_callback.json.gz 2026-03-26 08:57:34,536 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:57:34,556 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:57:34,558 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c4f4bdf63ce05331e2ae62/log.txt.gz 2026-03-26 08:57:34,610 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f4bff63ce05331e2ae68 HTTP/1.1" 200 None 2026-03-26 08:57:34,613 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:57:34,759 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebd8f63ce05331e2a492 HTTP/1.1" 200 None 2026-03-26 08:57:34,778 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:57:34,795 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:57:34,839 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c4f4c9f63ce05331e2ae7d/log.txt.gz 2026-03-26 08:57:34,910 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:57:34,956 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4f4bff63ce05331e2ae68 HTTP/1.1" 200 None 2026-03-26 08:57:35,174 - lava_callback - INFO - Completed processing callback for node 69c4f4bff63ce05331e2ae68 2026-03-26 08:57:35,203 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:57:35,611 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:57:35,613 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c4f4bdf63ce05331e2ae62/lava-logs.yaml 2026-03-26 08:57:35,620 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:57:35,634 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:57:35,637 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c4f4c9f63ce05331e2ae7d/lava-logs.yaml 2026-03-26 08:57:35,641 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:57:35,841 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:57:35,843 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c4f4bdf63ce05331e2ae62/lava_callback.json.gz 2026-03-26 08:57:35,912 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:57:35,943 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f4bdf63ce05331e2ae62 HTTP/1.1" 200 None 2026-03-26 08:57:35,945 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:57:35,975 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebd8f63ce05331e2a492 HTTP/1.1" 200 None 2026-03-26 08:57:35,977 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:57:36,023 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4f4bdf63ce05331e2ae62 HTTP/1.1" 200 None 2026-03-26 08:57:36,025 - lava_callback - INFO - Completed processing callback for node 69c4f4bdf63ce05331e2ae62 2026-03-26 08:57:36,169 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:57:36,171 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c4f4c9f63ce05331e2ae7d/lava_callback.json.gz 2026-03-26 08:57:36,224 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:57:36,253 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f4c9f63ce05331e2ae7d HTTP/1.1" 200 None 2026-03-26 08:57:36,255 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:57:36,281 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebd8f63ce05331e2a492 HTTP/1.1" 200 None 2026-03-26 08:57:36,284 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:57:36,334 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4f4c9f63ce05331e2ae7d HTTP/1.1" 200 None 2026-03-26 08:57:36,336 - lava_callback - INFO - Completed processing callback for node 69c4f4c9f63ce05331e2ae7d INFO: 172.21.0.1:60050 - "GET / HTTP/1.0" 200 OK 2026-03-26 08:57:42,204 - lava_callback - INFO - Received callback for node 69c4f0dcf63ce05331e2aad8 INFO: 172.21.0.1:50086 - "POST /node/69c4f0dcf63ce05331e2aad8 HTTP/1.0" 202 Accepted 2026-03-26 08:57:42,223 - lava_callback - INFO - Starting async job submit for node 69c4f0dcf63ce05331e2aad8 2026-03-26 08:57:42,373 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:57:42,398 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f0dcf63ce05331e2aad8 HTTP/1.1" 200 None 2026-03-26 08:57:42,400 - lava_callback - DEBUG - Processing node 69c4f0dcf63ce05331e2aad8: baseline-arm 2026-03-26 08:57:43,355 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:57:43,769 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:57:43,771 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69c4f0dcf63ce05331e2aad8/log.txt.gz 2026-03-26 08:57:44,190 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:57:44,672 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:57:44,675 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c4f0dcf63ce05331e2aad8/lava-logs.yaml 2026-03-26 08:57:44,698 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:57:45,113 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:57:45,115 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c4f0dcf63ce05331e2aad8/lava_callback.json.gz DEBUG: Setup failed for 69c4f0dcf63ce05331e2aad8. Transitting job node result: incomplete -> fail 2026-03-26 08:57:45,283 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:57:45,310 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f0dcf63ce05331e2aad8 HTTP/1.1" 200 None 2026-03-26 08:57:45,311 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:57:45,325 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebcbf63ce05331e2a47f HTTP/1.1" 200 None 2026-03-26 08:57:45,327 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:57:45,351 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4f0dcf63ce05331e2aad8 HTTP/1.1" 200 None 2026-03-26 08:57:45,498 - lava_callback - INFO - Completed processing callback for node 69c4f0dcf63ce05331e2aad8 2026-03-26 08:57:45,585 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:57:45,672 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 INFO: 172.21.0.1:50088 - "GET / HTTP/1.0" 200 OK 2026-03-26 08:57:49,492 - lava_callback - INFO - Received callback for node 69c4f4c5f63ce05331e2ae74 INFO: 172.21.0.1:50094 - "POST /node/69c4f4c5f63ce05331e2ae74 HTTP/1.0" 202 Accepted 2026-03-26 08:57:49,503 - lava_callback - INFO - Starting async job submit for node 69c4f4c5f63ce05331e2ae74 2026-03-26 08:57:49,620 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:57:49,644 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f4c5f63ce05331e2ae74 HTTP/1.1" 200 None 2026-03-26 08:57:49,645 - lava_callback - DEBUG - Processing node 69c4f4c5f63ce05331e2ae74: baseline-arm64 2026-03-26 08:57:50,201 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:57:50,533 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:57:50,535 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c4f4c5f63ce05331e2ae74/log.txt.gz 2026-03-26 08:57:50,895 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:57:51,602 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:57:51,604 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c4f4c5f63ce05331e2ae74/lava-logs.yaml 2026-03-26 08:57:51,616 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:57:51,878 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:57:51,880 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c4f4c5f63ce05331e2ae74/lava_callback.json.gz 2026-03-26 08:57:51,918 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:57:51,966 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f4c5f63ce05331e2ae74 HTTP/1.1" 200 None 2026-03-26 08:57:51,968 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:57:51,994 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebd8f63ce05331e2a492 HTTP/1.1" 200 None 2026-03-26 08:57:51,996 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:57:52,036 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4f4c5f63ce05331e2ae74 HTTP/1.1" 200 None 2026-03-26 08:57:52,037 - lava_callback - INFO - Completed processing callback for node 69c4f4c5f63ce05331e2ae74 INFO: 172.21.0.1:37768 - "GET / HTTP/1.0" 200 OK 2026-03-26 08:57:57,880 - lava_callback - INFO - Received callback for node 69c4f4bcf63ce05331e2ae5f INFO: 172.21.0.1:37778 - "POST /node/69c4f4bcf63ce05331e2ae5f HTTP/1.0" 202 Accepted 2026-03-26 08:57:57,891 - lava_callback - INFO - Starting async job submit for node 69c4f4bcf63ce05331e2ae5f 2026-03-26 08:57:57,942 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:57:57,976 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f4bcf63ce05331e2ae5f HTTP/1.1" 200 None 2026-03-26 08:57:57,977 - lava_callback - DEBUG - Processing node 69c4f4bcf63ce05331e2ae5f: baseline-arm64 2026-03-26 08:57:58,476 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:57:58,653 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:57:58,655 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c4f4bcf63ce05331e2ae5f/log.txt.gz 2026-03-26 08:57:58,892 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.21.0.1:37782 - "GET / HTTP/1.0" 200 OK 2026-03-26 08:57:59,204 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:57:59,206 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c4f4bcf63ce05331e2ae5f/lava-logs.yaml 2026-03-26 08:57:59,213 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:57:59,354 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:57:59,356 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c4f4bcf63ce05331e2ae5f/lava_callback.json.gz 2026-03-26 08:57:59,390 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:57:59,422 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f4bcf63ce05331e2ae5f HTTP/1.1" 200 None 2026-03-26 08:57:59,425 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:57:59,447 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebd8f63ce05331e2a492 HTTP/1.1" 200 None 2026-03-26 08:57:59,449 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:57:59,488 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4f4bcf63ce05331e2ae5f HTTP/1.1" 200 None 2026-03-26 08:57:59,489 - lava_callback - INFO - Completed processing callback for node 69c4f4bcf63ce05331e2ae5f 2026-03-26 08:57:59,865 - lava_callback - INFO - Received callback for node 69c252054a2c5046eb880426 INFO: 172.21.0.1:55996 - "POST /node/69c252054a2c5046eb880426 HTTP/1.0" 202 Accepted 2026-03-26 08:57:59,871 - lava_callback - INFO - Starting async job submit for node 69c252054a2c5046eb880426 2026-03-26 08:57:59,872 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:57:59,886 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c252054a2c5046eb880426 HTTP/1.1" 200 None 2026-03-26 08:57:59,887 - lava_callback - DEBUG - Processing node 69c252054a2c5046eb880426: baseline-arm-clabbe 2026-03-26 08:57:59,887 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c252054a2c5046eb880426 2026-03-26 08:57:59,887 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:57:59,901 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c252054a2c5046eb880426 HTTP/1.1" 200 None 2026-03-26 08:57:59,903 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:57:59,917 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ba5f6d537ae4d3c8236e46 HTTP/1.1" 200 None 2026-03-26 08:57:59,918 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:57:59,939 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c252054a2c5046eb880426 HTTP/1.1" 200 None 2026-03-26 08:57:59,944 - lava_callback - ERROR - Error processing callback for node 69c252054a2c5046eb880426: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-26 08:58:03,195 - lava_callback - INFO - Received callback for node 69c4f4c6f63ce05331e2ae77 INFO: 172.21.0.1:55998 - "POST /node/69c4f4c6f63ce05331e2ae77 HTTP/1.0" 202 Accepted 2026-03-26 08:58:03,202 - lava_callback - INFO - Starting async job submit for node 69c4f4c6f63ce05331e2ae77 2026-03-26 08:58:03,225 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:58:03,243 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f4c6f63ce05331e2ae77 HTTP/1.1" 200 None 2026-03-26 08:58:03,244 - lava_callback - DEBUG - Processing node 69c4f4c6f63ce05331e2ae77: baseline-arm64 2026-03-26 08:58:03,595 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:58:03,704 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:58:03,706 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c4f4c6f63ce05331e2ae77/log.txt.gz 2026-03-26 08:58:03,894 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:58:04,085 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:58:04,087 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c4f4c6f63ce05331e2ae77/lava-logs.yaml 2026-03-26 08:58:04,094 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:58:04,219 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:58:04,221 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c4f4c6f63ce05331e2ae77/lava_callback.json.gz 2026-03-26 08:58:04,264 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:58:04,316 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f4c6f63ce05331e2ae77 HTTP/1.1" 200 None 2026-03-26 08:58:04,317 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:58:04,330 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebd8f63ce05331e2a492 HTTP/1.1" 200 None 2026-03-26 08:58:04,332 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:58:04,361 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4f4c6f63ce05331e2ae77 HTTP/1.1" 200 None 2026-03-26 08:58:04,404 - lava_callback - INFO - Completed processing callback for node 69c4f4c6f63ce05331e2ae77 INFO: 172.21.0.1:56010 - "GET / HTTP/1.0" 200 OK 2026-03-26 08:58:12,357 - lava_callback - INFO - Received callback for node 69c4f4c8f63ce05331e2ae7a INFO: 172.21.0.1:39130 - "POST /node/69c4f4c8f63ce05331e2ae7a HTTP/1.0" 202 Accepted 2026-03-26 08:58:12,364 - lava_callback - INFO - Starting async job submit for node 69c4f4c8f63ce05331e2ae7a 2026-03-26 08:58:12,386 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:58:12,413 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f4c8f63ce05331e2ae7a HTTP/1.1" 200 None 2026-03-26 08:58:12,414 - lava_callback - DEBUG - Processing node 69c4f4c8f63ce05331e2ae7a: baseline-arm64 2026-03-26 08:58:12,783 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:58:12,893 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:58:12,894 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c4f4c8f63ce05331e2ae7a/log.txt.gz 2026-03-26 08:58:13,115 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:58:13,316 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:58:13,318 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c4f4c8f63ce05331e2ae7a/lava-logs.yaml 2026-03-26 08:58:13,326 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:58:13,463 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:58:13,465 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c4f4c8f63ce05331e2ae7a/lava_callback.json.gz 2026-03-26 08:58:13,509 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:58:13,524 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f4c8f63ce05331e2ae7a HTTP/1.1" 200 None 2026-03-26 08:58:13,526 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:58:13,541 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebd8f63ce05331e2a492 HTTP/1.1" 200 None 2026-03-26 08:58:13,542 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:58:13,569 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4f4c8f63ce05331e2ae7a HTTP/1.1" 200 None 2026-03-26 08:58:13,654 - lava_callback - INFO - Completed processing callback for node 69c4f4c8f63ce05331e2ae7a INFO: 172.21.0.1:39138 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-26 08:58:15,674 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:58:15,691 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 INFO: 172.21.0.1:39142 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39148 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39158 - "GET /favicon.ico HTTP/1.0" 404 Not Found 2026-03-26 08:58:19,329 - lava_callback - INFO - Received callback for node 69c4f4d8f63ce05331e2aec5 INFO: 172.21.0.1:39170 - "POST /node/69c4f4d8f63ce05331e2aec5 HTTP/1.0" 202 Accepted 2026-03-26 08:58:19,335 - lava_callback - INFO - Starting async job submit for node 69c4f4d8f63ce05331e2aec5 2026-03-26 08:58:19,358 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:58:19,375 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f4d8f63ce05331e2aec5 HTTP/1.1" 200 None 2026-03-26 08:58:19,376 - lava_callback - DEBUG - Processing node 69c4f4d8f63ce05331e2aec5: baseline-nfs-arm64 2026-03-26 08:58:19,774 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:58:19,925 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:58:19,927 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c4f4d8f63ce05331e2aec5/log.txt.gz 2026-03-26 08:58:20,165 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:58:20,371 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:58:20,373 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c4f4d8f63ce05331e2aec5/lava-logs.yaml 2026-03-26 08:58:20,382 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:58:20,609 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:58:20,611 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c4f4d8f63ce05331e2aec5/lava_callback.json.gz 2026-03-26 08:58:20,634 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:58:20,651 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f4d8f63ce05331e2aec5 HTTP/1.1" 200 None 2026-03-26 08:58:20,652 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:58:20,668 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebd8f63ce05331e2a492 HTTP/1.1" 200 None 2026-03-26 08:58:20,669 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:58:20,693 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4f4d8f63ce05331e2aec5 HTTP/1.1" 200 None 2026-03-26 08:58:20,695 - lava_callback - INFO - Completed processing callback for node 69c4f4d8f63ce05331e2aec5 2026-03-26 08:58:20,789 - lava_callback - INFO - Received callback for node 69c4f4c3f63ce05331e2ae71 INFO: 172.21.0.1:58830 - "POST /node/69c4f4c3f63ce05331e2ae71 HTTP/1.0" 202 Accepted 2026-03-26 08:58:20,798 - lava_callback - INFO - Starting async job submit for node 69c4f4c3f63ce05331e2ae71 2026-03-26 08:58:20,839 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:58:20,857 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f4c3f63ce05331e2ae71 HTTP/1.1" 200 None 2026-03-26 08:58:20,859 - lava_callback - DEBUG - Processing node 69c4f4c3f63ce05331e2ae71: baseline-arm64 2026-03-26 08:58:21,200 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:58:21,367 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:58:21,369 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c4f4c3f63ce05331e2ae71/log.txt.gz 2026-03-26 08:58:21,544 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:58:21,604 - lava_callback - INFO - Received callback for node 69c4f518f63ce05331e2af96 INFO: 172.21.0.1:58836 - "POST /node/69c4f518f63ce05331e2af96 HTTP/1.0" 202 Accepted 2026-03-26 08:58:21,612 - lava_callback - INFO - Starting async job submit for node 69c4f518f63ce05331e2af96 2026-03-26 08:58:21,616 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:58:21,644 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f518f63ce05331e2af96 HTTP/1.1" 200 None 2026-03-26 08:58:21,645 - lava_callback - DEBUG - Processing node 69c4f518f63ce05331e2af96: baseline-arm 2026-03-26 08:58:21,651 - lava_callback - WARNING - Failed to upload log 2026-03-26 08:58:21,655 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:58:21,724 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:58:21,727 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c4f4c3f63ce05331e2ae71/lava-logs.yaml 2026-03-26 08:58:21,733 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:58:21,780 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:58:21,782 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c4f518f63ce05331e2af96/lava-logs.yaml 2026-03-26 08:58:21,783 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:58:21,882 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:58:21,884 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c4f4c3f63ce05331e2ae71/lava_callback.json.gz 2026-03-26 08:58:21,915 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:58:21,929 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:58:21,932 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c4f518f63ce05331e2af96/lava_callback.json.gz 2026-03-26 08:58:21,936 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f4c3f63ce05331e2ae71 HTTP/1.1" 200 None 2026-03-26 08:58:21,936 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:58:21,938 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:58:21,953 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebd8f63ce05331e2a492 HTTP/1.1" 200 None 2026-03-26 08:58:21,955 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:58:21,958 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f518f63ce05331e2af96 HTTP/1.1" 200 None 2026-03-26 08:58:21,960 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:58:21,979 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebcbf63ce05331e2a47f HTTP/1.1" 200 None 2026-03-26 08:58:21,981 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:58:22,004 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4f4c3f63ce05331e2ae71 HTTP/1.1" 200 None 2026-03-26 08:58:22,006 - lava_callback - INFO - Completed processing callback for node 69c4f4c3f63ce05331e2ae71 2026-03-26 08:58:22,047 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4f518f63ce05331e2af96 HTTP/1.1" 200 None 2026-03-26 08:58:22,053 - lava_callback - INFO - Completed processing callback for node 69c4f518f63ce05331e2af96 INFO: 172.21.0.1:58840 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33216 - "GET / HTTP/1.0" 200 OK 2026-03-26 08:58:42,411 - lava_callback - INFO - Received callback for node 69c4f4cdf63ce05331e2ae86 INFO: 172.21.0.1:47794 - "POST /node/69c4f4cdf63ce05331e2ae86 HTTP/1.0" 202 Accepted 2026-03-26 08:58:42,418 - lava_callback - INFO - Starting async job submit for node 69c4f4cdf63ce05331e2ae86 2026-03-26 08:58:42,441 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:58:42,461 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f4cdf63ce05331e2ae86 HTTP/1.1" 200 None 2026-03-26 08:58:42,462 - lava_callback - DEBUG - Processing node 69c4f4cdf63ce05331e2ae86: baseline-arm64 2026-03-26 08:58:42,757 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:58:42,895 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:58:42,897 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c4f4cdf63ce05331e2ae86/log.txt.gz 2026-03-26 08:58:43,071 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:58:43,280 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:58:43,282 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c4f4cdf63ce05331e2ae86/lava-logs.yaml 2026-03-26 08:58:43,289 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:58:43,479 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:58:43,480 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c4f4cdf63ce05331e2ae86/lava_callback.json.gz 2026-03-26 08:58:43,496 - lava_callback - INFO - Received callback for node 69c4f4c1f63ce05331e2ae6b INFO: 172.21.0.1:47806 - "POST /node/69c4f4c1f63ce05331e2ae6b HTTP/1.0" 202 Accepted 2026-03-26 08:58:43,511 - lava_callback - INFO - Starting async job submit for node 69c4f4c1f63ce05331e2ae6b 2026-03-26 08:58:43,511 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:58:43,543 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:58:43,560 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f4cdf63ce05331e2ae86 HTTP/1.1" 200 None 2026-03-26 08:58:43,562 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f4c1f63ce05331e2ae6b HTTP/1.1" 200 None 2026-03-26 08:58:43,562 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:58:43,563 - lava_callback - DEBUG - Processing node 69c4f4c1f63ce05331e2ae6b: baseline-arm64 2026-03-26 08:58:43,638 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebd8f63ce05331e2a492 HTTP/1.1" 200 None 2026-03-26 08:58:43,660 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:58:43,772 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4f4cdf63ce05331e2ae86 HTTP/1.1" 200 None 2026-03-26 08:58:43,781 - lava_callback - INFO - Completed processing callback for node 69c4f4cdf63ce05331e2ae86 2026-03-26 08:58:43,810 - lava_callback - WARNING - Failed to upload log 2026-03-26 08:58:43,964 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:58:44,183 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:58:44,185 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c4f4c1f63ce05331e2ae6b/lava-logs.yaml 2026-03-26 08:58:44,190 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:58:44,300 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:58:44,301 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c4f4c1f63ce05331e2ae6b/lava_callback.json.gz 2026-03-26 08:58:44,333 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:58:44,349 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f4c1f63ce05331e2ae6b HTTP/1.1" 200 None 2026-03-26 08:58:44,350 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:58:44,367 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebd8f63ce05331e2a492 HTTP/1.1" 200 None 2026-03-26 08:58:44,369 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:58:44,393 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4f4c1f63ce05331e2ae6b HTTP/1.1" 200 None 2026-03-26 08:58:44,483 - lava_callback - INFO - Completed processing callback for node 69c4f4c1f63ce05331e2ae6b 2026-03-26 08:58:44,830 - lava_callback - INFO - Received callback for node 69c4f510f63ce05331e2af8b INFO: 172.21.0.1:47814 - "POST /node/69c4f510f63ce05331e2af8b HTTP/1.0" 202 Accepted 2026-03-26 08:58:44,837 - lava_callback - INFO - Starting async job submit for node 69c4f510f63ce05331e2af8b 2026-03-26 08:58:44,862 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:58:44,881 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f510f63ce05331e2af8b HTTP/1.1" 200 None 2026-03-26 08:58:44,882 - lava_callback - DEBUG - Processing node 69c4f510f63ce05331e2af8b: baseline-arm64-foundriesio 2026-03-26 08:58:45,132 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:58:45,251 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:58:45,253 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69c4f510f63ce05331e2af8b/log.txt.gz 2026-03-26 08:58:45,378 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:58:45,551 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:58:45,553 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69c4f510f63ce05331e2af8b/lava-logs.yaml 2026-03-26 08:58:45,559 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:58:45,689 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:58:45,691 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69c4f510f63ce05331e2af8b/lava_callback.json.gz 2026-03-26 08:58:45,708 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:58:45,715 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:58:45,731 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f510f63ce05331e2af8b HTTP/1.1" 200 None 2026-03-26 08:58:45,733 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:58:45,733 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 2026-03-26 08:58:45,747 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebd8f63ce05331e2a492 HTTP/1.1" 200 None 2026-03-26 08:58:45,748 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:58:45,775 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4f510f63ce05331e2af8b HTTP/1.1" 200 None 2026-03-26 08:58:45,777 - lava_callback - INFO - Completed processing callback for node 69c4f510f63ce05331e2af8b INFO: 172.21.0.1:47824 - "GET / HTTP/1.0" 200 OK 2026-03-26 08:58:47,081 - lava_callback - INFO - Received callback for node 69c4f511f63ce05331e2af8e INFO: 172.21.0.1:47836 - "POST /node/69c4f511f63ce05331e2af8e HTTP/1.0" 202 Accepted 2026-03-26 08:58:47,092 - lava_callback - INFO - Starting async job submit for node 69c4f511f63ce05331e2af8e 2026-03-26 08:58:47,133 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:58:47,148 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f511f63ce05331e2af8e HTTP/1.1" 200 None 2026-03-26 08:58:47,149 - lava_callback - DEBUG - Processing node 69c4f511f63ce05331e2af8e: baseline-arm64-foundriesio 2026-03-26 08:58:47,661 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:58:47,806 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:58:47,808 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69c4f511f63ce05331e2af8e/log.txt.gz 2026-03-26 08:58:48,080 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:58:48,273 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:58:48,275 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69c4f511f63ce05331e2af8e/lava-logs.yaml 2026-03-26 08:58:48,286 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:58:48,487 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:58:48,489 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69c4f511f63ce05331e2af8e/lava_callback.json.gz 2026-03-26 08:58:48,531 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:58:48,547 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f511f63ce05331e2af8e HTTP/1.1" 200 None 2026-03-26 08:58:48,549 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:58:48,604 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebd8f63ce05331e2a492 HTTP/1.1" 200 None 2026-03-26 08:58:48,607 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:58:48,634 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4f511f63ce05331e2af8e HTTP/1.1" 200 None 2026-03-26 08:58:48,636 - lava_callback - INFO - Completed processing callback for node 69c4f511f63ce05331e2af8e 2026-03-26 08:58:51,480 - lava_callback - INFO - Received callback for node 69c4f4bef63ce05331e2ae65 INFO: 172.21.0.1:41486 - "POST /node/69c4f4bef63ce05331e2ae65 HTTP/1.0" 202 Accepted 2026-03-26 08:58:51,486 - lava_callback - INFO - Starting async job submit for node 69c4f4bef63ce05331e2ae65 2026-03-26 08:58:51,507 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:58:51,527 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f4bef63ce05331e2ae65 HTTP/1.1" 200 None 2026-03-26 08:58:51,528 - lava_callback - DEBUG - Processing node 69c4f4bef63ce05331e2ae65: baseline-arm64 2026-03-26 08:58:51,767 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:58:51,915 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:58:51,916 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c4f4bef63ce05331e2ae65/log.txt.gz 2026-03-26 08:58:52,060 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:58:52,224 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:58:52,226 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c4f4bef63ce05331e2ae65/lava-logs.yaml 2026-03-26 08:58:52,232 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:58:52,391 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:58:52,393 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c4f4bef63ce05331e2ae65/lava_callback.json.gz 2026-03-26 08:58:52,415 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:58:52,430 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f4bef63ce05331e2ae65 HTTP/1.1" 200 None 2026-03-26 08:58:52,431 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:58:52,476 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebd8f63ce05331e2a492 HTTP/1.1" 200 None 2026-03-26 08:58:52,478 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:58:52,505 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4f4bef63ce05331e2ae65 HTTP/1.1" 200 None 2026-03-26 08:58:52,506 - lava_callback - INFO - Completed processing callback for node 69c4f4bef63ce05331e2ae65 INFO: 172.21.0.1:41498 - "GET / HTTP/1.0" 200 OK 2026-03-26 08:58:57,498 - lava_callback - INFO - Received callback for node 69c4f4fbf63ce05331e2af24 INFO: 172.21.0.1:41502 - "POST /node/69c4f4fbf63ce05331e2af24 HTTP/1.0" 202 Accepted 2026-03-26 08:58:57,504 - lava_callback - INFO - Starting async job submit for node 69c4f4fbf63ce05331e2af24 2026-03-26 08:58:57,528 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:58:57,543 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f4fbf63ce05331e2af24 HTTP/1.1" 200 None 2026-03-26 08:58:57,544 - lava_callback - DEBUG - Processing node 69c4f4fbf63ce05331e2af24: baseline-arm64-broonie 2026-03-26 08:58:57,777 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:58:57,906 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:58:57,907 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c4f4fbf63ce05331e2af24/log.txt.gz 2026-03-26 08:58:58,048 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:58:58,224 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:58:58,226 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c4f4fbf63ce05331e2af24/lava-logs.yaml 2026-03-26 08:58:58,232 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:58:58,387 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:58:58,388 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c4f4fbf63ce05331e2af24/lava_callback.json.gz 2026-03-26 08:58:58,411 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:58:58,425 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f4fbf63ce05331e2af24 HTTP/1.1" 200 None 2026-03-26 08:58:58,427 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:58:58,442 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebd8f63ce05331e2a492 HTTP/1.1" 200 None 2026-03-26 08:58:58,444 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:58:58,468 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4f4fbf63ce05331e2af24 HTTP/1.1" 200 None 2026-03-26 08:58:58,470 - lava_callback - INFO - Completed processing callback for node 69c4f4fbf63ce05331e2af24 INFO: 172.21.0.1:41508 - "GET / HTTP/1.0" 200 OK 2026-03-26 08:58:59,289 - lava_callback - INFO - Received callback for node 69c4f50cf63ce05331e2af64 INFO: 172.21.0.1:41510 - "POST /node/69c4f50cf63ce05331e2af64 HTTP/1.0" 202 Accepted 2026-03-26 08:58:59,296 - lava_callback - INFO - Starting async job submit for node 69c4f50cf63ce05331e2af64 2026-03-26 08:58:59,315 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:58:59,329 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f50cf63ce05331e2af64 HTTP/1.1" 200 None 2026-03-26 08:58:59,330 - lava_callback - DEBUG - Processing node 69c4f50cf63ce05331e2af64: baseline-arm64-pengutronix 2026-03-26 08:58:59,535 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:58:59,658 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:58:59,660 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69c4f50cf63ce05331e2af64/log.txt.gz 2026-03-26 08:58:59,764 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:58:59,932 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:58:59,933 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69c4f50cf63ce05331e2af64/lava-logs.yaml 2026-03-26 08:58:59,939 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:59:00,068 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:59:00,070 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69c4f50cf63ce05331e2af64/lava_callback.json.gz 2026-03-26 08:59:00,091 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:59:00,111 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f50cf63ce05331e2af64 HTTP/1.1" 200 None 2026-03-26 08:59:00,113 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:59:00,126 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebd8f63ce05331e2a492 HTTP/1.1" 200 None 2026-03-26 08:59:00,128 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:59:00,152 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4f50cf63ce05331e2af64 HTTP/1.1" 200 None 2026-03-26 08:59:00,153 - lava_callback - INFO - Completed processing callback for node 69c4f50cf63ce05331e2af64 2026-03-26 08:59:01,154 - lava_callback - INFO - Received callback for node 69c4f4e0f63ce05331e2aed4 INFO: 172.21.0.1:59788 - "POST /node/69c4f4e0f63ce05331e2aed4 HTTP/1.0" 202 Accepted 2026-03-26 08:59:01,161 - lava_callback - INFO - Starting async job submit for node 69c4f4e0f63ce05331e2aed4 2026-03-26 08:59:01,184 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:59:01,198 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f4e0f63ce05331e2aed4 HTTP/1.1" 200 None 2026-03-26 08:59:01,199 - lava_callback - DEBUG - Processing node 69c4f4e0f63ce05331e2aed4: baseline-nfs-arm64 2026-03-26 08:59:01,546 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:59:01,685 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:59:01,686 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c4f4e0f63ce05331e2aed4/log.txt.gz 2026-03-26 08:59:01,721 - lava_callback - INFO - Received callback for node 69c4f4daf63ce05331e2aec8 INFO: 172.21.0.1:59792 - "POST /node/69c4f4daf63ce05331e2aec8 HTTP/1.0" 202 Accepted 2026-03-26 08:59:01,743 - lava_callback - INFO - Starting async job submit for node 69c4f4daf63ce05331e2aec8 2026-03-26 08:59:01,791 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:59:01,853 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f4daf63ce05331e2aec8 HTTP/1.1" 200 None 2026-03-26 08:59:01,863 - lava_callback - DEBUG - Processing node 69c4f4daf63ce05331e2aec8: baseline-nfs-arm64 2026-03-26 08:59:02,064 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:59:02,365 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:59:02,397 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c4f4e0f63ce05331e2aed4/lava-logs.yaml 2026-03-26 08:59:02,416 - lava_callback - INFO - Received callback for node 69c4f4cff63ce05331e2ae8f INFO: 172.21.0.1:59802 - "POST /node/69c4f4cff63ce05331e2ae8f HTTP/1.0" 202 Accepted 2026-03-26 08:59:02,443 - lava_callback - INFO - Starting async job submit for node 69c4f4cff63ce05331e2ae8f 2026-03-26 08:59:02,477 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:59:02,477 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:59:02,481 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:59:02,494 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f4cff63ce05331e2ae8f HTTP/1.1" 200 None 2026-03-26 08:59:02,495 - lava_callback - DEBUG - Processing node 69c4f4cff63ce05331e2ae8f: baseline-arm64 2026-03-26 08:59:02,619 - lava_callback - INFO - Received callback for node 69c4f4e2f63ce05331e2aedb 2026-03-26 08:59:02,649 - lava_callback - INFO - Starting async job submit for node 69c4f4e2f63ce05331e2aedb 2026-03-26 08:59:02,665 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 INFO: 172.21.0.1:59804 - "POST /node/69c4f4e2f63ce05331e2aedb HTTP/1.0" 202 Accepted 2026-03-26 08:59:02,677 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:59:02,728 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:59:02,757 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c4f4e0f63ce05331e2aed4/lava_callback.json.gz 2026-03-26 08:59:02,762 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c4f4daf63ce05331e2aec8/log.txt.gz 2026-03-26 08:59:02,921 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:59:02,937 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f4e2f63ce05331e2aedb HTTP/1.1" 200 None 2026-03-26 08:59:02,950 - lava_callback - DEBUG - Processing node 69c4f4e2f63ce05331e2aedb: baseline-nfs-arm64 2026-03-26 08:59:03,049 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f4e0f63ce05331e2aed4 HTTP/1.1" 200 None 2026-03-26 08:59:03,231 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:59:03,365 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:59:03,438 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:59:03,496 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebd8f63ce05331e2a492 HTTP/1.1" 200 None 2026-03-26 08:59:03,519 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:59:03,585 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:59:03,600 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c4f4cff63ce05331e2ae8f/log.txt.gz 2026-03-26 08:59:03,624 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4f4e0f63ce05331e2aed4 HTTP/1.1" 200 None 2026-03-26 08:59:03,647 - lava_callback - INFO - Completed processing callback for node 69c4f4e0f63ce05331e2aed4 2026-03-26 08:59:03,643 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:59:03,836 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:59:03,860 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c4f4daf63ce05331e2aec8/lava-logs.yaml 2026-03-26 08:59:03,883 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:59:03,896 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:59:03,943 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c4f4e2f63ce05331e2aedb/log.txt.gz 2026-03-26 08:59:03,972 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:59:04,169 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:59:04,201 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c4f4daf63ce05331e2aec8/lava_callback.json.gz 2026-03-26 08:59:04,257 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:59:04,261 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:59:04,278 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:59:04,278 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c4f4cff63ce05331e2ae8f/lava-logs.yaml 2026-03-26 08:59:04,287 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:59:04,291 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f4daf63ce05331e2aec8 HTTP/1.1" 200 None 2026-03-26 08:59:04,293 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:59:04,324 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebd8f63ce05331e2a492 HTTP/1.1" 200 None 2026-03-26 08:59:04,327 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:59:04,353 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4f4daf63ce05331e2aec8 HTTP/1.1" 200 None 2026-03-26 08:59:04,354 - lava_callback - INFO - Completed processing callback for node 69c4f4daf63ce05331e2aec8 2026-03-26 08:59:04,503 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:59:04,505 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c4f4cff63ce05331e2ae8f/lava_callback.json.gz 2026-03-26 08:59:04,565 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:59:04,565 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:59:04,567 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c4f4e2f63ce05331e2aedb/lava-logs.yaml 2026-03-26 08:59:04,578 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:59:04,584 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f4cff63ce05331e2ae8f HTTP/1.1" 200 None 2026-03-26 08:59:04,586 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:59:04,600 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebd8f63ce05331e2a492 HTTP/1.1" 200 None 2026-03-26 08:59:04,602 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:59:04,628 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4f4cff63ce05331e2ae8f HTTP/1.1" 200 None 2026-03-26 08:59:04,672 - lava_callback - INFO - Completed processing callback for node 69c4f4cff63ce05331e2ae8f 2026-03-26 08:59:04,806 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:59:04,808 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c4f4e2f63ce05331e2aedb/lava_callback.json.gz 2026-03-26 08:59:04,845 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:59:04,862 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f4e2f63ce05331e2aedb HTTP/1.1" 200 None 2026-03-26 08:59:04,864 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:59:04,879 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebd8f63ce05331e2a492 HTTP/1.1" 200 None 2026-03-26 08:59:04,881 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:59:04,912 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4f4e2f63ce05331e2aedb HTTP/1.1" 200 None 2026-03-26 08:59:04,914 - lava_callback - INFO - Completed processing callback for node 69c4f4e2f63ce05331e2aedb INFO: 172.21.0.1:59808 - "GET / HTTP/1.0" 200 OK 2026-03-26 08:59:07,990 - lava_callback - INFO - Received callback for node 69c4f4fff63ce05331e2af31 INFO: 172.21.0.1:59814 - "POST /node/69c4f4fff63ce05331e2af31 HTTP/1.0" 202 Accepted 2026-03-26 08:59:07,996 - lava_callback - INFO - Starting async job submit for node 69c4f4fff63ce05331e2af31 2026-03-26 08:59:08,018 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:59:08,033 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f4fff63ce05331e2af31 HTTP/1.1" 200 None 2026-03-26 08:59:08,034 - lava_callback - DEBUG - Processing node 69c4f4fff63ce05331e2af31: baseline-arm64-broonie 2026-03-26 08:59:08,273 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:59:08,401 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:59:08,403 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c4f4fff63ce05331e2af31/log.txt.gz 2026-03-26 08:59:08,570 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:59:08,749 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:59:08,751 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c4f4fff63ce05331e2af31/lava-logs.yaml 2026-03-26 08:59:08,758 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:59:08,889 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:59:08,890 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c4f4fff63ce05331e2af31/lava_callback.json.gz 2026-03-26 08:59:08,913 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:59:08,932 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f4fff63ce05331e2af31 HTTP/1.1" 200 None 2026-03-26 08:59:08,934 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:59:08,949 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebd8f63ce05331e2a492 HTTP/1.1" 200 None 2026-03-26 08:59:08,950 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:59:08,974 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4f4fff63ce05331e2af31 HTTP/1.1" 200 None 2026-03-26 08:59:08,975 - lava_callback - INFO - Completed processing callback for node 69c4f4fff63ce05331e2af31 INFO: 172.21.0.1:48184 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-26 08:59:15,735 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:59:15,753 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 2026-03-26 08:59:16,542 - lava_callback - INFO - Received callback for node 69c252574a2c5046eb88046e INFO: 172.21.0.1:48196 - "POST /node/69c252574a2c5046eb88046e HTTP/1.0" 202 Accepted 2026-03-26 08:59:16,548 - lava_callback - INFO - Starting async job submit for node 69c252574a2c5046eb88046e 2026-03-26 08:59:16,549 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:59:16,569 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c252574a2c5046eb88046e HTTP/1.1" 200 None 2026-03-26 08:59:16,570 - lava_callback - DEBUG - Processing node 69c252574a2c5046eb88046e: baseline-arm64-baylibre 2026-03-26 08:59:16,570 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c252574a2c5046eb88046e 2026-03-26 08:59:16,571 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:59:16,584 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c252574a2c5046eb88046e HTTP/1.1" 200 None 2026-03-26 08:59:16,585 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:59:16,598 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5827544ab5cd4a7d9dd HTTP/1.1" 200 None 2026-03-26 08:59:16,600 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:59:16,620 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c252574a2c5046eb88046e HTTP/1.1" 200 None 2026-03-26 08:59:16,626 - lava_callback - ERROR - Error processing callback for node 69c252574a2c5046eb88046e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:48206 - "GET / HTTP/1.0" 200 OK 2026-03-26 08:59:17,417 - lava_callback - INFO - Received callback for node 69c4f4f0f63ce05331e2aef9 INFO: 172.21.0.1:48220 - "POST /node/69c4f4f0f63ce05331e2aef9 HTTP/1.0" 202 Accepted 2026-03-26 08:59:17,423 - lava_callback - INFO - Starting async job submit for node 69c4f4f0f63ce05331e2aef9 2026-03-26 08:59:17,446 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:59:17,460 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f4f0f63ce05331e2aef9 HTTP/1.1" 200 None 2026-03-26 08:59:17,461 - lava_callback - DEBUG - Processing node 69c4f4f0f63ce05331e2aef9: baseline-arm64-broonie 2026-03-26 08:59:17,676 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:59:17,787 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:59:17,789 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c4f4f0f63ce05331e2aef9/log.txt.gz 2026-03-26 08:59:17,918 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:59:18,098 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:59:18,100 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c4f4f0f63ce05331e2aef9/lava-logs.yaml 2026-03-26 08:59:18,106 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:59:18,238 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:59:18,240 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c4f4f0f63ce05331e2aef9/lava_callback.json.gz 2026-03-26 08:59:18,263 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:59:18,291 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f4f0f63ce05331e2aef9 HTTP/1.1" 200 None 2026-03-26 08:59:18,293 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:59:18,310 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebd8f63ce05331e2a492 HTTP/1.1" 200 None 2026-03-26 08:59:18,312 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:59:18,337 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4f4f0f63ce05331e2aef9 HTTP/1.1" 200 None 2026-03-26 08:59:18,342 - lava_callback - INFO - Completed processing callback for node 69c4f4f0f63ce05331e2aef9 2026-03-26 08:59:19,442 - lava_callback - INFO - Received callback for node 69c4f4d3f63ce05331e2ae99 INFO: 172.21.0.1:48236 - "POST /node/69c4f4d3f63ce05331e2ae99 HTTP/1.0" 202 Accepted 2026-03-26 08:59:19,449 - lava_callback - INFO - Starting async job submit for node 69c4f4d3f63ce05331e2ae99 2026-03-26 08:59:19,471 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:59:19,486 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f4d3f63ce05331e2ae99 HTTP/1.1" 200 None 2026-03-26 08:59:19,487 - lava_callback - DEBUG - Processing node 69c4f4d3f63ce05331e2ae99: baseline-nfs-arm64 2026-03-26 08:59:19,559 - lava_callback - INFO - Received callback for node 69c4f50df63ce05331e2af67 INFO: 172.21.0.1:48246 - "POST /node/69c4f50df63ce05331e2af67 HTTP/1.0" 202 Accepted 2026-03-26 08:59:19,585 - lava_callback - INFO - Starting async job submit for node 69c4f50df63ce05331e2af67 2026-03-26 08:59:19,626 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:59:19,689 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f50df63ce05331e2af67 HTTP/1.1" 200 None 2026-03-26 08:59:19,701 - lava_callback - DEBUG - Processing node 69c4f50df63ce05331e2af67: baseline-arm64-pengutronix 2026-03-26 08:59:20,065 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:59:20,068 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:59:20,258 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:59:20,260 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69c4f50df63ce05331e2af67/log.txt.gz 2026-03-26 08:59:20,275 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:59:20,324 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c4f4d3f63ce05331e2ae99/log.txt.gz 2026-03-26 08:59:20,482 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:59:20,591 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:59:20,734 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:59:20,736 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69c4f50df63ce05331e2af67/lava-logs.yaml 2026-03-26 08:59:20,742 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:59:20,800 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:59:20,802 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c4f4d3f63ce05331e2ae99/lava-logs.yaml 2026-03-26 08:59:20,811 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:59:20,883 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:59:20,885 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69c4f50df63ce05331e2af67/lava_callback.json.gz 2026-03-26 08:59:20,907 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:59:20,924 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f50df63ce05331e2af67 HTTP/1.1" 200 None 2026-03-26 08:59:20,925 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:59:20,940 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebd8f63ce05331e2a492 HTTP/1.1" 200 None 2026-03-26 08:59:20,942 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:59:20,949 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:59:20,951 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c4f4d3f63ce05331e2ae99/lava_callback.json.gz 2026-03-26 08:59:20,981 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:59:20,997 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4f50df63ce05331e2af67 HTTP/1.1" 200 None 2026-03-26 08:59:21,004 - lava_callback - INFO - Completed processing callback for node 69c4f50df63ce05331e2af67 2026-03-26 08:59:21,028 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f4d3f63ce05331e2ae99 HTTP/1.1" 200 None 2026-03-26 08:59:21,031 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:59:21,078 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebd8f63ce05331e2a492 HTTP/1.1" 200 None 2026-03-26 08:59:21,087 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:59:21,175 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4f4d3f63ce05331e2ae99 HTTP/1.1" 200 None 2026-03-26 08:59:21,177 - lava_callback - INFO - Completed processing callback for node 69c4f4d3f63ce05331e2ae99 2026-03-26 08:59:21,816 - lava_callback - INFO - Received callback for node 69c4f507f63ce05331e2af4b INFO: 172.21.0.1:40132 - "POST /node/69c4f507f63ce05331e2af4b HTTP/1.0" 202 Accepted 2026-03-26 08:59:21,823 - lava_callback - INFO - Starting async job submit for node 69c4f507f63ce05331e2af4b 2026-03-26 08:59:21,844 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:59:21,864 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f507f63ce05331e2af4b HTTP/1.1" 200 None 2026-03-26 08:59:21,865 - lava_callback - DEBUG - Processing node 69c4f507f63ce05331e2af4b: baseline-arm64-clabbe 2026-03-26 08:59:22,090 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:59:22,216 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:59:22,218 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-clabbe-69c4f507f63ce05331e2af4b/log.txt.gz 2026-03-26 08:59:22,362 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:59:22,583 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:59:22,585 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-clabbe-69c4f507f63ce05331e2af4b/lava-logs.yaml 2026-03-26 08:59:22,591 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:59:22,741 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:59:22,743 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-clabbe-69c4f507f63ce05331e2af4b/lava_callback.json.gz 2026-03-26 08:59:22,765 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:59:22,783 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f507f63ce05331e2af4b HTTP/1.1" 200 None 2026-03-26 08:59:22,784 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:59:22,807 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebd8f63ce05331e2a492 HTTP/1.1" 200 None 2026-03-26 08:59:22,809 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:59:22,840 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4f507f63ce05331e2af4b HTTP/1.1" 200 None 2026-03-26 08:59:22,842 - lava_callback - INFO - Completed processing callback for node 69c4f507f63ce05331e2af4b INFO: 172.21.0.1:40142 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40148 - "GET / HTTP/1.0" 200 OK 2026-03-26 08:59:27,961 - lava_callback - INFO - Received callback for node 69c4f4f6f63ce05331e2af1b INFO: 172.21.0.1:40156 - "POST /node/69c4f4f6f63ce05331e2af1b HTTP/1.0" 202 Accepted 2026-03-26 08:59:27,967 - lava_callback - INFO - Starting async job submit for node 69c4f4f6f63ce05331e2af1b 2026-03-26 08:59:27,989 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:59:28,004 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f4f6f63ce05331e2af1b HTTP/1.1" 200 None 2026-03-26 08:59:28,005 - lava_callback - DEBUG - Processing node 69c4f4f6f63ce05331e2af1b: baseline-arm64-broonie 2026-03-26 08:59:28,287 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:59:28,408 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:59:28,410 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c4f4f6f63ce05331e2af1b/log.txt.gz 2026-03-26 08:59:28,572 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:59:28,769 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:59:28,771 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c4f4f6f63ce05331e2af1b/lava-logs.yaml 2026-03-26 08:59:28,778 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:59:28,923 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:59:28,924 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c4f4f6f63ce05331e2af1b/lava_callback.json.gz 2026-03-26 08:59:28,947 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:59:28,962 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f4f6f63ce05331e2af1b HTTP/1.1" 200 None 2026-03-26 08:59:28,964 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:59:28,980 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebd8f63ce05331e2a492 HTTP/1.1" 200 None 2026-03-26 08:59:28,982 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:59:29,006 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4f4f6f63ce05331e2af1b HTTP/1.1" 200 None 2026-03-26 08:59:29,013 - lava_callback - INFO - Completed processing callback for node 69c4f4f6f63ce05331e2af1b 2026-03-26 08:59:29,607 - lava_callback - INFO - Received callback for node 69c4f4f8f63ce05331e2af1e INFO: 172.21.0.1:40172 - "POST /node/69c4f4f8f63ce05331e2af1e HTTP/1.0" 202 Accepted 2026-03-26 08:59:29,614 - lava_callback - INFO - Starting async job submit for node 69c4f4f8f63ce05331e2af1e 2026-03-26 08:59:29,637 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:59:29,651 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f4f8f63ce05331e2af1e HTTP/1.1" 200 None 2026-03-26 08:59:29,652 - lava_callback - DEBUG - Processing node 69c4f4f8f63ce05331e2af1e: baseline-arm64-broonie 2026-03-26 08:59:29,905 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:59:30,016 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:59:30,017 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c4f4f8f63ce05331e2af1e/log.txt.gz 2026-03-26 08:59:30,171 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:59:30,342 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:59:30,344 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c4f4f8f63ce05331e2af1e/lava-logs.yaml 2026-03-26 08:59:30,351 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:59:30,485 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:59:30,486 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c4f4f8f63ce05331e2af1e/lava_callback.json.gz 2026-03-26 08:59:30,510 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:59:30,526 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f4f8f63ce05331e2af1e HTTP/1.1" 200 None 2026-03-26 08:59:30,528 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:59:30,541 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebd8f63ce05331e2a492 HTTP/1.1" 200 None 2026-03-26 08:59:30,543 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:59:30,573 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4f4f8f63ce05331e2af1e HTTP/1.1" 200 None 2026-03-26 08:59:30,574 - lava_callback - INFO - Completed processing callback for node 69c4f4f8f63ce05331e2af1e 2026-03-26 08:59:30,984 - lava_callback - INFO - Received callback for node 69c4f4fdf63ce05331e2af2c INFO: 172.21.0.1:55188 - "POST /node/69c4f4fdf63ce05331e2af2c HTTP/1.0" 202 Accepted 2026-03-26 08:59:30,993 - lava_callback - INFO - Starting async job submit for node 69c4f4fdf63ce05331e2af2c 2026-03-26 08:59:31,017 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:59:31,031 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f4fdf63ce05331e2af2c HTTP/1.1" 200 None 2026-03-26 08:59:31,032 - lava_callback - DEBUG - Processing node 69c4f4fdf63ce05331e2af2c: baseline-arm64-broonie 2026-03-26 08:59:31,329 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:59:31,440 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:59:31,442 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c4f4fdf63ce05331e2af2c/log.txt.gz 2026-03-26 08:59:31,607 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:59:31,783 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:59:31,785 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c4f4fdf63ce05331e2af2c/lava-logs.yaml 2026-03-26 08:59:31,792 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:59:31,936 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:59:31,937 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c4f4fdf63ce05331e2af2c/lava_callback.json.gz 2026-03-26 08:59:31,960 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:59:31,975 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f4fdf63ce05331e2af2c HTTP/1.1" 200 None 2026-03-26 08:59:31,977 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:59:32,006 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebd8f63ce05331e2a492 HTTP/1.1" 200 None 2026-03-26 08:59:32,008 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:59:32,034 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4f4fdf63ce05331e2af2c HTTP/1.1" 200 None 2026-03-26 08:59:32,040 - lava_callback - INFO - Completed processing callback for node 69c4f4fdf63ce05331e2af2c INFO: 172.21.0.1:55196 - "GET / HTTP/1.0" 200 OK 2026-03-26 08:59:37,459 - lava_callback - INFO - Received callback for node 69c4f501f63ce05331e2af3c INFO: 172.21.0.1:55206 - "POST /node/69c4f501f63ce05331e2af3c HTTP/1.0" 202 Accepted 2026-03-26 08:59:37,470 - lava_callback - INFO - Starting async job submit for node 69c4f501f63ce05331e2af3c 2026-03-26 08:59:37,506 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:59:37,537 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f501f63ce05331e2af3c HTTP/1.1" 200 None 2026-03-26 08:59:37,539 - lava_callback - DEBUG - Processing node 69c4f501f63ce05331e2af3c: baseline-arm64-broonie 2026-03-26 08:59:37,833 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:59:37,970 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:59:37,971 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c4f501f63ce05331e2af3c/log.txt.gz 2026-03-26 08:59:38,131 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:59:38,301 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:59:38,303 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c4f501f63ce05331e2af3c/lava-logs.yaml 2026-03-26 08:59:38,309 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 08:59:38,464 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 08:59:38,465 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c4f501f63ce05331e2af3c/lava_callback.json.gz 2026-03-26 08:59:38,489 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:59:38,509 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f501f63ce05331e2af3c HTTP/1.1" 200 None 2026-03-26 08:59:38,511 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:59:38,530 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebd8f63ce05331e2a492 HTTP/1.1" 200 None 2026-03-26 08:59:38,532 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:59:38,560 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4f501f63ce05331e2af3c HTTP/1.1" 200 None 2026-03-26 08:59:38,566 - lava_callback - INFO - Completed processing callback for node 69c4f501f63ce05331e2af3c 2026-03-26 08:59:45,755 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:59:45,788 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 INFO: 172.21.0.1:34988 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57416 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57432 - "GET / HTTP/1.0" 200 OK 2026-03-26 09:00:05,190 - lava_callback - INFO - Received callback for node 69c4f4d5f63ce05331e2aebe INFO: 172.21.0.1:46782 - "POST /node/69c4f4d5f63ce05331e2aebe HTTP/1.0" 202 Accepted 2026-03-26 09:00:05,197 - lava_callback - INFO - Starting async job submit for node 69c4f4d5f63ce05331e2aebe 2026-03-26 09:00:05,219 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:00:05,235 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f4d5f63ce05331e2aebe HTTP/1.1" 200 None 2026-03-26 09:00:05,235 - lava_callback - DEBUG - Processing node 69c4f4d5f63ce05331e2aebe: baseline-nfs-arm64 2026-03-26 09:00:05,562 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 09:00:05,698 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 09:00:05,700 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c4f4d5f63ce05331e2aebe/log.txt.gz 2026-03-26 09:00:05,913 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 09:00:06,299 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 09:00:06,301 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c4f4d5f63ce05331e2aebe/lava-logs.yaml 2026-03-26 09:00:06,309 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.21.0.1:46784 - "GET / HTTP/1.0" 200 OK 2026-03-26 09:00:06,612 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 09:00:06,614 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c4f4d5f63ce05331e2aebe/lava_callback.json.gz 2026-03-26 09:00:06,637 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:00:06,651 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f4d5f63ce05331e2aebe HTTP/1.1" 200 None 2026-03-26 09:00:06,653 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:00:06,666 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebd8f63ce05331e2a492 HTTP/1.1" 200 None 2026-03-26 09:00:06,667 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:00:06,698 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4f4d5f63ce05331e2aebe HTTP/1.1" 200 None 2026-03-26 09:00:06,709 - lava_callback - INFO - Completed processing callback for node 69c4f4d5f63ce05331e2aebe 2026-03-26 09:00:07,456 - lava_callback - INFO - Received callback for node 69c4f0fcf63ce05331e2ab6a INFO: 172.21.0.1:46800 - "POST /node/69c4f0fcf63ce05331e2ab6a HTTP/1.0" 202 Accepted 2026-03-26 09:00:07,505 - lava_callback - INFO - Starting async job submit for node 69c4f0fcf63ce05331e2ab6a 2026-03-26 09:00:07,530 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:00:07,544 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f0fcf63ce05331e2ab6a HTTP/1.1" 200 None 2026-03-26 09:00:07,545 - lava_callback - DEBUG - Processing node 69c4f0fcf63ce05331e2ab6a: baseline-arm-pengutronix 2026-03-26 09:00:07,652 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 09:00:07,769 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 09:00:07,771 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c4f0fcf63ce05331e2ab6a/log.txt.gz 2026-03-26 09:00:07,830 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 09:00:07,987 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 09:00:07,989 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c4f0fcf63ce05331e2ab6a/lava-logs.yaml 2026-03-26 09:00:07,992 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 09:00:08,104 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 09:00:08,106 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c4f0fcf63ce05331e2ab6a/lava_callback.json.gz DEBUG: Setup failed for 69c4f0fcf63ce05331e2ab6a. Transitting job node result: incomplete -> fail 2026-03-26 09:00:08,155 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:00:08,171 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f0fcf63ce05331e2ab6a HTTP/1.1" 200 None 2026-03-26 09:00:08,172 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:00:08,188 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebcbf63ce05331e2a47f HTTP/1.1" 200 None 2026-03-26 09:00:08,190 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:00:08,213 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4f0fcf63ce05331e2ab6a HTTP/1.1" 200 None 2026-03-26 09:00:08,281 - lava_callback - INFO - Completed processing callback for node 69c4f0fcf63ce05331e2ab6a INFO: 172.21.0.1:46804 - "GET /favicon.ico HTTP/1.0" 404 Not Found 2026-03-26 09:00:08,676 - lava_callback - INFO - Received callback for node 69c4f50ff63ce05331e2af6a INFO: 172.21.0.1:46818 - "POST /node/69c4f50ff63ce05331e2af6a HTTP/1.0" 202 Accepted 2026-03-26 09:00:08,726 - lava_callback - INFO - Starting async job submit for node 69c4f50ff63ce05331e2af6a 2026-03-26 09:00:08,746 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:00:08,760 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f50ff63ce05331e2af6a HTTP/1.1" 200 None 2026-03-26 09:00:08,761 - lava_callback - DEBUG - Processing node 69c4f50ff63ce05331e2af6a: baseline-arm64-pengutronix 2026-03-26 09:00:08,879 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 09:00:08,994 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 09:00:08,996 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69c4f50ff63ce05331e2af6a/log.txt.gz 2026-03-26 09:00:09,064 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 09:00:09,239 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 09:00:09,241 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69c4f50ff63ce05331e2af6a/lava-logs.yaml 2026-03-26 09:00:09,244 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 09:00:09,357 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 09:00:09,358 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69c4f50ff63ce05331e2af6a/lava_callback.json.gz 2026-03-26 09:00:09,380 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:00:09,397 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f50ff63ce05331e2af6a HTTP/1.1" 200 None 2026-03-26 09:00:09,398 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:00:09,417 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebd8f63ce05331e2a492 HTTP/1.1" 200 None 2026-03-26 09:00:09,419 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:00:09,444 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4f50ff63ce05331e2af6a HTTP/1.1" 200 None 2026-03-26 09:00:09,453 - lava_callback - INFO - Completed processing callback for node 69c4f50ff63ce05331e2af6a INFO: 172.21.0.1:57732 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-26 09:00:15,790 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:00:15,807 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:57734 - "GET / HTTP/1.0" 200 OK 2026-03-26 09:00:17,360 - lava_callback - INFO - Received callback for node 69c4f4f2f63ce05331e2af15 INFO: 172.21.0.1:57750 - "POST /node/69c4f4f2f63ce05331e2af15 HTTP/1.0" 202 Accepted 2026-03-26 09:00:17,366 - lava_callback - INFO - Starting async job submit for node 69c4f4f2f63ce05331e2af15 2026-03-26 09:00:17,388 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:00:17,402 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f4f2f63ce05331e2af15 HTTP/1.1" 200 None 2026-03-26 09:00:17,403 - lava_callback - DEBUG - Processing node 69c4f4f2f63ce05331e2af15: baseline-arm64-broonie 2026-03-26 09:00:17,646 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 09:00:17,773 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 09:00:17,775 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c4f4f2f63ce05331e2af15/log.txt.gz 2026-03-26 09:00:17,912 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 09:00:18,102 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 09:00:18,104 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c4f4f2f63ce05331e2af15/lava-logs.yaml 2026-03-26 09:00:18,110 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 09:00:18,243 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 09:00:18,245 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c4f4f2f63ce05331e2af15/lava_callback.json.gz 2026-03-26 09:00:18,268 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:00:18,283 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f4f2f63ce05331e2af15 HTTP/1.1" 200 None 2026-03-26 09:00:18,284 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:00:18,302 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebd8f63ce05331e2a492 HTTP/1.1" 200 None 2026-03-26 09:00:18,304 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:00:18,333 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4f4f2f63ce05331e2af15 HTTP/1.1" 200 None 2026-03-26 09:00:18,335 - lava_callback - INFO - Completed processing callback for node 69c4f4f2f63ce05331e2af15 2026-03-26 09:00:18,516 - lava_callback - INFO - Received callback for node 69c4f4c2f63ce05331e2ae6e INFO: 172.21.0.1:57766 - "POST /node/69c4f4c2f63ce05331e2ae6e HTTP/1.0" 202 Accepted 2026-03-26 09:00:18,523 - lava_callback - INFO - Starting async job submit for node 69c4f4c2f63ce05331e2ae6e 2026-03-26 09:00:18,547 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:00:18,561 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f4c2f63ce05331e2ae6e HTTP/1.1" 200 None 2026-03-26 09:00:18,562 - lava_callback - DEBUG - Processing node 69c4f4c2f63ce05331e2ae6e: baseline-arm64 INFO: 172.21.0.1:57778 - "GET /login HTTP/1.0" 404 Not Found 2026-03-26 09:00:18,943 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 09:00:19,066 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 09:00:19,067 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c4f4c2f63ce05331e2ae6e/log.txt.gz 2026-03-26 09:00:19,264 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 09:00:19,473 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 09:00:19,475 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c4f4c2f63ce05331e2ae6e/lava-logs.yaml 2026-03-26 09:00:19,484 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 09:00:19,615 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 09:00:19,617 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c4f4c2f63ce05331e2ae6e/lava_callback.json.gz 2026-03-26 09:00:19,636 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:00:19,654 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f4c2f63ce05331e2ae6e HTTP/1.1" 200 None 2026-03-26 09:00:19,656 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:00:19,670 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebd8f63ce05331e2a492 HTTP/1.1" 200 None 2026-03-26 09:00:19,671 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:00:19,699 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4f4c2f63ce05331e2ae6e HTTP/1.1" 200 None 2026-03-26 09:00:19,706 - lava_callback - INFO - Completed processing callback for node 69c4f4c2f63ce05331e2ae6e INFO: 172.21.0.1:39294 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45746 - "GET / HTTP/1.0" 200 OK 2026-03-26 09:00:39,953 - lava_callback - INFO - Received callback for node 69c252a24a2c5046eb880492 INFO: 172.21.0.1:47824 - "POST /node/69c252a24a2c5046eb880492 HTTP/1.0" 202 Accepted 2026-03-26 09:00:39,959 - lava_callback - INFO - Starting async job submit for node 69c252a24a2c5046eb880492 2026-03-26 09:00:39,959 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:00:40,017 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c252a24a2c5046eb880492 HTTP/1.1" 200 None 2026-03-26 09:00:40,018 - lava_callback - DEBUG - Processing node 69c252a24a2c5046eb880492: baseline-arm-clabbe 2026-03-26 09:00:40,018 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c252a24a2c5046eb880492 2026-03-26 09:00:40,018 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:00:40,051 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c252a24a2c5046eb880492 HTTP/1.1" 200 None 2026-03-26 09:00:40,053 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:00:40,089 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ba5f6d537ae4d3c8236e46 HTTP/1.1" 200 None 2026-03-26 09:00:40,090 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:00:40,112 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c252a24a2c5046eb880492 HTTP/1.1" 200 None 2026-03-26 09:00:40,113 - lava_callback - ERROR - Error processing callback for node 69c252a24a2c5046eb880492: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-26 09:00:41,656 - lava_callback - INFO - Received callback for node 69c4f4d4f63ce05331e2aeb9 INFO: 172.21.0.1:47838 - "POST /node/69c4f4d4f63ce05331e2aeb9 HTTP/1.0" 202 Accepted 2026-03-26 09:00:41,663 - lava_callback - INFO - Starting async job submit for node 69c4f4d4f63ce05331e2aeb9 2026-03-26 09:00:41,681 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:00:41,696 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f4d4f63ce05331e2aeb9 HTTP/1.1" 200 None 2026-03-26 09:00:41,697 - lava_callback - DEBUG - Processing node 69c4f4d4f63ce05331e2aeb9: baseline-nfs-arm64 2026-03-26 09:00:42,122 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 09:00:42,265 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 09:00:42,267 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c4f4d4f63ce05331e2aeb9/log.txt.gz 2026-03-26 09:00:42,507 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 09:00:42,701 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 09:00:42,703 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c4f4d4f63ce05331e2aeb9/lava-logs.yaml 2026-03-26 09:00:42,714 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 09:00:42,876 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 09:00:42,878 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c4f4d4f63ce05331e2aeb9/lava_callback.json.gz 2026-03-26 09:00:42,897 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:00:42,917 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f4d4f63ce05331e2aeb9 HTTP/1.1" 200 None 2026-03-26 09:00:42,919 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:00:42,932 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebd8f63ce05331e2a492 HTTP/1.1" 200 None 2026-03-26 09:00:42,934 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:00:42,959 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4f4d4f63ce05331e2aeb9 HTTP/1.1" 200 None 2026-03-26 09:00:42,961 - lava_callback - INFO - Completed processing callback for node 69c4f4d4f63ce05331e2aeb9 2026-03-26 09:00:45,809 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:00:45,826 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:47840 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42918 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42928 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35614 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49234 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49240 - "GET / HTTP/1.0" 200 OK 2026-03-26 09:01:19,918 - lava_callback - INFO - Received callback for node 69c252d54a2c5046eb8804b3 INFO: 172.21.0.1:56742 - "POST /node/69c252d54a2c5046eb8804b3 HTTP/1.0" 202 Accepted 2026-03-26 09:01:19,923 - lava_callback - INFO - Starting async job submit for node 69c252d54a2c5046eb8804b3 2026-03-26 09:01:19,924 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:01:19,943 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c252d54a2c5046eb8804b3 HTTP/1.1" 200 None 2026-03-26 09:01:19,944 - lava_callback - DEBUG - Processing node 69c252d54a2c5046eb8804b3: baseline-arm64-clabbe 2026-03-26 09:01:19,944 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c252d54a2c5046eb8804b3 2026-03-26 09:01:19,945 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:01:19,961 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c252d54a2c5046eb8804b3 HTTP/1.1" 200 None 2026-03-26 09:01:19,962 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:01:19,975 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5827544ab5cd4a7d9dd HTTP/1.1" 200 None 2026-03-26 09:01:19,977 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:01:19,997 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c252d54a2c5046eb8804b3 HTTP/1.1" 200 None 2026-03-26 09:01:20,008 - lava_callback - ERROR - Error processing callback for node 69c252d54a2c5046eb8804b3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:56750 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38598 - "GET / HTTP/1.0" 200 OK 2026-03-26 09:01:38,058 - lava_callback - INFO - Received callback for node 69c4f4f9f63ce05331e2af21 INFO: 172.21.0.1:38604 - "POST /node/69c4f4f9f63ce05331e2af21 HTTP/1.0" 202 Accepted 2026-03-26 09:01:38,064 - lava_callback - INFO - Starting async job submit for node 69c4f4f9f63ce05331e2af21 2026-03-26 09:01:38,086 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:01:38,101 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f4f9f63ce05331e2af21 HTTP/1.1" 200 None 2026-03-26 09:01:38,102 - lava_callback - DEBUG - Processing node 69c4f4f9f63ce05331e2af21: baseline-arm64-broonie 2026-03-26 09:01:38,362 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 09:01:38,534 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 09:01:38,535 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c4f4f9f63ce05331e2af21/log.txt.gz 2026-03-26 09:01:38,696 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 09:01:38,902 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 09:01:38,904 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c4f4f9f63ce05331e2af21/lava-logs.yaml 2026-03-26 09:01:38,911 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 09:01:39,058 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 09:01:39,060 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c4f4f9f63ce05331e2af21/lava_callback.json.gz 2026-03-26 09:01:39,083 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:01:39,097 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f4f9f63ce05331e2af21 HTTP/1.1" 200 None 2026-03-26 09:01:39,099 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:01:39,111 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebd8f63ce05331e2a492 HTTP/1.1" 200 None 2026-03-26 09:01:39,113 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:01:39,137 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4f4f9f63ce05331e2af21 HTTP/1.1" 200 None 2026-03-26 09:01:39,147 - lava_callback - INFO - Completed processing callback for node 69c4f4f9f63ce05331e2af21 2026-03-26 09:01:45,828 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:01:45,864 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:42534 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33940 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33952 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54330 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59342 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59346 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40898 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45946 - "GET / HTTP/1.0" 200 OK 2026-03-26 09:02:40,052 - lava_callback - INFO - Received callback for node 69c253274a2c5046eb8804c3 INFO: 172.21.0.1:55770 - "POST /node/69c253274a2c5046eb8804c3 HTTP/1.0" 202 Accepted 2026-03-26 09:02:40,057 - lava_callback - INFO - Starting async job submit for node 69c253274a2c5046eb8804c3 2026-03-26 09:02:40,058 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:02:40,083 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c253274a2c5046eb8804c3 HTTP/1.1" 200 None 2026-03-26 09:02:40,084 - lava_callback - DEBUG - Processing node 69c253274a2c5046eb8804c3: baseline-arm64-clabbe 2026-03-26 09:02:40,085 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c253274a2c5046eb8804c3 2026-03-26 09:02:40,085 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:02:40,101 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c253274a2c5046eb8804c3 HTTP/1.1" 200 None 2026-03-26 09:02:40,103 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:02:40,121 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5827544ab5cd4a7d9dd HTTP/1.1" 200 None 2026-03-26 09:02:40,123 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:02:40,144 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c253274a2c5046eb8804c3 HTTP/1.1" 200 None 2026-03-26 09:02:40,146 - lava_callback - ERROR - Error processing callback for node 69c253274a2c5046eb8804c3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:55782 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55226 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55232 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33008 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35722 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35734 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37328 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35086 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57610 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32964 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32974 - "GET / HTTP/1.0" 200 OK 2026-03-26 09:04:00,102 - lava_callback - INFO - Received callback for node 69c253784a2c5046eb8804e9 INFO: 172.21.0.1:52322 - "POST /node/69c253784a2c5046eb8804e9 HTTP/1.0" 202 Accepted 2026-03-26 09:04:00,108 - lava_callback - INFO - Starting async job submit for node 69c253784a2c5046eb8804e9 2026-03-26 09:04:00,109 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:04:00,123 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c253784a2c5046eb8804e9 HTTP/1.1" 200 None 2026-03-26 09:04:00,124 - lava_callback - DEBUG - Processing node 69c253784a2c5046eb8804e9: baseline-arm64-clabbe 2026-03-26 09:04:00,124 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c253784a2c5046eb8804e9 2026-03-26 09:04:00,125 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:04:00,138 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c253784a2c5046eb8804e9 HTTP/1.1" 200 None 2026-03-26 09:04:00,140 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:04:00,154 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5827544ab5cd4a7d9dd HTTP/1.1" 200 None 2026-03-26 09:04:00,155 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:04:00,176 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c253784a2c5046eb8804e9 HTTP/1.1" 200 None 2026-03-26 09:04:00,178 - lava_callback - ERROR - Error processing callback for node 69c253784a2c5046eb8804e9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:52326 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48500 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48514 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40674 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42630 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41526 - "GET / HTTP/1.0" 200 OK 2026-03-26 09:04:55,159 - lava_callback - INFO - Received callback for node 69c4f4d1f63ce05331e2ae92 INFO: 172.21.0.1:38450 - "POST /node/69c4f4d1f63ce05331e2ae92 HTTP/1.0" 202 Accepted 2026-03-26 09:04:55,166 - lava_callback - INFO - Starting async job submit for node 69c4f4d1f63ce05331e2ae92 2026-03-26 09:04:55,203 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:04:55,222 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f4d1f63ce05331e2ae92 HTTP/1.1" 200 None 2026-03-26 09:04:55,223 - lava_callback - DEBUG - Processing node 69c4f4d1f63ce05331e2ae92: baseline-arm64 2026-03-26 09:04:55,628 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 09:04:55,925 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 09:04:55,927 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c4f4d1f63ce05331e2ae92/log.txt.gz 2026-03-26 09:04:56,151 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 09:04:56,378 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 09:04:56,380 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c4f4d1f63ce05331e2ae92/lava-logs.yaml 2026-03-26 09:04:56,389 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.21.0.1:38456 - "GET / HTTP/1.0" 200 OK 2026-03-26 09:04:56,708 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 09:04:56,710 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c4f4d1f63ce05331e2ae92/lava_callback.json.gz DEBUG: Setup failed for 69c4f4d1f63ce05331e2ae92. Transitting job node result: incomplete -> fail 2026-03-26 09:04:56,785 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:04:56,799 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f4d1f63ce05331e2ae92 HTTP/1.1" 200 None 2026-03-26 09:04:56,800 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:04:56,816 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebd8f63ce05331e2a492 HTTP/1.1" 200 None 2026-03-26 09:04:56,818 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:04:56,851 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4f4d1f63ce05331e2ae92 HTTP/1.1" 200 None 2026-03-26 09:04:56,940 - lava_callback - INFO - Completed processing callback for node 69c4f4d1f63ce05331e2ae92 INFO: 172.21.0.1:38468 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39816 - "GET / HTTP/1.0" 200 OK 2026-03-26 09:05:08,126 - lava_callback - INFO - Received callback for node 69c4f115f63ce05331e2abaf INFO: 172.21.0.1:39820 - "POST /node/69c4f115f63ce05331e2abaf HTTP/1.0" 202 Accepted 2026-03-26 09:05:08,173 - lava_callback - INFO - Starting async job submit for node 69c4f115f63ce05331e2abaf 2026-03-26 09:05:08,198 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:05:08,213 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f115f63ce05331e2abaf HTTP/1.1" 200 None 2026-03-26 09:05:08,214 - lava_callback - DEBUG - Processing node 69c4f115f63ce05331e2abaf: ltp-fs_bind 2026-03-26 09:05:08,309 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 09:05:08,453 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 09:05:08,455 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-fs_bind-69c4f115f63ce05331e2abaf/log.txt.gz 2026-03-26 09:05:08,508 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 09:05:08,682 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 09:05:08,684 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-fs_bind-69c4f115f63ce05331e2abaf/lava-logs.yaml 2026-03-26 09:05:08,687 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 09:05:08,808 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 09:05:08,809 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-fs_bind-69c4f115f63ce05331e2abaf/lava_callback.json.gz 2026-03-26 09:05:08,835 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:05:08,850 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f115f63ce05331e2abaf HTTP/1.1" 200 None 2026-03-26 09:05:08,852 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:05:08,869 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebcbf63ce05331e2a47f HTTP/1.1" 200 None 2026-03-26 09:05:08,870 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:05:08,893 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4f115f63ce05331e2abaf HTTP/1.1" 200 None 2026-03-26 09:05:08,970 - lava_callback - INFO - Completed processing callback for node 69c4f115f63ce05331e2abaf INFO: 172.21.0.1:47954 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-26 09:05:15,866 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:05:15,882 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:47968 - "GET / HTTP/1.0" 200 OK 2026-03-26 09:05:20,352 - lava_callback - INFO - Received callback for node 69c253ca4a2c5046eb8804f8 INFO: 172.21.0.1:39062 - "POST /node/69c253ca4a2c5046eb8804f8 HTTP/1.0" 202 Accepted 2026-03-26 09:05:20,357 - lava_callback - INFO - Starting async job submit for node 69c253ca4a2c5046eb8804f8 2026-03-26 09:05:20,358 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:05:20,372 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c253ca4a2c5046eb8804f8 HTTP/1.1" 200 None 2026-03-26 09:05:20,373 - lava_callback - DEBUG - Processing node 69c253ca4a2c5046eb8804f8: baseline-arm64-clabbe 2026-03-26 09:05:20,373 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c253ca4a2c5046eb8804f8 2026-03-26 09:05:20,374 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:05:20,397 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c253ca4a2c5046eb8804f8 HTTP/1.1" 200 None 2026-03-26 09:05:20,398 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:05:20,412 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5827544ab5cd4a7d9dd HTTP/1.1" 200 None 2026-03-26 09:05:20,413 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:05:20,439 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c253ca4a2c5046eb8804f8 HTTP/1.1" 200 None 2026-03-26 09:05:20,440 - lava_callback - ERROR - Error processing callback for node 69c253ca4a2c5046eb8804f8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:39068 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40580 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32872 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51784 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51798 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42218 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56596 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56612 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49400 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35108 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45546 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33232 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33248 - "GET / HTTP/1.0" 200 OK 2026-03-26 09:07:00,226 - lava_callback - INFO - Received callback for node 69c2541e4a2c5046eb880515 INFO: 172.21.0.1:60014 - "POST /node/69c2541e4a2c5046eb880515 HTTP/1.0" 202 Accepted 2026-03-26 09:07:00,231 - lava_callback - INFO - Starting async job submit for node 69c2541e4a2c5046eb880515 2026-03-26 09:07:00,232 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:07:00,259 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2541e4a2c5046eb880515 HTTP/1.1" 200 None 2026-03-26 09:07:00,260 - lava_callback - DEBUG - Processing node 69c2541e4a2c5046eb880515: baseline-arm64-clabbe 2026-03-26 09:07:00,261 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2541e4a2c5046eb880515 2026-03-26 09:07:00,261 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:07:00,275 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2541e4a2c5046eb880515 HTTP/1.1" 200 None 2026-03-26 09:07:00,276 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:07:00,293 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5827544ab5cd4a7d9dd HTTP/1.1" 200 None 2026-03-26 09:07:00,294 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:07:00,320 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2541e4a2c5046eb880515 HTTP/1.1" 200 None 2026-03-26 09:07:00,322 - lava_callback - ERROR - Error processing callback for node 69c2541e4a2c5046eb880515: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:60024 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43746 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43748 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44066 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45282 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59380 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59834 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59838 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46974 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41028 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41034 - "GET / HTTP/1.0" 200 OK 2026-03-26 09:08:20,332 - lava_callback - INFO - Received callback for node 69c2546f4a2c5046eb880527 INFO: 172.21.0.1:52530 - "POST /node/69c2546f4a2c5046eb880527 HTTP/1.0" 202 Accepted 2026-03-26 09:08:20,337 - lava_callback - INFO - Starting async job submit for node 69c2546f4a2c5046eb880527 2026-03-26 09:08:20,338 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:08:20,353 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2546f4a2c5046eb880527 HTTP/1.1" 200 None 2026-03-26 09:08:20,353 - lava_callback - DEBUG - Processing node 69c2546f4a2c5046eb880527: baseline-arm64-clabbe 2026-03-26 09:08:20,354 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2546f4a2c5046eb880527 2026-03-26 09:08:20,354 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:08:20,389 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2546f4a2c5046eb880527 HTTP/1.1" 200 None 2026-03-26 09:08:20,391 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:08:20,405 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5827544ab5cd4a7d9dd HTTP/1.1" 200 None 2026-03-26 09:08:20,406 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:08:20,426 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2546f4a2c5046eb880527 HTTP/1.1" 200 None 2026-03-26 09:08:20,432 - lava_callback - ERROR - Error processing callback for node 69c2546f4a2c5046eb880527: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:52536 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41334 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57594 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44396 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44410 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39222 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45366 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45374 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42762 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60102 - "GET / HTTP/1.0" 200 OK 2026-03-26 09:09:36,977 - lava_callback - INFO - Received callback for node 69c254be4a2c5046eb880538 INFO: 172.21.0.1:60110 - "POST /node/69c254be4a2c5046eb880538 HTTP/1.0" 202 Accepted 2026-03-26 09:09:36,983 - lava_callback - INFO - Starting async job submit for node 69c254be4a2c5046eb880538 2026-03-26 09:09:36,983 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:09:37,001 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c254be4a2c5046eb880538 HTTP/1.1" 200 None 2026-03-26 09:09:37,002 - lava_callback - DEBUG - Processing node 69c254be4a2c5046eb880538: baseline-arm64-baylibre 2026-03-26 09:09:37,002 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c254be4a2c5046eb880538 2026-03-26 09:09:37,003 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:09:37,017 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c254be4a2c5046eb880538 HTTP/1.1" 200 None 2026-03-26 09:09:37,019 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:09:37,032 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd027905e4bbd6ba634c66 HTTP/1.1" 200 None 2026-03-26 09:09:37,034 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:09:37,057 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c254be4a2c5046eb880538 HTTP/1.1" 200 None 2026-03-26 09:09:37,059 - lava_callback - ERROR - Error processing callback for node 69c254be4a2c5046eb880538: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:60398 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56132 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56136 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38974 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47002 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47014 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50358 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39592 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37428 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36804 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36818 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53230 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33750 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33760 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49858 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45222 - "GET / HTTP/1.0" 200 OK 2026-03-26 09:11:45,223 - lava_callback - INFO - Received callback for node 69c4f80af63ce05331e2b128 INFO: 172.21.0.1:51222 - "POST /node/69c4f80af63ce05331e2b128 HTTP/1.0" 202 Accepted 2026-03-26 09:11:45,230 - lava_callback - INFO - Starting async job submit for node 69c4f80af63ce05331e2b128 2026-03-26 09:11:45,249 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:11:45,265 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f80af63ce05331e2b128 HTTP/1.1" 200 None 2026-03-26 09:11:45,266 - lava_callback - DEBUG - Processing node 69c4f80af63ce05331e2b128: baseline-x86 2026-03-26 09:11:45,615 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 09:11:45,814 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 09:11:45,816 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69c4f80af63ce05331e2b128/log.txt.gz 2026-03-26 09:11:46,037 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 09:11:46,321 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 09:11:46,323 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69c4f80af63ce05331e2b128/lava-logs.yaml 2026-03-26 09:11:46,332 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.21.0.1:51232 - "GET / HTTP/1.0" 200 OK 2026-03-26 09:11:46,529 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 09:11:46,531 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69c4f80af63ce05331e2b128/lava_callback.json.gz 2026-03-26 09:11:46,554 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:11:46,571 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f80af63ce05331e2b128 HTTP/1.1" 200 None 2026-03-26 09:11:46,572 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:11:46,586 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebecf63ce05331e2a4ae HTTP/1.1" 200 None 2026-03-26 09:11:46,588 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:11:46,614 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4f80af63ce05331e2b128 HTTP/1.1" 200 None 2026-03-26 09:11:46,625 - lava_callback - INFO - Completed processing callback for node 69c4f80af63ce05331e2b128 INFO: 172.21.0.1:58146 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58160 - "GET / HTTP/1.0" 200 OK 2026-03-26 09:12:00,426 - lava_callback - INFO - Received callback for node 69c2555c4a2c5046eb880569 INFO: 172.21.0.1:40512 - "POST /node/69c2555c4a2c5046eb880569 HTTP/1.0" 202 Accepted 2026-03-26 09:12:00,434 - lava_callback - INFO - Starting async job submit for node 69c2555c4a2c5046eb880569 2026-03-26 09:12:00,435 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:12:00,457 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2555c4a2c5046eb880569 HTTP/1.1" 200 None 2026-03-26 09:12:00,458 - lava_callback - DEBUG - Processing node 69c2555c4a2c5046eb880569: baseline-arm64-clabbe 2026-03-26 09:12:00,459 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2555c4a2c5046eb880569 2026-03-26 09:12:00,459 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:12:00,482 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2555c4a2c5046eb880569 HTTP/1.1" 200 None 2026-03-26 09:12:00,484 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:12:00,505 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd027905e4bbd6ba634c66 HTTP/1.1" 200 None 2026-03-26 09:12:00,507 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:12:00,542 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2555c4a2c5046eb880569 HTTP/1.1" 200 None 2026-03-26 09:12:00,544 - lava_callback - ERROR - Error processing callback for node 69c2555c4a2c5046eb880569: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:40522 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58284 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-26 09:12:15,889 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:12:15,906 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:58294 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43832 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57382 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40424 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40396 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40410 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35088 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40858 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40862 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37602 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33238 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33302 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39874 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39878 - "GET / HTTP/1.0" 200 OK 2026-03-26 09:14:00,445 - lava_callback - INFO - Received callback for node 69c255d34a2c5046eb880589 INFO: 172.21.0.1:37706 - "POST /node/69c255d34a2c5046eb880589 HTTP/1.0" 202 Accepted 2026-03-26 09:14:00,451 - lava_callback - INFO - Starting async job submit for node 69c255d34a2c5046eb880589 2026-03-26 09:14:00,452 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:14:00,470 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c255d34a2c5046eb880589 HTTP/1.1" 200 None 2026-03-26 09:14:00,471 - lava_callback - DEBUG - Processing node 69c255d34a2c5046eb880589: baseline-arm64-clabbe 2026-03-26 09:14:00,471 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c255d34a2c5046eb880589 2026-03-26 09:14:00,472 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:14:00,492 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c255d34a2c5046eb880589 HTTP/1.1" 200 None 2026-03-26 09:14:00,493 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:14:00,507 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd027905e4bbd6ba634c66 HTTP/1.1" 200 None 2026-03-26 09:14:00,508 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:14:00,529 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c255d34a2c5046eb880589 HTTP/1.1" 200 None 2026-03-26 09:14:00,534 - lava_callback - ERROR - Error processing callback for node 69c255d34a2c5046eb880589: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:37712 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54586 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54602 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59592 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54830 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50352 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45064 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45066 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50894 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38184 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38190 - "GET / HTTP/1.0" 200 OK 2026-03-26 09:15:20,532 - lava_callback - INFO - Received callback for node 69c256244a2c5046eb88058f INFO: 172.21.0.1:54576 - "POST /node/69c256244a2c5046eb88058f HTTP/1.0" 202 Accepted 2026-03-26 09:15:20,538 - lava_callback - INFO - Starting async job submit for node 69c256244a2c5046eb88058f 2026-03-26 09:15:20,539 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:15:20,558 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c256244a2c5046eb88058f HTTP/1.1" 200 None 2026-03-26 09:15:20,559 - lava_callback - DEBUG - Processing node 69c256244a2c5046eb88058f: baseline-arm64-clabbe 2026-03-26 09:15:20,559 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c256244a2c5046eb88058f 2026-03-26 09:15:20,560 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:15:20,585 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c256244a2c5046eb88058f HTTP/1.1" 200 None 2026-03-26 09:15:20,587 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:15:20,623 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd027905e4bbd6ba634c66 HTTP/1.1" 200 None 2026-03-26 09:15:20,624 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:15:20,645 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c256244a2c5046eb88058f HTTP/1.1" 200 None 2026-03-26 09:15:20,651 - lava_callback - ERROR - Error processing callback for node 69c256244a2c5046eb88058f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:54582 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40452 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57444 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35440 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35446 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50302 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36282 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36296 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39134 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49156 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59916 - "GET / HTTP/1.0" 200 OK 2026-03-26 09:16:53,097 - lava_callback - INFO - Received callback for node 69c4f4d7f63ce05331e2aec2 INFO: 172.21.0.1:47430 - "POST /node/69c4f4d7f63ce05331e2aec2 HTTP/1.0" 202 Accepted 2026-03-26 09:16:53,105 - lava_callback - INFO - Starting async job submit for node 69c4f4d7f63ce05331e2aec2 2026-03-26 09:16:53,149 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:16:53,163 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f4d7f63ce05331e2aec2 HTTP/1.1" 200 None 2026-03-26 09:16:53,164 - lava_callback - DEBUG - Processing node 69c4f4d7f63ce05331e2aec2: baseline-nfs-arm64 2026-03-26 09:16:54,247 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 09:16:54,441 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 09:16:54,443 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c4f4d7f63ce05331e2aec2/log.txt.gz 2026-03-26 09:16:55,022 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 09:16:55,288 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 09:16:55,291 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c4f4d7f63ce05331e2aec2/lava-logs.yaml 2026-03-26 09:16:55,311 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 09:16:55,505 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 09:16:55,506 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c4f4d7f63ce05331e2aec2/lava_callback.json.gz DEBUG: Setup failed for 69c4f4d7f63ce05331e2aec2. Transitting job node result: incomplete -> fail 2026-03-26 09:16:55,595 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:16:55,610 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f4d7f63ce05331e2aec2 HTTP/1.1" 200 None 2026-03-26 09:16:55,611 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:16:55,628 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebd8f63ce05331e2a492 HTTP/1.1" 200 None 2026-03-26 09:16:55,629 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:16:55,652 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4f4d7f63ce05331e2aec2 HTTP/1.1" 200 None 2026-03-26 09:16:55,764 - lava_callback - INFO - Completed processing callback for node 69c4f4d7f63ce05331e2aec2 INFO: 172.21.0.1:47432 - "GET / HTTP/1.0" 200 OK 2026-03-26 09:16:57,340 - lava_callback - INFO - Received callback for node 69c256864a2c5046eb8805ac INFO: 172.21.0.1:47440 - "POST /node/69c256864a2c5046eb8805ac HTTP/1.0" 202 Accepted 2026-03-26 09:16:57,346 - lava_callback - INFO - Starting async job submit for node 69c256864a2c5046eb8805ac 2026-03-26 09:16:57,346 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:16:57,360 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c256864a2c5046eb8805ac HTTP/1.1" 200 None 2026-03-26 09:16:57,361 - lava_callback - DEBUG - Processing node 69c256864a2c5046eb8805ac: baseline-arm-baylibre 2026-03-26 09:16:57,361 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c256864a2c5046eb8805ac 2026-03-26 09:16:57,362 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:16:57,384 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c256864a2c5046eb8805ac HTTP/1.1" 200 None 2026-03-26 09:16:57,385 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:16:57,400 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5737544ab5cd4a7d9ce HTTP/1.1" 200 None 2026-03-26 09:16:57,401 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:16:57,421 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c256864a2c5046eb8805ac HTTP/1.1" 200 None 2026-03-26 09:16:57,423 - lava_callback - ERROR - Error processing callback for node 69c256864a2c5046eb8805ac: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:47448 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58168 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60372 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-26 09:17:15,909 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:17:15,925 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:60388 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43520 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55598 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56260 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38422 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38424 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51902 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58028 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58040 - "GET / HTTP/1.0" 200 OK 2026-03-26 09:18:18,366 - lava_callback - INFO - Received callback for node 69c4f5aaf63ce05331e2b095 INFO: 172.21.0.1:58044 - "POST /node/69c4f5aaf63ce05331e2b095 HTTP/1.0" 202 Accepted 2026-03-26 09:18:18,372 - lava_callback - INFO - Starting async job submit for node 69c4f5aaf63ce05331e2b095 2026-03-26 09:18:18,450 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:18:18,466 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f5aaf63ce05331e2b095 HTTP/1.1" 200 None 2026-03-26 09:18:18,466 - lava_callback - DEBUG - Processing node 69c4f5aaf63ce05331e2b095: ltp-smoke 2026-03-26 09:18:18,956 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 09:18:19,119 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 09:18:19,121 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoke-69c4f5aaf63ce05331e2b095/log.txt.gz 2026-03-26 09:18:19,399 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 09:18:19,664 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 09:18:19,667 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-smoke-69c4f5aaf63ce05331e2b095/lava-logs.yaml 2026-03-26 09:18:19,678 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 09:18:19,853 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 09:18:19,855 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoke-69c4f5aaf63ce05331e2b095/lava_callback.json.gz 2026-03-26 09:18:19,974 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:18:19,997 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f5aaf63ce05331e2b095 HTTP/1.1" 200 None 2026-03-26 09:18:19,999 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:18:20,019 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebd0f63ce05331e2a489 HTTP/1.1" 200 None 2026-03-26 09:18:20,021 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:18:20,044 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4f5aaf63ce05331e2b095 HTTP/1.1" 200 None 2026-03-26 09:18:20,213 - lava_callback - INFO - Completed processing callback for node 69c4f5aaf63ce05331e2b095 INFO: 172.21.0.1:49768 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44144 - "GET / HTTP/1.0" 200 OK 2026-03-26 09:18:45,927 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:18:45,943 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:41186 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43844 - "GET / HTTP/1.0" 200 OK 2026-03-26 09:18:57,448 - lava_callback - INFO - Received callback for node 69c256fc4a2c5046eb8805c5 INFO: 172.21.0.1:43848 - "POST /node/69c256fc4a2c5046eb8805c5 HTTP/1.0" 202 Accepted 2026-03-26 09:18:57,454 - lava_callback - INFO - Starting async job submit for node 69c256fc4a2c5046eb8805c5 2026-03-26 09:18:57,454 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:18:57,478 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c256fc4a2c5046eb8805c5 HTTP/1.1" 200 None 2026-03-26 09:18:57,479 - lava_callback - DEBUG - Processing node 69c256fc4a2c5046eb8805c5: baseline-arm-baylibre 2026-03-26 09:18:57,479 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c256fc4a2c5046eb8805c5 2026-03-26 09:18:57,479 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:18:57,499 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c256fc4a2c5046eb8805c5 HTTP/1.1" 200 None 2026-03-26 09:18:57,500 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:18:57,514 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5737544ab5cd4a7d9ce HTTP/1.1" 200 None 2026-03-26 09:18:57,515 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:18:57,535 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c256fc4a2c5046eb8805c5 HTTP/1.1" 200 None 2026-03-26 09:18:57,547 - lava_callback - ERROR - Error processing callback for node 69c256fc4a2c5046eb8805c5: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:43852 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37826 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54966 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54976 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59122 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37424 - "GET / HTTP/1.0" 200 OK 2026-03-26 09:19:37,419 - lava_callback - INFO - Received callback for node 69c257204a2c5046eb8805ce INFO: 172.21.0.1:37440 - "POST /node/69c257204a2c5046eb8805ce HTTP/1.0" 202 Accepted 2026-03-26 09:19:37,424 - lava_callback - INFO - Starting async job submit for node 69c257204a2c5046eb8805ce 2026-03-26 09:19:37,425 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:19:37,440 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c257204a2c5046eb8805ce HTTP/1.1" 200 None 2026-03-26 09:19:37,441 - lava_callback - DEBUG - Processing node 69c257204a2c5046eb8805ce: baseline-arm-baylibre 2026-03-26 09:19:37,441 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c257204a2c5046eb8805ce 2026-03-26 09:19:37,442 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:19:37,456 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c257204a2c5046eb8805ce HTTP/1.1" 200 None 2026-03-26 09:19:37,458 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:19:37,473 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5737544ab5cd4a7d9ce HTTP/1.1" 200 None 2026-03-26 09:19:37,475 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:19:37,497 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c257204a2c5046eb8805ce HTTP/1.1" 200 None 2026-03-26 09:19:37,499 - lava_callback - ERROR - Error processing callback for node 69c257204a2c5046eb8805ce: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:46738 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57384 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57396 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58782 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45432 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45442 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43200 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58994 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35436 - "GET / HTTP/1.0" 200 OK 2026-03-26 09:20:47,124 - lava_callback - INFO - Received callback for node 69c4f4caf63ce05331e2ae80 INFO: 172.21.0.1:35440 - "POST /node/69c4f4caf63ce05331e2ae80 HTTP/1.0" 202 Accepted 2026-03-26 09:20:47,131 - lava_callback - INFO - Starting async job submit for node 69c4f4caf63ce05331e2ae80 2026-03-26 09:20:47,151 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:20:47,171 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f4caf63ce05331e2ae80 HTTP/1.1" 200 None 2026-03-26 09:20:47,172 - lava_callback - DEBUG - Processing node 69c4f4caf63ce05331e2ae80: baseline-arm64 2026-03-26 09:20:47,397 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 09:20:47,567 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 09:20:47,568 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c4f4caf63ce05331e2ae80/log.txt.gz 2026-03-26 09:20:47,704 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 09:20:47,923 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 09:20:47,925 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c4f4caf63ce05331e2ae80/lava-logs.yaml 2026-03-26 09:20:47,930 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 09:20:48,091 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 09:20:48,093 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c4f4caf63ce05331e2ae80/lava_callback.json.gz 2026-03-26 09:20:48,115 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:20:48,129 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f4caf63ce05331e2ae80 HTTP/1.1" 200 None 2026-03-26 09:20:48,131 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:20:48,151 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebd8f63ce05331e2a492 HTTP/1.1" 200 None 2026-03-26 09:20:48,153 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:20:48,183 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4f4caf63ce05331e2ae80 HTTP/1.1" 200 None 2026-03-26 09:20:48,184 - lava_callback - INFO - Completed processing callback for node 69c4f4caf63ce05331e2ae80 INFO: 172.21.0.1:54692 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54700 - "GET / HTTP/1.0" 200 OK 2026-03-26 09:21:00,741 - lava_callback - INFO - Received callback for node 69c257784a2c5046eb8805e6 INFO: 172.21.0.1:54870 - "POST /node/69c257784a2c5046eb8805e6 HTTP/1.0" 202 Accepted 2026-03-26 09:21:00,747 - lava_callback - INFO - Starting async job submit for node 69c257784a2c5046eb8805e6 2026-03-26 09:21:00,748 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:21:00,762 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c257784a2c5046eb8805e6 HTTP/1.1" 200 None 2026-03-26 09:21:00,763 - lava_callback - DEBUG - Processing node 69c257784a2c5046eb8805e6: baseline-arm64-clabbe 2026-03-26 09:21:00,764 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c257784a2c5046eb8805e6 2026-03-26 09:21:00,764 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:21:00,784 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c257784a2c5046eb8805e6 HTTP/1.1" 200 None 2026-03-26 09:21:00,786 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:21:00,836 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd027905e4bbd6ba634c66 HTTP/1.1" 200 None 2026-03-26 09:21:00,838 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:21:00,860 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c257784a2c5046eb8805e6 HTTP/1.1" 200 None 2026-03-26 09:21:00,861 - lava_callback - ERROR - Error processing callback for node 69c257784a2c5046eb8805e6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:54882 - "GET / HTTP/1.0" 200 OK 2026-03-26 09:21:14,833 - lava_callback - INFO - Received callback for node 69c4fa64f63ce05331e2b194 INFO: 172.21.0.1:46124 - "POST /node/69c4fa64f63ce05331e2b194 HTTP/1.0" 202 Accepted 2026-03-26 09:21:14,840 - lava_callback - INFO - Starting async job submit for node 69c4fa64f63ce05331e2b194 2026-03-26 09:21:14,858 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:21:14,872 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4fa64f63ce05331e2b194 HTTP/1.1" 200 None 2026-03-26 09:21:14,873 - lava_callback - DEBUG - Processing node 69c4fa64f63ce05331e2b194: baseline-arm64 2026-03-26 09:21:15,023 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.21.0.1:46126 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-26 09:21:15,324 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 09:21:15,326 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c4fa64f63ce05331e2b194/log.txt.gz 2026-03-26 09:21:15,416 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 09:21:15,646 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 09:21:15,648 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c4fa64f63ce05331e2b194/lava-logs.yaml 2026-03-26 09:21:15,652 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 09:21:15,788 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 09:21:15,790 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c4fa64f63ce05331e2b194/lava_callback.json.gz 2026-03-26 09:21:15,809 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:21:15,825 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4fa64f63ce05331e2b194 HTTP/1.1" 200 None 2026-03-26 09:21:15,826 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:21:15,840 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebd8f63ce05331e2a492 HTTP/1.1" 200 None 2026-03-26 09:21:15,841 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:21:15,865 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4fa64f63ce05331e2b194 HTTP/1.1" 200 None 2026-03-26 09:21:15,872 - lava_callback - INFO - Completed processing callback for node 69c4fa64f63ce05331e2b194 2026-03-26 09:21:15,946 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:21:15,967 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:46142 - "GET / HTTP/1.0" 200 OK 2026-03-26 09:21:19,655 - lava_callback - INFO - Received callback for node 69c4fa69f63ce05331e2b197 INFO: 172.21.0.1:46150 - "POST /node/69c4fa69f63ce05331e2b197 HTTP/1.0" 202 Accepted 2026-03-26 09:21:19,662 - lava_callback - INFO - Starting async job submit for node 69c4fa69f63ce05331e2b197 2026-03-26 09:21:19,681 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:21:19,695 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4fa69f63ce05331e2b197 HTTP/1.1" 200 None 2026-03-26 09:21:19,696 - lava_callback - DEBUG - Processing node 69c4fa69f63ce05331e2b197: baseline-arm64 2026-03-26 09:21:19,849 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 09:21:19,982 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 09:21:19,984 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c4fa69f63ce05331e2b197/log.txt.gz 2026-03-26 09:21:20,074 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 09:21:20,426 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 09:21:20,428 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c4fa69f63ce05331e2b197/lava-logs.yaml 2026-03-26 09:21:20,433 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 09:21:20,583 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 09:21:20,585 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c4fa69f63ce05331e2b197/lava_callback.json.gz 2026-03-26 09:21:20,602 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:21:20,622 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4fa69f63ce05331e2b197 HTTP/1.1" 200 None 2026-03-26 09:21:20,623 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:21:20,637 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebd8f63ce05331e2a492 HTTP/1.1" 200 None 2026-03-26 09:21:20,638 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:21:20,664 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4fa69f63ce05331e2b197 HTTP/1.1" 200 None 2026-03-26 09:21:20,669 - lava_callback - INFO - Completed processing callback for node 69c4fa69f63ce05331e2b197 INFO: 172.21.0.1:41604 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40456 - "GET / HTTP/1.0" 200 OK 2026-03-26 09:21:45,970 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:21:45,986 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:38482 - "GET / HTTP/1.0" 200 OK 2026-03-26 09:21:52,274 - lava_callback - INFO - Received callback for node 69c4f4ddf63ce05331e2aece INFO: 172.21.0.1:42650 - "POST /node/69c4f4ddf63ce05331e2aece HTTP/1.0" 202 Accepted 2026-03-26 09:21:52,281 - lava_callback - INFO - Starting async job submit for node 69c4f4ddf63ce05331e2aece 2026-03-26 09:21:52,303 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:21:52,318 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f4ddf63ce05331e2aece HTTP/1.1" 200 None 2026-03-26 09:21:52,319 - lava_callback - DEBUG - Processing node 69c4f4ddf63ce05331e2aece: baseline-nfs-arm64 2026-03-26 09:21:52,646 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 09:21:52,784 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 09:21:52,786 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c4f4ddf63ce05331e2aece/log.txt.gz 2026-03-26 09:21:52,974 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 09:21:53,201 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 09:21:53,203 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c4f4ddf63ce05331e2aece/lava-logs.yaml 2026-03-26 09:21:53,211 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 09:21:53,369 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 09:21:53,370 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c4f4ddf63ce05331e2aece/lava_callback.json.gz 2026-03-26 09:21:53,394 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:21:53,411 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f4ddf63ce05331e2aece HTTP/1.1" 200 None 2026-03-26 09:21:53,413 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:21:53,426 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebd8f63ce05331e2a492 HTTP/1.1" 200 None 2026-03-26 09:21:53,428 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:21:53,453 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4f4ddf63ce05331e2aece HTTP/1.1" 200 None 2026-03-26 09:21:53,454 - lava_callback - INFO - Completed processing callback for node 69c4f4ddf63ce05331e2aece INFO: 172.21.0.1:42652 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42668 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57210 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45260 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-26 09:22:15,988 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:22:16,014 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:45274 - "GET / HTTP/1.0" 200 OK 2026-03-26 09:22:20,818 - lava_callback - INFO - Received callback for node 69c257bb4a2c5046eb8805ed INFO: 172.21.0.1:48248 - "POST /node/69c257bb4a2c5046eb8805ed HTTP/1.0" 202 Accepted 2026-03-26 09:22:20,824 - lava_callback - INFO - Starting async job submit for node 69c257bb4a2c5046eb8805ed 2026-03-26 09:22:20,825 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:22:20,839 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c257bb4a2c5046eb8805ed HTTP/1.1" 200 None 2026-03-26 09:22:20,840 - lava_callback - DEBUG - Processing node 69c257bb4a2c5046eb8805ed: baseline-arm-clabbe 2026-03-26 09:22:20,841 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c257bb4a2c5046eb8805ed 2026-03-26 09:22:20,841 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:22:20,855 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c257bb4a2c5046eb8805ed HTTP/1.1" 200 None 2026-03-26 09:22:20,857 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:22:20,870 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5737544ab5cd4a7d9ce HTTP/1.1" 200 None 2026-03-26 09:22:20,872 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:22:20,896 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c257bb4a2c5046eb8805ed HTTP/1.1" 200 None 2026-03-26 09:22:20,897 - lava_callback - ERROR - Error processing callback for node 69c257bb4a2c5046eb8805ed: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:48258 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39330 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43938 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44108 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44122 - "GET / HTTP/1.0" 200 OK 2026-03-26 09:23:00,791 - lava_callback - INFO - Received callback for node 69c257e04a2c5046eb8805f7 INFO: 172.21.0.1:36220 - "POST /node/69c257e04a2c5046eb8805f7 HTTP/1.0" 202 Accepted 2026-03-26 09:23:00,799 - lava_callback - INFO - Starting async job submit for node 69c257e04a2c5046eb8805f7 2026-03-26 09:23:00,800 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:23:00,813 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c257e04a2c5046eb8805f7 HTTP/1.1" 200 None 2026-03-26 09:23:00,814 - lava_callback - DEBUG - Processing node 69c257e04a2c5046eb8805f7: baseline-arm-clabbe 2026-03-26 09:23:00,814 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c257e04a2c5046eb8805f7 2026-03-26 09:23:00,814 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:23:00,829 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c257e04a2c5046eb8805f7 HTTP/1.1" 200 None 2026-03-26 09:23:00,830 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:23:00,846 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5737544ab5cd4a7d9ce HTTP/1.1" 200 None 2026-03-26 09:23:00,847 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:23:00,868 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c257e04a2c5046eb8805f7 HTTP/1.1" 200 None 2026-03-26 09:23:00,869 - lava_callback - ERROR - Error processing callback for node 69c257e04a2c5046eb8805f7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:36236 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44078 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44090 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55188 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48644 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43720 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46378 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46388 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43844 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50530 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50542 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60542 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50870 - "GET / HTTP/1.0" 200 OK 2026-03-26 09:24:40,916 - lava_callback - INFO - Received callback for node 69c258544a2c5046eb88060d INFO: 172.21.0.1:36174 - "POST /node/69c258544a2c5046eb88060d HTTP/1.0" 202 Accepted 2026-03-26 09:24:40,927 - lava_callback - INFO - Starting async job submit for node 69c258544a2c5046eb88060d 2026-03-26 09:24:40,929 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:24:40,960 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c258544a2c5046eb88060d HTTP/1.1" 200 None 2026-03-26 09:24:40,962 - lava_callback - DEBUG - Processing node 69c258544a2c5046eb88060d: baseline-arm-clabbe 2026-03-26 09:24:40,962 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c258544a2c5046eb88060d 2026-03-26 09:24:40,963 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:24:40,987 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c258544a2c5046eb88060d HTTP/1.1" 200 None 2026-03-26 09:24:40,989 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:24:41,019 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5737544ab5cd4a7d9ce HTTP/1.1" 200 None 2026-03-26 09:24:41,021 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:24:41,043 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c258544a2c5046eb88060d HTTP/1.1" 200 None 2026-03-26 09:24:41,044 - lava_callback - ERROR - Error processing callback for node 69c258544a2c5046eb88060d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:36176 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54350 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54364 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42886 - "GET / HTTP/1.0" 200 OK 2026-03-26 09:25:12,360 - lava_callback - INFO - Received callback for node 69c4fb58f63ce05331e2b1d9 INFO: 172.21.0.1:50620 - "POST /node/69c4fb58f63ce05331e2b1d9 HTTP/1.0" 202 Accepted 2026-03-26 09:25:12,365 - lava_callback - INFO - Starting async job submit for node 69c4fb58f63ce05331e2b1d9 2026-03-26 09:25:12,367 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:25:12,381 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4fb58f63ce05331e2b1d9 HTTP/1.1" 200 None 2026-03-26 09:25:12,382 - lava_callback - DEBUG - Processing node 69c4fb58f63ce05331e2b1d9: baseline-nfs-arm64 2026-03-26 09:25:12,383 - lava_callback - WARNING - Failed to upload log 2026-03-26 09:25:12,385 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 09:25:12,542 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 09:25:12,544 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c4fb58f63ce05331e2b1d9/lava-logs.yaml 2026-03-26 09:25:12,545 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 09:25:12,686 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 09:25:12,687 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c4fb58f63ce05331e2b1d9/lava_callback.json.gz 2026-03-26 09:25:12,689 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:25:12,703 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4fb58f63ce05331e2b1d9 HTTP/1.1" 200 None 2026-03-26 09:25:12,705 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:25:12,718 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebd8f63ce05331e2a492 HTTP/1.1" 200 None 2026-03-26 09:25:12,719 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:25:12,745 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4fb58f63ce05331e2b1d9 HTTP/1.1" 200 None 2026-03-26 09:25:12,749 - lava_callback - INFO - Completed processing callback for node 69c4fb58f63ce05331e2b1d9 INFO: 172.21.0.1:50628 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-26 09:25:16,017 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:25:16,033 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:50632 - "GET / HTTP/1.0" 200 OK 2026-03-26 09:25:20,926 - lava_callback - INFO - Received callback for node 69c258784a2c5046eb880617 INFO: 172.21.0.1:53436 - "POST /node/69c258784a2c5046eb880617 HTTP/1.0" 202 Accepted 2026-03-26 09:25:20,931 - lava_callback - INFO - Starting async job submit for node 69c258784a2c5046eb880617 2026-03-26 09:25:20,932 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:25:20,947 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c258784a2c5046eb880617 HTTP/1.1" 200 None 2026-03-26 09:25:20,948 - lava_callback - DEBUG - Processing node 69c258784a2c5046eb880617: baseline-arm-clabbe 2026-03-26 09:25:20,948 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c258784a2c5046eb880617 2026-03-26 09:25:20,948 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:25:20,962 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c258784a2c5046eb880617 HTTP/1.1" 200 None 2026-03-26 09:25:20,963 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:25:20,976 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5737544ab5cd4a7d9ce HTTP/1.1" 200 None 2026-03-26 09:25:20,977 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:25:21,003 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c258784a2c5046eb880617 HTTP/1.1" 200 None 2026-03-26 09:25:21,005 - lava_callback - ERROR - Error processing callback for node 69c258784a2c5046eb880617: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:53448 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51974 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51668 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35030 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35034 - "GET / HTTP/1.0" 200 OK 2026-03-26 09:26:00,960 - lava_callback - INFO - Received callback for node 69c2589d4a2c5046eb880621 INFO: 172.21.0.1:44242 - "POST /node/69c2589d4a2c5046eb880621 HTTP/1.0" 202 Accepted 2026-03-26 09:26:00,966 - lava_callback - INFO - Starting async job submit for node 69c2589d4a2c5046eb880621 2026-03-26 09:26:00,967 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:26:00,984 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2589d4a2c5046eb880621 HTTP/1.1" 200 None 2026-03-26 09:26:00,985 - lava_callback - DEBUG - Processing node 69c2589d4a2c5046eb880621: baseline-arm-clabbe 2026-03-26 09:26:00,985 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2589d4a2c5046eb880621 2026-03-26 09:26:00,986 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:26:00,999 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2589d4a2c5046eb880621 HTTP/1.1" 200 None 2026-03-26 09:26:01,001 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:26:01,014 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5737544ab5cd4a7d9ce HTTP/1.1" 200 None 2026-03-26 09:26:01,015 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:26:01,036 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2589d4a2c5046eb880621 HTTP/1.1" 200 None 2026-03-26 09:26:01,041 - lava_callback - ERROR - Error processing callback for node 69c2589d4a2c5046eb880621: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:44250 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56614 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56616 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33448 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34886 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45674 - "GET / HTTP/1.0" 200 OK 2026-03-26 09:26:55,297 - lava_callback - INFO - Received callback for node 69c4fb9ef63ce05331e2b1e9 INFO: 172.21.0.1:60666 - "POST /node/69c4fb9ef63ce05331e2b1e9 HTTP/1.0" 202 Accepted 2026-03-26 09:26:55,304 - lava_callback - INFO - Starting async job submit for node 69c4fb9ef63ce05331e2b1e9 2026-03-26 09:26:55,334 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:26:55,353 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4fb9ef63ce05331e2b1e9 HTTP/1.1" 200 None 2026-03-26 09:26:55,354 - lava_callback - DEBUG - Processing node 69c4fb9ef63ce05331e2b1e9: baseline-arm64 2026-03-26 09:26:55,489 - lava_callback - WARNING - Failed to upload log 2026-03-26 09:26:55,560 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 09:26:55,750 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 09:26:55,752 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c4fb9ef63ce05331e2b1e9/lava-logs.yaml 2026-03-26 09:26:55,755 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 09:26:55,895 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 09:26:55,897 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c4fb9ef63ce05331e2b1e9/lava_callback.json.gz 2026-03-26 09:26:55,927 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:26:55,942 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4fb9ef63ce05331e2b1e9 HTTP/1.1" 200 None 2026-03-26 09:26:55,943 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:26:55,956 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebd8f63ce05331e2a492 HTTP/1.1" 200 None 2026-03-26 09:26:55,958 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:26:55,982 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4fb9ef63ce05331e2b1e9 HTTP/1.1" 200 None 2026-03-26 09:26:56,062 - lava_callback - INFO - Completed processing callback for node 69c4fb9ef63ce05331e2b1e9 INFO: 172.21.0.1:60670 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60674 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55286 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56864 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-26 09:27:16,035 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:27:16,053 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:56866 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38286 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43748 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49816 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51438 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51450 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57920 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52354 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52368 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39148 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47356 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49580 - "GET / HTTP/1.0" 200 OK 2026-03-26 09:28:53,512 - lava_callback - INFO - Received callback for node 69c4fc11f63ce05331e2b201 INFO: 172.21.0.1:52910 - "POST /node/69c4fc11f63ce05331e2b201 HTTP/1.0" 202 Accepted 2026-03-26 09:28:53,521 - lava_callback - INFO - Starting async job submit for node 69c4fc11f63ce05331e2b201 2026-03-26 09:28:53,540 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:28:53,554 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4fc11f63ce05331e2b201 HTTP/1.1" 200 None 2026-03-26 09:28:53,555 - lava_callback - DEBUG - Processing node 69c4fc11f63ce05331e2b201: baseline-arm64 2026-03-26 09:28:54,817 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 09:28:55,039 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 09:28:55,040 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c4fc11f63ce05331e2b201/log.txt.gz 2026-03-26 09:28:55,805 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 09:28:56,077 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 09:28:56,080 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c4fc11f63ce05331e2b201/lava-logs.yaml 2026-03-26 09:28:56,099 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 09:28:56,284 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 09:28:56,286 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c4fc11f63ce05331e2b201/lava_callback.json.gz 2026-03-26 09:28:56,287 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:28:56,301 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4fc11f63ce05331e2b201 HTTP/1.1" 200 None 2026-03-26 09:28:56,302 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:28:56,320 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebd8f63ce05331e2a492 HTTP/1.1" 200 None 2026-03-26 09:28:56,321 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:28:56,342 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4fc11f63ce05331e2b201 HTTP/1.1" 200 None 2026-03-26 09:28:56,343 - lava_callback - INFO - Completed processing callback for node 69c4fc11f63ce05331e2b201 INFO: 172.21.0.1:52914 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52918 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59688 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54712 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-26 09:29:16,055 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:29:16,071 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:54718 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35480 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39272 - "GET / HTTP/1.0" 200 OK 2026-03-26 09:29:41,115 - lava_callback - INFO - Received callback for node 69c2596e4a2c5046eb880648 INFO: 172.21.0.1:32996 - "POST /node/69c2596e4a2c5046eb880648 HTTP/1.0" 202 Accepted 2026-03-26 09:29:41,121 - lava_callback - INFO - Starting async job submit for node 69c2596e4a2c5046eb880648 2026-03-26 09:29:41,121 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:29:41,136 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2596e4a2c5046eb880648 HTTP/1.1" 200 None 2026-03-26 09:29:41,137 - lava_callback - DEBUG - Processing node 69c2596e4a2c5046eb880648: baseline-arm64-clabbe 2026-03-26 09:29:41,137 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2596e4a2c5046eb880648 2026-03-26 09:29:41,137 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:29:41,157 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2596e4a2c5046eb880648 HTTP/1.1" 200 None 2026-03-26 09:29:41,158 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:29:41,171 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd027905e4bbd6ba634c66 HTTP/1.1" 200 None 2026-03-26 09:29:41,173 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:29:41,193 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2596e4a2c5046eb880648 HTTP/1.1" 200 None 2026-03-26 09:29:41,195 - lava_callback - ERROR - Error processing callback for node 69c2596e4a2c5046eb880648: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:33000 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44926 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44942 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51982 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43614 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43620 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46346 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56586 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39520 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38384 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38392 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36864 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56326 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56336 - "GET / HTTP/1.0" 200 OK 2026-03-26 09:31:22,097 - lava_callback - INFO - Received callback for node 69c4fc5ff63ce05331e2b20c INFO: 172.21.0.1:41740 - "POST /node/69c4fc5ff63ce05331e2b20c HTTP/1.0" 202 Accepted 2026-03-26 09:31:22,104 - lava_callback - INFO - Starting async job submit for node 69c4fc5ff63ce05331e2b20c 2026-03-26 09:31:22,128 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:31:22,142 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4fc5ff63ce05331e2b20c HTTP/1.1" 200 None 2026-03-26 09:31:22,143 - lava_callback - DEBUG - Processing node 69c4fc5ff63ce05331e2b20c: baseline-arm64 2026-03-26 09:31:22,547 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 09:31:22,714 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 09:31:22,715 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c4fc5ff63ce05331e2b20c/log.txt.gz 2026-03-26 09:31:22,946 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 09:31:23,320 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 09:31:23,322 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c4fc5ff63ce05331e2b20c/lava-logs.yaml 2026-03-26 09:31:23,331 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 09:31:23,523 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 09:31:23,525 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c4fc5ff63ce05331e2b20c/lava_callback.json.gz 2026-03-26 09:31:23,574 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:31:23,589 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4fc5ff63ce05331e2b20c HTTP/1.1" 200 None 2026-03-26 09:31:23,590 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:31:23,604 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebd8f63ce05331e2a492 HTTP/1.1" 200 None 2026-03-26 09:31:23,605 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:31:23,632 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4fc5ff63ce05331e2b20c HTTP/1.1" 200 None 2026-03-26 09:31:23,693 - lava_callback - INFO - Completed processing callback for node 69c4fc5ff63ce05331e2b20c INFO: 172.21.0.1:41744 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55570 - "GET / HTTP/1.0" 200 OK 2026-03-26 09:31:46,074 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:31:46,090 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:50768 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35782 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35790 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43666 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47354 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47368 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50444 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41218 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40752 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44830 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44834 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56090 - "GET / HTTP/1.0" 200 OK 2026-03-26 09:33:13,574 - lava_callback - INFO - Received callback for node 69c4fd00f63ce05331e2b225 INFO: 172.21.0.1:36772 - "POST /node/69c4fd00f63ce05331e2b225 HTTP/1.0" 202 Accepted 2026-03-26 09:33:13,580 - lava_callback - INFO - Starting async job submit for node 69c4fd00f63ce05331e2b225 2026-03-26 09:33:13,601 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:33:13,616 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4fd00f63ce05331e2b225 HTTP/1.1" 200 None 2026-03-26 09:33:13,618 - lava_callback - DEBUG - Processing node 69c4fd00f63ce05331e2b225: baseline-arm64 2026-03-26 09:33:14,007 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 09:33:14,147 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 09:33:14,149 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c4fd00f63ce05331e2b225/log.txt.gz 2026-03-26 09:33:14,327 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 09:33:14,736 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 09:33:14,739 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c4fd00f63ce05331e2b225/lava-logs.yaml 2026-03-26 09:33:14,749 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 09:33:14,897 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 09:33:14,899 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c4fd00f63ce05331e2b225/lava_callback.json.gz 2026-03-26 09:33:14,921 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:33:14,939 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4fd00f63ce05331e2b225 HTTP/1.1" 200 None 2026-03-26 09:33:14,940 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:33:14,956 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebd8f63ce05331e2a492 HTTP/1.1" 200 None 2026-03-26 09:33:14,958 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:33:14,996 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4fd00f63ce05331e2b225 HTTP/1.1" 200 None 2026-03-26 09:33:15,005 - lava_callback - INFO - Completed processing callback for node 69c4fd00f63ce05331e2b225 INFO: 172.21.0.1:36780 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-26 09:33:16,092 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:33:16,108 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:36796 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58336 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54562 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33506 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49030 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49036 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38706 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49808 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49820 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32946 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51190 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51408 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46454 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46458 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51608 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40768 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40776 - "GET / HTTP/1.0" 200 OK 2026-03-26 09:35:24,098 - lava_callback - INFO - Received callback for node 69c4fcacf63ce05331e2b216 INFO: 172.21.0.1:56832 - "POST /node/69c4fcacf63ce05331e2b216 HTTP/1.0" 202 Accepted 2026-03-26 09:35:24,104 - lava_callback - INFO - Starting async job submit for node 69c4fcacf63ce05331e2b216 2026-03-26 09:35:24,135 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:35:24,150 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4fcacf63ce05331e2b216 HTTP/1.1" 200 None 2026-03-26 09:35:24,151 - lava_callback - DEBUG - Processing node 69c4fcacf63ce05331e2b216: baseline-arm64 2026-03-26 09:35:24,319 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 09:35:24,483 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 09:35:24,485 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c4fcacf63ce05331e2b216/log.txt.gz 2026-03-26 09:35:24,573 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 09:35:24,757 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 09:35:24,759 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c4fcacf63ce05331e2b216/lava-logs.yaml 2026-03-26 09:35:24,763 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 09:35:24,901 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 09:35:24,903 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c4fcacf63ce05331e2b216/lava_callback.json.gz 2026-03-26 09:35:24,933 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:35:24,948 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4fcacf63ce05331e2b216 HTTP/1.1" 200 None 2026-03-26 09:35:24,949 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:35:24,962 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebd8f63ce05331e2a492 HTTP/1.1" 200 None 2026-03-26 09:35:24,964 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:35:24,985 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4fcacf63ce05331e2b216 HTTP/1.1" 200 None 2026-03-26 09:35:25,049 - lava_callback - INFO - Completed processing callback for node 69c4fcacf63ce05331e2b216 INFO: 172.21.0.1:56844 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41356 - "GET / HTTP/1.0" 200 OK 2026-03-26 09:35:41,369 - lava_callback - INFO - Received callback for node 69c25ad74a2c5046eb880695 INFO: 172.21.0.1:58036 - "POST /node/69c25ad74a2c5046eb880695 HTTP/1.0" 202 Accepted 2026-03-26 09:35:41,374 - lava_callback - INFO - Starting async job submit for node 69c25ad74a2c5046eb880695 2026-03-26 09:35:41,375 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:35:41,389 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c25ad74a2c5046eb880695 HTTP/1.1" 200 None 2026-03-26 09:35:41,390 - lava_callback - DEBUG - Processing node 69c25ad74a2c5046eb880695: baseline-arm64-clabbe 2026-03-26 09:35:41,390 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c25ad74a2c5046eb880695 2026-03-26 09:35:41,391 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:35:41,404 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c25ad74a2c5046eb880695 HTTP/1.1" 200 None 2026-03-26 09:35:41,406 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:35:41,419 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd027905e4bbd6ba634c66 HTTP/1.1" 200 None 2026-03-26 09:35:41,421 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:35:41,447 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c25ad74a2c5046eb880695 HTTP/1.1" 200 None 2026-03-26 09:35:41,452 - lava_callback - ERROR - Error processing callback for node 69c25ad74a2c5046eb880695: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-26 09:35:46,110 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:35:46,126 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:58052 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58238 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58242 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49426 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39990 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39996 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43980 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45570 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32846 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33818 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33828 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33022 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45798 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45804 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49518 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36830 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34138 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50726 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50732 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49300 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55042 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55056 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49056 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36922 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39128 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33136 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33144 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55484 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42174 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42190 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36716 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33590 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55694 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38288 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38300 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38268 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36880 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36894 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53278 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54034 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43214 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38682 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38684 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59934 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45680 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45684 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60300 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55976 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49476 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33388 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34078 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34092 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51742 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51758 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36214 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58030 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37852 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55792 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54402 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54404 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40490 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40492 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46170 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57940 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60436 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39120 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42536 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42544 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35818 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35822 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56238 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49934 - "GET / HTTP/1.0" 200 OK 2026-03-26 09:44:40,779 - lava_callback - INFO - Received callback for node 69c4fbecf63ce05331e2b1f5 INFO: 172.21.0.1:58044 - "POST /node/69c4fbecf63ce05331e2b1f5 HTTP/1.0" 202 Accepted 2026-03-26 09:44:40,787 - lava_callback - INFO - Starting async job submit for node 69c4fbecf63ce05331e2b1f5 2026-03-26 09:44:40,842 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:44:40,856 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4fbecf63ce05331e2b1f5 HTTP/1.1" 200 None 2026-03-26 09:44:40,857 - lava_callback - DEBUG - Processing node 69c4fbecf63ce05331e2b1f5: baseline-arm 2026-03-26 09:44:41,501 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 09:44:41,807 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 09:44:41,809 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69c4fbecf63ce05331e2b1f5/log.txt.gz 2026-03-26 09:44:42,164 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 09:44:42,461 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 09:44:42,463 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c4fbecf63ce05331e2b1f5/lava-logs.yaml 2026-03-26 09:44:42,477 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 09:44:42,669 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 09:44:42,671 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c4fbecf63ce05331e2b1f5/lava_callback.json.gz DEBUG: Setup failed for 69c4fbecf63ce05331e2b1f5. Transitting job node result: incomplete -> fail 2026-03-26 09:44:42,782 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:44:42,803 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4fbecf63ce05331e2b1f5 HTTP/1.1" 200 None 2026-03-26 09:44:42,805 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:44:42,820 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebcbf63ce05331e2a47f HTTP/1.1" 200 None 2026-03-26 09:44:42,822 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:44:42,849 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4fbecf63ce05331e2b1f5 HTTP/1.1" 200 None 2026-03-26 09:44:42,955 - lava_callback - INFO - Completed processing callback for node 69c4fbecf63ce05331e2b1f5 2026-03-26 09:44:46,130 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:44:46,146 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:58060 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35716 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56194 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56206 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60458 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60470 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58266 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45510 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48142 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49024 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49252 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49258 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54144 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54148 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46490 - "GET / HTTP/1.0" 200 OK 2026-03-26 09:46:30,068 - lava_callback - INFO - Received callback for node 69c4ff1cf63ce05331e2b27a INFO: 172.21.0.1:38644 - "POST /node/69c4ff1cf63ce05331e2b27a HTTP/1.0" 202 Accepted 2026-03-26 09:46:30,075 - lava_callback - INFO - Starting async job submit for node 69c4ff1cf63ce05331e2b27a 2026-03-26 09:46:30,101 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:46:30,116 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ff1cf63ce05331e2b27a HTTP/1.1" 200 None 2026-03-26 09:46:30,116 - lava_callback - DEBUG - Processing node 69c4ff1cf63ce05331e2b27a: ltp-fs_bind 2026-03-26 09:46:30,481 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 09:46:30,784 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 09:46:30,786 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-fs_bind-69c4ff1cf63ce05331e2b27a/log.txt.gz 2026-03-26 09:46:31,030 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 09:46:31,260 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 09:46:31,262 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-fs_bind-69c4ff1cf63ce05331e2b27a/lava-logs.yaml 2026-03-26 09:46:31,271 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 09:46:31,501 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 09:46:31,503 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-fs_bind-69c4ff1cf63ce05331e2b27a/lava_callback.json.gz 2026-03-26 09:46:31,544 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:46:31,585 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ff1cf63ce05331e2b27a HTTP/1.1" 200 None 2026-03-26 09:46:31,587 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:46:31,606 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebcbf63ce05331e2a47f HTTP/1.1" 200 None 2026-03-26 09:46:31,608 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:46:31,637 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4ff1cf63ce05331e2b27a HTTP/1.1" 200 None 2026-03-26 09:46:31,645 - lava_callback - INFO - Completed processing callback for node 69c4ff1cf63ce05331e2b27a 2026-03-26 09:46:33,663 - lava_callback - INFO - Received callback for node 69c4feb4f63ce05331e2b26d INFO: 172.21.0.1:38646 - "POST /node/69c4feb4f63ce05331e2b26d HTTP/1.0" 202 Accepted 2026-03-26 09:46:33,670 - lava_callback - INFO - Starting async job submit for node 69c4feb4f63ce05331e2b26d 2026-03-26 09:46:33,706 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:46:33,721 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4feb4f63ce05331e2b26d HTTP/1.1" 200 None 2026-03-26 09:46:33,722 - lava_callback - DEBUG - Processing node 69c4feb4f63ce05331e2b26d: baseline-arm64 2026-03-26 09:46:34,114 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 09:46:34,237 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 09:46:34,238 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c4feb4f63ce05331e2b26d/log.txt.gz 2026-03-26 09:46:34,447 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 09:46:34,664 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 09:46:34,666 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c4feb4f63ce05331e2b26d/lava-logs.yaml 2026-03-26 09:46:34,675 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 09:46:34,869 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 09:46:34,870 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c4feb4f63ce05331e2b26d/lava_callback.json.gz DEBUG: Setup failed for 69c4feb4f63ce05331e2b26d. Transitting job node result: incomplete -> fail 2026-03-26 09:46:34,944 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:46:34,961 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4feb4f63ce05331e2b26d HTTP/1.1" 200 None 2026-03-26 09:46:34,963 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:46:34,977 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebd8f63ce05331e2a492 HTTP/1.1" 200 None 2026-03-26 09:46:34,979 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:46:35,004 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4feb4f63ce05331e2b26d HTTP/1.1" 200 None 2026-03-26 09:46:35,088 - lava_callback - INFO - Completed processing callback for node 69c4feb4f63ce05331e2b26d INFO: 172.21.0.1:38648 - "GET / HTTP/1.0" 200 OK 2026-03-26 09:46:46,149 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 09:46:46,168 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:45772 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46032 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56498 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56504 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46548 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46564 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56522 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53546 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55994 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36636 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52710 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52716 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36850 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36852 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42518 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60856 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59862 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51842 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56092 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56096 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57244 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57248 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59180 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45856 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56010 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33316 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39528 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39540 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43594 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43602 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42040 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57886 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39534 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43272 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55676 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55688 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50278 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50284 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47746 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44870 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49406 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50722 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50040 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50046 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53428 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53430 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33904 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48140 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38916 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56002 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50792 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50796 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47662 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47668 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38056 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49290 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34214 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56040 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57438 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57440 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39122 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39126 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60162 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47720 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54190 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46040 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43480 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43496 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58396 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58402 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35578 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55784 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58120 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53238 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40500 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40508 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43972 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43980 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40652 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34806 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37168 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54188 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59672 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59678 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51738 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51740 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40570 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43440 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48276 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33444 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39282 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39288 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49546 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49562 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33992 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42952 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55522 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48158 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58758 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58766 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56550 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56560 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42870 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57856 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48094 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56372 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53266 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53272 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43668 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43680 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38708 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57546 - "GET / HTTP/1.0" 200 OK 2026-03-26 10:00:45,854 - lava_callback - INFO - Received callback for node 69c503bdf63ce05331e2b33c INFO: 172.21.0.1:54600 - "POST /node/69c503bdf63ce05331e2b33c HTTP/1.0" 202 Accepted 2026-03-26 10:00:45,860 - lava_callback - INFO - Starting async job submit for node 69c503bdf63ce05331e2b33c 2026-03-26 10:00:45,862 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:00:45,876 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c503bdf63ce05331e2b33c HTTP/1.1" 200 None 2026-03-26 10:00:45,877 - lava_callback - DEBUG - Processing node 69c503bdf63ce05331e2b33c: baseline-nfs-arm64 2026-03-26 10:00:45,878 - lava_callback - WARNING - Failed to upload log 2026-03-26 10:00:45,880 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 10:00:46,024 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 10:00:46,026 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c503bdf63ce05331e2b33c/lava-logs.yaml 2026-03-26 10:00:46,027 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 10:00:46,200 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 10:00:46,202 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c503bdf63ce05331e2b33c/lava_callback.json.gz 2026-03-26 10:00:46,204 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:00:46,217 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c503bdf63ce05331e2b33c HTTP/1.1" 200 None 2026-03-26 10:00:46,219 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:00:46,239 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebd8f63ce05331e2a492 HTTP/1.1" 200 None 2026-03-26 10:00:46,240 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:00:46,261 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c503bdf63ce05331e2b33c HTTP/1.1" 200 None 2026-03-26 10:00:46,269 - lava_callback - INFO - Completed processing callback for node 69c503bdf63ce05331e2b33c INFO: 172.21.0.1:54604 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53778 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55378 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55386 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40124 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-26 10:01:16,173 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:01:16,191 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:40136 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40440 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50050 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54324 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53354 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40962 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40974 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40020 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40036 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50754 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49466 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45120 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41748 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39268 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39280 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56018 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56026 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41650 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59050 - "GET / HTTP/1.0" 200 OK 2026-03-26 10:03:46,160 - lava_callback - INFO - Received callback for node 69c50450f63ce05331e2b35b INFO: 172.21.0.1:40764 - "POST /node/69c50450f63ce05331e2b35b HTTP/1.0" 202 Accepted 2026-03-26 10:03:46,166 - lava_callback - INFO - Starting async job submit for node 69c50450f63ce05331e2b35b 2026-03-26 10:03:46,196 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:03:46,211 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c50450f63ce05331e2b35b HTTP/1.1" 200 None 2026-03-26 10:03:46,212 - lava_callback - DEBUG - Processing node 69c50450f63ce05331e2b35b: baseline-arm64 2026-03-26 10:03:46,378 - lava_callback - WARNING - Failed to upload log 2026-03-26 10:03:46,449 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.21.0.1:40778 - "GET / HTTP/1.0" 200 OK 2026-03-26 10:03:46,650 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 10:03:46,651 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c50450f63ce05331e2b35b/lava-logs.yaml 2026-03-26 10:03:46,655 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 10:03:46,800 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 10:03:46,802 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c50450f63ce05331e2b35b/lava_callback.json.gz 2026-03-26 10:03:46,833 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:03:46,848 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c50450f63ce05331e2b35b HTTP/1.1" 200 None 2026-03-26 10:03:46,850 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:03:46,864 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebd8f63ce05331e2a492 HTTP/1.1" 200 None 2026-03-26 10:03:46,866 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:03:46,901 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c50450f63ce05331e2b35b HTTP/1.1" 200 None 2026-03-26 10:03:46,984 - lava_callback - INFO - Completed processing callback for node 69c50450f63ce05331e2b35b INFO: 172.21.0.1:44184 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37828 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37832 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59398 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-26 10:04:16,197 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:04:16,224 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:59414 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57318 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37362 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53922 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36692 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53650 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53658 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39154 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39164 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43276 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49734 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36974 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56624 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56344 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56348 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33590 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33594 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44984 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43668 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33436 - "GET / HTTP/1.0" 200 OK 2026-03-26 10:06:52,663 - lava_callback - INFO - Received callback for node 69c504eff63ce05331e2b36e INFO: 172.21.0.1:36986 - "POST /node/69c504eff63ce05331e2b36e HTTP/1.0" 202 Accepted 2026-03-26 10:06:52,670 - lava_callback - INFO - Starting async job submit for node 69c504eff63ce05331e2b36e 2026-03-26 10:06:52,692 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:06:52,706 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c504eff63ce05331e2b36e HTTP/1.1" 200 None 2026-03-26 10:06:52,707 - lava_callback - DEBUG - Processing node 69c504eff63ce05331e2b36e: baseline-arm64 2026-03-26 10:06:53,131 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 10:06:53,292 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 10:06:53,294 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c504eff63ce05331e2b36e/log.txt.gz 2026-03-26 10:06:53,510 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 10:06:53,720 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 10:06:53,722 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c504eff63ce05331e2b36e/lava-logs.yaml 2026-03-26 10:06:53,730 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 10:06:53,900 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 10:06:53,902 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c504eff63ce05331e2b36e/lava_callback.json.gz 2026-03-26 10:06:53,952 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:06:53,967 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c504eff63ce05331e2b36e HTTP/1.1" 200 None 2026-03-26 10:06:53,969 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:06:53,983 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebd8f63ce05331e2a492 HTTP/1.1" 200 None 2026-03-26 10:06:53,984 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:06:54,012 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c504eff63ce05331e2b36e HTTP/1.1" 200 None 2026-03-26 10:06:54,066 - lava_callback - INFO - Completed processing callback for node 69c504eff63ce05331e2b36e INFO: 172.21.0.1:36992 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40678 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40684 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34740 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-26 10:07:16,227 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:07:16,242 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:34746 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41770 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60090 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45704 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32998 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49424 - "GET / HTTP/1.0" 200 OK 2026-03-26 10:08:05,274 - lava_callback - INFO - Received callback for node 69c4fda3f63ce05331e2b245 INFO: 172.21.0.1:49426 - "POST /node/69c4fda3f63ce05331e2b245 HTTP/1.0" 202 Accepted 2026-03-26 10:08:05,321 - lava_callback - INFO - Starting async job submit for node 69c4fda3f63ce05331e2b245 2026-03-26 10:08:05,345 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:08:05,360 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4fda3f63ce05331e2b245 HTTP/1.1" 200 None 2026-03-26 10:08:05,360 - lava_callback - DEBUG - Processing node 69c4fda3f63ce05331e2b245: baseline-arm-pengutronix 2026-03-26 10:08:05,485 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 10:08:05,702 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 10:08:05,704 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c4fda3f63ce05331e2b245/log.txt.gz 2026-03-26 10:08:05,763 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 10:08:05,962 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 10:08:05,964 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c4fda3f63ce05331e2b245/lava-logs.yaml 2026-03-26 10:08:05,967 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 10:08:06,104 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 10:08:06,105 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c4fda3f63ce05331e2b245/lava_callback.json.gz DEBUG: Setup failed for 69c4fda3f63ce05331e2b245. Transitting job node result: incomplete -> fail 2026-03-26 10:08:06,154 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:08:06,169 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4fda3f63ce05331e2b245 HTTP/1.1" 200 None 2026-03-26 10:08:06,170 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:08:06,184 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebcbf63ce05331e2a47f HTTP/1.1" 200 None 2026-03-26 10:08:06,185 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:08:06,209 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4fda3f63ce05331e2b245 HTTP/1.1" 200 None 2026-03-26 10:08:06,266 - lava_callback - INFO - Completed processing callback for node 69c4fda3f63ce05331e2b245 INFO: 172.21.0.1:49438 - "GET / HTTP/1.0" 200 OK 2026-03-26 10:08:12,502 - lava_callback - INFO - Received callback for node 69c5053cf63ce05331e2b375 INFO: 172.21.0.1:45152 - "POST /node/69c5053cf63ce05331e2b375 HTTP/1.0" 202 Accepted 2026-03-26 10:08:12,509 - lava_callback - INFO - Starting async job submit for node 69c5053cf63ce05331e2b375 2026-03-26 10:08:12,539 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:08:12,553 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5053cf63ce05331e2b375 HTTP/1.1" 200 None 2026-03-26 10:08:12,554 - lava_callback - DEBUG - Processing node 69c5053cf63ce05331e2b375: baseline-arm64 2026-03-26 10:08:12,766 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 10:08:12,885 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 10:08:12,887 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c5053cf63ce05331e2b375/log.txt.gz 2026-03-26 10:08:12,990 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 10:08:13,222 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 10:08:13,224 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c5053cf63ce05331e2b375/lava-logs.yaml 2026-03-26 10:08:13,228 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 10:08:13,511 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 10:08:13,512 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c5053cf63ce05331e2b375/lava_callback.json.gz 2026-03-26 10:08:13,543 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:08:13,558 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5053cf63ce05331e2b375 HTTP/1.1" 200 None 2026-03-26 10:08:13,559 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:08:13,574 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebd8f63ce05331e2a492 HTTP/1.1" 200 None 2026-03-26 10:08:13,575 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:08:13,596 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5053cf63ce05331e2b375 HTTP/1.1" 200 None 2026-03-26 10:08:13,656 - lava_callback - INFO - Completed processing callback for node 69c5053cf63ce05331e2b375 INFO: 172.21.0.1:45166 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-26 10:08:16,245 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:08:16,261 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:45172 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35534 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51218 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36088 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36742 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47524 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47536 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41064 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41074 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44794 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42130 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60232 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50236 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55064 - "GET / HTTP/1.0" 200 OK 2026-03-26 10:10:00,381 - lava_callback - INFO - Received callback for node 69c501c5f63ce05331e2b2de INFO: 172.21.0.1:55076 - "POST /node/69c501c5f63ce05331e2b2de HTTP/1.0" 202 Accepted 2026-03-26 10:10:00,388 - lava_callback - INFO - Starting async job submit for node 69c501c5f63ce05331e2b2de 2026-03-26 10:10:00,447 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:10:00,462 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c501c5f63ce05331e2b2de HTTP/1.1" 200 None 2026-03-26 10:10:00,463 - lava_callback - DEBUG - Processing node 69c501c5f63ce05331e2b2de: ltp-smoke 2026-03-26 10:10:01,061 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 10:10:01,213 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 10:10:01,214 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoke-69c501c5f63ce05331e2b2de/log.txt.gz 2026-03-26 10:10:01,497 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 10:10:01,738 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 10:10:01,740 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-smoke-69c501c5f63ce05331e2b2de/lava-logs.yaml 2026-03-26 10:10:01,752 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 10:10:01,926 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 10:10:01,928 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoke-69c501c5f63ce05331e2b2de/lava_callback.json.gz 2026-03-26 10:10:02,048 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:10:02,068 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c501c5f63ce05331e2b2de HTTP/1.1" 200 None 2026-03-26 10:10:02,069 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:10:02,086 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebd0f63ce05331e2a489 HTTP/1.1" 200 None 2026-03-26 10:10:02,088 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:10:02,112 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c501c5f63ce05331e2b2de HTTP/1.1" 200 None 2026-03-26 10:10:02,246 - lava_callback - INFO - Completed processing callback for node 69c501c5f63ce05331e2b2de 2026-03-26 10:10:02,249 - lava_callback - INFO - Received callback for node 69c50159f63ce05331e2b2cb INFO: 172.21.0.1:55090 - "POST /node/69c50159f63ce05331e2b2cb HTTP/1.0" 202 Accepted 2026-03-26 10:10:02,262 - lava_callback - INFO - Starting async job submit for node 69c50159f63ce05331e2b2cb 2026-03-26 10:10:02,324 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:10:02,370 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c50159f63ce05331e2b2cb HTTP/1.1" 200 None 2026-03-26 10:10:02,371 - lava_callback - DEBUG - Processing node 69c50159f63ce05331e2b2cb: baseline-nfs-arm64 2026-03-26 10:10:03,431 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 10:10:04,621 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 10:10:04,623 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c50159f63ce05331e2b2cb/log.txt.gz 2026-03-26 10:10:05,236 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 10:10:05,500 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 10:10:05,503 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c50159f63ce05331e2b2cb/lava-logs.yaml 2026-03-26 10:10:05,524 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 10:10:05,712 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 10:10:05,714 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c50159f63ce05331e2b2cb/lava_callback.json.gz DEBUG: Setup failed for 69c50159f63ce05331e2b2cb. Transitting job node result: incomplete -> fail 2026-03-26 10:10:05,802 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:10:05,820 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c50159f63ce05331e2b2cb HTTP/1.1" 200 None 2026-03-26 10:10:05,822 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:10:05,838 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebd8f63ce05331e2a492 HTTP/1.1" 200 None 2026-03-26 10:10:05,840 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:10:05,865 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c50159f63ce05331e2b2cb HTTP/1.1" 200 None 2026-03-26 10:10:05,998 - lava_callback - INFO - Completed processing callback for node 69c50159f63ce05331e2b2cb INFO: 172.21.0.1:55102 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42268 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-26 10:10:16,264 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:10:16,281 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:42270 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60448 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54938 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51040 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43452 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40756 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40770 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38710 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38726 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46238 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39408 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59192 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32834 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50284 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50292 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34694 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34708 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59398 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55402 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37098 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58784 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40474 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40490 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39014 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39016 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34708 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46598 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33468 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56488 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56852 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56860 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43830 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43834 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32862 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60122 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38960 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56746 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53562 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53568 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52520 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52522 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40386 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52314 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34762 - "GET / HTTP/1.0" 200 OK 2026-03-26 10:15:54,019 - lava_callback - INFO - Received callback for node 69c5073ef63ce05331e2b3d2 INFO: 172.21.0.1:47776 - "POST /node/69c5073ef63ce05331e2b3d2 HTTP/1.0" 202 Accepted 2026-03-26 10:15:54,025 - lava_callback - INFO - Starting async job submit for node 69c5073ef63ce05331e2b3d2 2026-03-26 10:15:54,027 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:15:54,045 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5073ef63ce05331e2b3d2 HTTP/1.1" 200 None 2026-03-26 10:15:54,046 - lava_callback - DEBUG - Processing node 69c5073ef63ce05331e2b3d2: baseline-nfs-arm64 2026-03-26 10:15:54,047 - lava_callback - WARNING - Failed to upload log 2026-03-26 10:15:54,048 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 10:15:54,210 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 10:15:54,212 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c5073ef63ce05331e2b3d2/lava-logs.yaml 2026-03-26 10:15:54,213 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 10:15:54,351 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 10:15:54,353 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c5073ef63ce05331e2b3d2/lava_callback.json.gz 2026-03-26 10:15:54,355 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:15:54,369 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5073ef63ce05331e2b3d2 HTTP/1.1" 200 None 2026-03-26 10:15:54,371 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:15:54,384 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebd8f63ce05331e2a492 HTTP/1.1" 200 None 2026-03-26 10:15:54,386 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:15:54,423 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5073ef63ce05331e2b3d2 HTTP/1.1" 200 None 2026-03-26 10:15:54,427 - lava_callback - INFO - Completed processing callback for node 69c5073ef63ce05331e2b3d2 INFO: 172.21.0.1:47778 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58046 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58054 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44088 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-26 10:16:16,284 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:16:16,304 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:44098 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51576 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52596 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58338 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53682 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42766 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42770 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49230 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49240 - "GET / HTTP/1.0" 200 OK 2026-03-26 10:17:23,142 - lava_callback - INFO - Received callback for node 69c50784f63ce05331e2b3d9 INFO: 172.21.0.1:36138 - "POST /node/69c50784f63ce05331e2b3d9 HTTP/1.0" 202 Accepted 2026-03-26 10:17:23,149 - lava_callback - INFO - Starting async job submit for node 69c50784f63ce05331e2b3d9 2026-03-26 10:17:23,178 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:17:23,192 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c50784f63ce05331e2b3d9 HTTP/1.1" 200 None 2026-03-26 10:17:23,193 - lava_callback - DEBUG - Processing node 69c50784f63ce05331e2b3d9: baseline-arm64 2026-03-26 10:17:23,329 - lava_callback - WARNING - Failed to upload log 2026-03-26 10:17:23,399 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 10:17:23,624 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 10:17:23,626 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c50784f63ce05331e2b3d9/lava-logs.yaml 2026-03-26 10:17:23,629 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 10:17:23,829 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 10:17:23,830 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c50784f63ce05331e2b3d9/lava_callback.json.gz 2026-03-26 10:17:23,860 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:17:23,876 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c50784f63ce05331e2b3d9 HTTP/1.1" 200 None 2026-03-26 10:17:23,877 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:17:23,891 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebd8f63ce05331e2a492 HTTP/1.1" 200 None 2026-03-26 10:17:23,893 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:17:23,915 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c50784f63ce05331e2b3d9 HTTP/1.1" 200 None 2026-03-26 10:17:23,992 - lava_callback - INFO - Completed processing callback for node 69c50784f63ce05331e2b3d9 INFO: 172.21.0.1:36150 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57816 - "GET / HTTP/1.0" 200 OK 2026-03-26 10:17:46,306 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:17:46,334 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:54094 - "GET / HTTP/1.0" 200 OK 2026-03-26 10:17:46,925 - lava_callback - INFO - Received callback for node 69c50601f63ce05331e2b3ab INFO: 172.21.0.1:54100 - "POST /node/69c50601f63ce05331e2b3ab HTTP/1.0" 202 Accepted 2026-03-26 10:17:46,932 - lava_callback - INFO - Starting async job submit for node 69c50601f63ce05331e2b3ab 2026-03-26 10:17:46,968 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:17:46,983 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c50601f63ce05331e2b3ab HTTP/1.1" 200 None 2026-03-26 10:17:46,984 - lava_callback - DEBUG - Processing node 69c50601f63ce05331e2b3ab: baseline-arm64 2026-03-26 10:17:47,368 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 10:17:47,677 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 10:17:47,679 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c50601f63ce05331e2b3ab/log.txt.gz 2026-03-26 10:17:47,887 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 10:17:48,084 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 10:17:48,086 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c50601f63ce05331e2b3ab/lava-logs.yaml 2026-03-26 10:17:48,095 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 10:17:48,264 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 10:17:48,266 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c50601f63ce05331e2b3ab/lava_callback.json.gz DEBUG: Setup failed for 69c50601f63ce05331e2b3ab. Transitting job node result: incomplete -> fail 2026-03-26 10:17:48,339 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:17:48,353 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c50601f63ce05331e2b3ab HTTP/1.1" 200 None 2026-03-26 10:17:48,354 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:17:48,368 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebd8f63ce05331e2a492 HTTP/1.1" 200 None 2026-03-26 10:17:48,369 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:17:48,408 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c50601f63ce05331e2b3ab HTTP/1.1" 200 None 2026-03-26 10:17:48,496 - lava_callback - INFO - Completed processing callback for node 69c50601f63ce05331e2b3ab INFO: 172.21.0.1:53582 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40998 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41010 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54484 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-26 10:18:16,336 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:18:16,354 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:54498 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33808 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35166 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48484 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55402 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58776 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58790 - "GET / HTTP/1.0" 200 OK 2026-03-26 10:19:07,342 - lava_callback - INFO - Received callback for node 69c507d6f63ce05331e2b3e9 INFO: 172.21.0.1:58806 - "POST /node/69c507d6f63ce05331e2b3e9 HTTP/1.0" 202 Accepted 2026-03-26 10:19:07,348 - lava_callback - INFO - Starting async job submit for node 69c507d6f63ce05331e2b3e9 2026-03-26 10:19:07,371 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:19:07,385 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c507d6f63ce05331e2b3e9 HTTP/1.1" 200 None 2026-03-26 10:19:07,386 - lava_callback - DEBUG - Processing node 69c507d6f63ce05331e2b3e9: baseline-arm64 2026-03-26 10:19:07,756 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 10:19:07,898 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 10:19:07,900 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c507d6f63ce05331e2b3e9/log.txt.gz 2026-03-26 10:19:08,140 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 10:19:08,379 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 10:19:08,381 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c507d6f63ce05331e2b3e9/lava-logs.yaml 2026-03-26 10:19:08,389 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 10:19:08,541 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 10:19:08,543 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c507d6f63ce05331e2b3e9/lava_callback.json.gz 2026-03-26 10:19:08,587 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:19:08,612 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c507d6f63ce05331e2b3e9 HTTP/1.1" 200 None 2026-03-26 10:19:08,613 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:19:08,627 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebd8f63ce05331e2a492 HTTP/1.1" 200 None 2026-03-26 10:19:08,628 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:19:08,653 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c507d6f63ce05331e2b3e9 HTTP/1.1" 200 None 2026-03-26 10:19:08,713 - lava_callback - INFO - Completed processing callback for node 69c507d6f63ce05331e2b3e9 INFO: 172.21.0.1:45034 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-26 10:19:16,357 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:19:16,372 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:45038 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34508 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51366 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36460 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40360 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46372 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46386 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53926 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53936 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54614 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56994 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44814 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42748 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34438 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34448 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42168 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42172 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52612 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58238 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55612 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55608 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57154 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57162 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33672 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33688 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53908 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53998 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45316 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38772 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55892 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55894 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60618 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60620 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55036 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47422 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41084 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58702 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41270 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41286 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35888 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35892 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41650 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51338 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42542 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57210 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44932 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44944 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42380 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42384 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37572 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33862 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52424 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47286 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43816 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43824 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59450 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59454 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60904 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57758 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60858 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47522 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54888 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54894 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57652 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57658 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39990 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37808 - "GET / HTTP/1.0" 200 OK 2026-03-26 10:27:37,497 - lava_callback - INFO - Received callback for node 69c505d7f63ce05331e2b397 INFO: 172.21.0.1:37820 - "POST /node/69c505d7f63ce05331e2b397 HTTP/1.0" 202 Accepted 2026-03-26 10:27:37,504 - lava_callback - INFO - Starting async job submit for node 69c505d7f63ce05331e2b397 2026-03-26 10:27:37,558 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:27:37,599 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c505d7f63ce05331e2b397 HTTP/1.1" 200 None 2026-03-26 10:27:37,600 - lava_callback - DEBUG - Processing node 69c505d7f63ce05331e2b397: baseline-arm 2026-03-26 10:27:38,184 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 10:27:38,332 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 10:27:38,334 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69c505d7f63ce05331e2b397/log.txt.gz 2026-03-26 10:27:38,710 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 10:27:38,983 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 10:27:38,985 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c505d7f63ce05331e2b397/lava-logs.yaml 2026-03-26 10:27:39,000 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 10:27:39,176 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 10:27:39,178 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c505d7f63ce05331e2b397/lava_callback.json.gz DEBUG: Setup failed for 69c505d7f63ce05331e2b397. Transitting job node result: incomplete -> fail 2026-03-26 10:27:39,286 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:27:39,322 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c505d7f63ce05331e2b397 HTTP/1.1" 200 None 2026-03-26 10:27:39,324 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:27:39,338 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebcbf63ce05331e2a47f HTTP/1.1" 200 None 2026-03-26 10:27:39,340 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:27:39,372 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c505d7f63ce05331e2b397 HTTP/1.1" 200 None 2026-03-26 10:27:39,505 - lava_callback - INFO - Completed processing callback for node 69c505d7f63ce05331e2b397 2026-03-26 10:27:46,376 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:27:46,392 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:56408 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55954 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44976 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44988 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54444 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54448 - "GET / HTTP/1.0" 200 OK 2026-03-26 10:28:19,961 - lava_callback - INFO - Received callback for node 69c508f6f63ce05331e2b417 INFO: 172.21.0.1:37958 - "POST /node/69c508f6f63ce05331e2b417 HTTP/1.0" 202 Accepted 2026-03-26 10:28:19,969 - lava_callback - INFO - Starting async job submit for node 69c508f6f63ce05331e2b417 2026-03-26 10:28:20,005 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:28:20,020 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c508f6f63ce05331e2b417 HTTP/1.1" 200 None 2026-03-26 10:28:20,021 - lava_callback - DEBUG - Processing node 69c508f6f63ce05331e2b417: baseline-arm64 2026-03-26 10:28:20,397 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 10:28:20,514 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 10:28:20,516 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c508f6f63ce05331e2b417/log.txt.gz 2026-03-26 10:28:20,741 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 10:28:20,936 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 10:28:20,938 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c508f6f63ce05331e2b417/lava-logs.yaml 2026-03-26 10:28:20,946 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 10:28:21,128 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 10:28:21,130 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c508f6f63ce05331e2b417/lava_callback.json.gz DEBUG: Setup failed for 69c508f6f63ce05331e2b417. Transitting job node result: incomplete -> fail 2026-03-26 10:28:21,200 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:28:21,215 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c508f6f63ce05331e2b417 HTTP/1.1" 200 None 2026-03-26 10:28:21,216 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:28:21,230 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebd8f63ce05331e2a492 HTTP/1.1" 200 None 2026-03-26 10:28:21,231 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:28:21,254 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c508f6f63ce05331e2b417 HTTP/1.1" 200 None 2026-03-26 10:28:21,320 - lava_callback - INFO - Completed processing callback for node 69c508f6f63ce05331e2b417 INFO: 172.21.0.1:37964 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47202 - "GET / HTTP/1.0" 200 OK 2026-03-26 10:28:42,611 - lava_callback - INFO - Received callback for node 69c5084ef63ce05331e2b404 INFO: 172.21.0.1:46452 - "POST /node/69c5084ef63ce05331e2b404 HTTP/1.0" 202 Accepted 2026-03-26 10:28:42,618 - lava_callback - INFO - Starting async job submit for node 69c5084ef63ce05331e2b404 2026-03-26 10:28:42,647 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:28:42,683 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5084ef63ce05331e2b404 HTTP/1.1" 200 None 2026-03-26 10:28:42,684 - lava_callback - DEBUG - Processing node 69c5084ef63ce05331e2b404: baseline-arm64 2026-03-26 10:28:42,824 - lava_callback - WARNING - Failed to upload log 2026-03-26 10:28:42,897 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 10:28:43,112 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 10:28:43,114 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c5084ef63ce05331e2b404/lava-logs.yaml 2026-03-26 10:28:43,117 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 10:28:43,250 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 10:28:43,252 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c5084ef63ce05331e2b404/lava_callback.json.gz 2026-03-26 10:28:43,283 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:28:43,320 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5084ef63ce05331e2b404 HTTP/1.1" 200 None 2026-03-26 10:28:43,322 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:28:43,336 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebd8f63ce05331e2a492 HTTP/1.1" 200 None 2026-03-26 10:28:43,338 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:28:43,359 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5084ef63ce05331e2b404 HTTP/1.1" 200 None 2026-03-26 10:28:43,434 - lava_callback - INFO - Completed processing callback for node 69c5084ef63ce05331e2b404 2026-03-26 10:28:46,394 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:28:46,412 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:46462 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44610 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44820 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44832 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42840 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42852 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50914 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56768 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38752 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48888 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44350 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44364 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36330 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36342 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38404 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53346 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39408 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32818 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38614 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38622 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51018 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51020 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37054 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57344 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39122 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52338 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59400 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59406 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46110 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46120 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54032 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43598 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38198 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40824 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36844 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36846 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40026 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40040 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55528 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55640 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48922 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32944 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34126 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34132 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36494 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36510 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55556 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41108 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41022 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43750 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51008 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51016 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53786 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53796 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44234 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57546 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34392 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44740 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48166 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48170 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57578 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57586 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57508 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41912 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46534 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43322 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44576 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44590 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41040 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41044 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60446 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37982 - "GET / HTTP/1.0" 200 OK 2026-03-26 10:37:41,609 - lava_callback - INFO - Received callback for node 69c3c3fcbbc022e792645221 INFO: 172.21.0.1:44886 - "POST /node/69c3c3fcbbc022e792645221 HTTP/1.0" 202 Accepted 2026-03-26 10:37:41,618 - lava_callback - INFO - Starting async job submit for node 69c3c3fcbbc022e792645221 2026-03-26 10:37:41,639 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:37:41,676 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3c3fcbbc022e792645221 HTTP/1.1" 200 None 2026-03-26 10:37:41,677 - lava_callback - DEBUG - Processing node 69c3c3fcbbc022e792645221: baseline-arm64-broonie 2026-03-26 10:37:41,999 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 10:37:42,147 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 10:37:42,149 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c3c3fcbbc022e792645221/log.txt.gz 2026-03-26 10:37:42,356 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 10:37:42,747 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 10:37:42,749 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c3c3fcbbc022e792645221/lava-logs.yaml 2026-03-26 10:37:42,757 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 10:37:43,060 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 10:37:43,062 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c3c3fcbbc022e792645221/lava_callback.json.gz 2026-03-26 10:37:43,080 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:37:43,095 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3c3fcbbc022e792645221 HTTP/1.1" 200 None 2026-03-26 10:37:43,096 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:37:43,111 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3295318a3717aa65d01c3 HTTP/1.1" 200 None 2026-03-26 10:37:43,112 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:37:43,151 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3c3fcbbc022e792645221 HTTP/1.1" 200 None 2026-03-26 10:37:43,153 - lava_callback - INFO - Completed processing callback for node 69c3c3fcbbc022e792645221 2026-03-26 10:37:46,416 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:37:46,473 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:44890 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34550 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57546 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57556 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45370 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45386 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33462 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36310 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42908 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42002 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54662 - "GET / HTTP/1.0" 200 OK 2026-03-26 10:39:01,932 - lava_callback - INFO - Received callback for node 69c5089ff63ce05331e2b40b INFO: 172.21.0.1:54676 - "POST /node/69c5089ff63ce05331e2b40b HTTP/1.0" 202 Accepted 2026-03-26 10:39:01,939 - lava_callback - INFO - Starting async job submit for node 69c5089ff63ce05331e2b40b 2026-03-26 10:39:01,998 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:39:02,035 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5089ff63ce05331e2b40b HTTP/1.1" 200 None 2026-03-26 10:39:02,036 - lava_callback - DEBUG - Processing node 69c5089ff63ce05331e2b40b: ltp-smoke 2026-03-26 10:39:02,494 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 10:39:02,669 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 10:39:02,670 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoke-69c5089ff63ce05331e2b40b/log.txt.gz 2026-03-26 10:39:02,945 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 10:39:03,333 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 10:39:03,335 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-smoke-69c5089ff63ce05331e2b40b/lava-logs.yaml 2026-03-26 10:39:03,346 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 10:39:03,492 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 10:39:03,494 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoke-69c5089ff63ce05331e2b40b/lava_callback.json.gz 2026-03-26 10:39:03,636 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:39:03,661 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5089ff63ce05331e2b40b HTTP/1.1" 200 None 2026-03-26 10:39:03,663 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:39:03,676 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebd0f63ce05331e2a489 HTTP/1.1" 200 None 2026-03-26 10:39:03,677 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:39:03,730 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5089ff63ce05331e2b40b HTTP/1.1" 200 None 2026-03-26 10:39:03,805 - lava_callback - INFO - Completed processing callback for node 69c5089ff63ce05331e2b40b INFO: 172.21.0.1:54682 - "GET / HTTP/1.0" 200 OK 2026-03-26 10:39:07,569 - lava_callback - INFO - Received callback for node 69c50831f63ce05331e2b3fd INFO: 172.21.0.1:54694 - "POST /node/69c50831f63ce05331e2b3fd HTTP/1.0" 202 Accepted 2026-03-26 10:39:07,617 - lava_callback - INFO - Starting async job submit for node 69c50831f63ce05331e2b3fd 2026-03-26 10:39:07,641 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:39:07,656 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c50831f63ce05331e2b3fd HTTP/1.1" 200 None 2026-03-26 10:39:07,657 - lava_callback - DEBUG - Processing node 69c50831f63ce05331e2b3fd: baseline-arm-pengutronix 2026-03-26 10:39:07,767 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 10:39:07,891 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 10:39:07,893 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c50831f63ce05331e2b3fd/log.txt.gz 2026-03-26 10:39:07,951 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 10:39:08,131 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 10:39:08,133 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c50831f63ce05331e2b3fd/lava-logs.yaml 2026-03-26 10:39:08,136 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 10:39:08,263 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 10:39:08,265 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c50831f63ce05331e2b3fd/lava_callback.json.gz DEBUG: Setup failed for 69c50831f63ce05331e2b3fd. Transitting job node result: incomplete -> fail 2026-03-26 10:39:08,314 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:39:08,330 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c50831f63ce05331e2b3fd HTTP/1.1" 200 None 2026-03-26 10:39:08,331 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:39:08,345 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebcbf63ce05331e2a47f HTTP/1.1" 200 None 2026-03-26 10:39:08,346 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:39:08,369 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c50831f63ce05331e2b3fd HTTP/1.1" 200 None 2026-03-26 10:39:08,419 - lava_callback - INFO - Completed processing callback for node 69c50831f63ce05331e2b3fd INFO: 172.21.0.1:43950 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-26 10:39:16,477 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:39:16,493 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:43958 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41714 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34086 - "GET / HTTP/1.0" 200 OK 2026-03-26 10:39:41,413 - lava_callback - INFO - Received callback for node 69c411a6944faaf0e7f60e41 INFO: 172.21.0.1:50086 - "POST /node/69c411a6944faaf0e7f60e41 HTTP/1.0" 202 Accepted 2026-03-26 10:39:41,419 - lava_callback - INFO - Starting async job submit for node 69c411a6944faaf0e7f60e41 2026-03-26 10:39:41,436 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:39:41,450 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c411a6944faaf0e7f60e41 HTTP/1.1" 200 None 2026-03-26 10:39:41,451 - lava_callback - DEBUG - Processing node 69c411a6944faaf0e7f60e41: baseline-arm64-broonie 2026-03-26 10:39:41,768 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 10:39:41,893 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 10:39:41,895 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c411a6944faaf0e7f60e41/log.txt.gz 2026-03-26 10:39:42,088 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 10:39:42,283 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 10:39:42,285 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c411a6944faaf0e7f60e41/lava-logs.yaml 2026-03-26 10:39:42,293 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 10:39:42,434 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 10:39:42,436 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c411a6944faaf0e7f60e41/lava_callback.json.gz 2026-03-26 10:39:42,454 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:39:42,468 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c411a6944faaf0e7f60e41 HTTP/1.1" 200 None 2026-03-26 10:39:42,470 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:39:42,483 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ad9944faaf0e7f5e41d HTTP/1.1" 200 None 2026-03-26 10:39:42,484 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:39:42,531 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c411a6944faaf0e7f60e41 HTTP/1.1" 200 None 2026-03-26 10:39:42,532 - lava_callback - INFO - Completed processing callback for node 69c411a6944faaf0e7f60e41 2026-03-26 10:39:46,495 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.21.0.1:50102 - "GET / HTTP/1.0" 200 OK 2026-03-26 10:39:46,515 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:57650 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55634 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55642 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55954 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55968 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35640 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60838 - "GET / HTTP/1.0" 200 OK 2026-03-26 10:40:42,142 - lava_callback - INFO - Received callback for node 69c26a274a2c5046eb880876 INFO: 172.21.0.1:43512 - "POST /node/69c26a274a2c5046eb880876 HTTP/1.0" 202 Accepted 2026-03-26 10:40:42,148 - lava_callback - INFO - Starting async job submit for node 69c26a274a2c5046eb880876 2026-03-26 10:40:42,149 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:40:42,163 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c26a274a2c5046eb880876 HTTP/1.1" 200 None 2026-03-26 10:40:42,164 - lava_callback - DEBUG - Processing node 69c26a274a2c5046eb880876: baseline-arm64-baylibre 2026-03-26 10:40:42,164 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c26a274a2c5046eb880876 2026-03-26 10:40:42,164 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:40:42,177 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c26a274a2c5046eb880876 HTTP/1.1" 200 None 2026-03-26 10:40:42,179 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:40:42,191 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ba5f7e537ae4d3c8236e55 HTTP/1.1" 200 None 2026-03-26 10:40:42,193 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:40:42,218 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c26a274a2c5046eb880876 HTTP/1.1" 200 None 2026-03-26 10:40:42,220 - lava_callback - ERROR - Error processing callback for node 69c26a274a2c5046eb880876: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:43514 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33356 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43692 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43706 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33612 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33628 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47708 - "GET / HTTP/1.0" 200 OK 2026-03-26 10:41:28,892 - lava_callback - INFO - Received callback for node 69c508a9f63ce05331e2b411 INFO: 172.21.0.1:47710 - "POST /node/69c508a9f63ce05331e2b411 HTTP/1.0" 202 Accepted 2026-03-26 10:41:28,900 - lava_callback - INFO - Starting async job submit for node 69c508a9f63ce05331e2b411 2026-03-26 10:41:28,943 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:41:28,958 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c508a9f63ce05331e2b411 HTTP/1.1" 200 None 2026-03-26 10:41:28,959 - lava_callback - DEBUG - Processing node 69c508a9f63ce05331e2b411: baseline-nfs-arm64 2026-03-26 10:41:29,969 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 10:41:30,145 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 10:41:30,147 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c508a9f63ce05331e2b411/log.txt.gz 2026-03-26 10:41:30,719 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 10:41:35,118 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 10:41:35,120 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c508a9f63ce05331e2b411/lava-logs.yaml 2026-03-26 10:41:35,140 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 10:41:35,471 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 10:41:35,473 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c508a9f63ce05331e2b411/lava_callback.json.gz DEBUG: Setup failed for 69c508a9f63ce05331e2b411. Transitting job node result: incomplete -> fail 2026-03-26 10:41:35,561 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:41:35,583 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c508a9f63ce05331e2b411 HTTP/1.1" 200 None 2026-03-26 10:41:35,584 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:41:35,600 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebd8f63ce05331e2a492 HTTP/1.1" 200 None 2026-03-26 10:41:35,602 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:41:35,634 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c508a9f63ce05331e2b411 HTTP/1.1" 200 None 2026-03-26 10:41:35,737 - lava_callback - INFO - Completed processing callback for node 69c508a9f63ce05331e2b411 INFO: 172.21.0.1:36742 - "GET / HTTP/1.0" 200 OK 2026-03-26 10:41:46,518 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.21.0.1:50934 - "GET / HTTP/1.0" 200 OK 2026-03-26 10:41:46,534 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:40216 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57536 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57542 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42790 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42798 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57160 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59080 - "GET / HTTP/1.0" 200 OK 2026-03-26 10:42:41,545 - lava_callback - INFO - Received callback for node 69c46d53944faaf0e7f6206c INFO: 172.21.0.1:38386 - "POST /node/69c46d53944faaf0e7f6206c HTTP/1.0" 202 Accepted 2026-03-26 10:42:41,551 - lava_callback - INFO - Starting async job submit for node 69c46d53944faaf0e7f6206c 2026-03-26 10:42:41,569 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:42:41,608 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c46d53944faaf0e7f6206c HTTP/1.1" 200 None 2026-03-26 10:42:41,609 - lava_callback - DEBUG - Processing node 69c46d53944faaf0e7f6206c: baseline-arm64-broonie 2026-03-26 10:42:41,964 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 10:42:42,119 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 10:42:42,121 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c46d53944faaf0e7f6206c/log.txt.gz 2026-03-26 10:42:42,316 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 10:42:42,583 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 10:42:42,585 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c46d53944faaf0e7f6206c/lava-logs.yaml 2026-03-26 10:42:42,593 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 10:42:42,758 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 10:42:42,760 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c46d53944faaf0e7f6206c/lava_callback.json.gz 2026-03-26 10:42:42,802 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:42:42,817 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c46d53944faaf0e7f6206c HTTP/1.1" 200 None 2026-03-26 10:42:42,819 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:42:42,857 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ad9944faaf0e7f5e41d HTTP/1.1" 200 None 2026-03-26 10:42:42,858 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:42:42,883 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c46d53944faaf0e7f6206c HTTP/1.1" 200 None 2026-03-26 10:42:42,936 - lava_callback - INFO - Completed processing callback for node 69c46d53944faaf0e7f6206c INFO: 172.21.0.1:38402 - "GET / HTTP/1.0" 200 OK 2026-03-26 10:42:46,536 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:42:46,557 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:42764 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48308 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48318 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36914 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36924 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45476 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43506 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56370 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57822 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54350 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54364 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58568 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58582 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42904 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51624 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43202 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45112 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54854 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54870 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34154 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34156 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56642 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59916 - "GET / HTTP/1.0" 200 OK 2026-03-26 10:45:41,808 - lava_callback - INFO - Received callback for node 69c48871590270ad7b6d9584 INFO: 172.21.0.1:47868 - "POST /node/69c48871590270ad7b6d9584 HTTP/1.0" 202 Accepted 2026-03-26 10:45:41,814 - lava_callback - INFO - Starting async job submit for node 69c48871590270ad7b6d9584 2026-03-26 10:45:41,832 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:45:41,846 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48871590270ad7b6d9584 HTTP/1.1" 200 None 2026-03-26 10:45:41,847 - lava_callback - DEBUG - Processing node 69c48871590270ad7b6d9584: baseline-arm64-broonie 2026-03-26 10:45:42,178 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 10:45:42,223 - lava_callback - INFO - Received callback for node 69c50a1cf63ce05331e2b425 INFO: 172.21.0.1:47878 - "POST /node/69c50a1cf63ce05331e2b425 HTTP/1.0" 202 Accepted 2026-03-26 10:45:42,229 - lava_callback - INFO - Starting async job submit for node 69c50a1cf63ce05331e2b425 2026-03-26 10:45:42,285 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:45:42,299 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c50a1cf63ce05331e2b425 HTTP/1.1" 200 None 2026-03-26 10:45:42,300 - lava_callback - DEBUG - Processing node 69c50a1cf63ce05331e2b425: baseline-arm 2026-03-26 10:45:42,561 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 10:45:42,614 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c48871590270ad7b6d9584/log.txt.gz 2026-03-26 10:45:43,124 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 10:45:43,153 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 10:45:43,330 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 10:45:43,332 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69c50a1cf63ce05331e2b425/log.txt.gz 2026-03-26 10:45:43,367 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 10:45:43,398 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c48871590270ad7b6d9584/lava-logs.yaml 2026-03-26 10:45:43,482 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 10:45:43,704 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 10:45:43,713 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 10:45:43,715 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c48871590270ad7b6d9584/lava_callback.json.gz 2026-03-26 10:45:43,771 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:45:43,791 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48871590270ad7b6d9584 HTTP/1.1" 200 None 2026-03-26 10:45:43,793 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:45:43,806 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b4a590270ad7b6d699d HTTP/1.1" 200 None 2026-03-26 10:45:43,808 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:45:43,836 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c48871590270ad7b6d9584 HTTP/1.1" 200 None 2026-03-26 10:45:43,902 - lava_callback - INFO - Completed processing callback for node 69c48871590270ad7b6d9584 2026-03-26 10:45:44,054 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 10:45:44,057 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c50a1cf63ce05331e2b425/lava-logs.yaml 2026-03-26 10:45:44,077 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 10:45:44,253 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 10:45:44,254 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c50a1cf63ce05331e2b425/lava_callback.json.gz DEBUG: Setup failed for 69c50a1cf63ce05331e2b425. Transitting job node result: incomplete -> fail 2026-03-26 10:45:44,365 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:45:44,380 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c50a1cf63ce05331e2b425 HTTP/1.1" 200 None 2026-03-26 10:45:44,382 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:45:44,397 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebcbf63ce05331e2a47f HTTP/1.1" 200 None 2026-03-26 10:45:44,398 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:45:44,424 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c50a1cf63ce05331e2b425 HTTP/1.1" 200 None 2026-03-26 10:45:44,537 - lava_callback - INFO - Completed processing callback for node 69c50a1cf63ce05331e2b425 INFO: 172.21.0.1:47880 - "GET / HTTP/1.0" 200 OK 2026-03-26 10:45:46,559 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:45:46,576 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:47250 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49536 - "GET / HTTP/1.0" 200 OK 2026-03-26 10:46:02,254 - lava_callback - INFO - Received callback for node 69c26b564a2c5046eb88089e INFO: 172.21.0.1:49542 - "POST /node/69c26b564a2c5046eb88089e HTTP/1.0" 202 Accepted 2026-03-26 10:46:02,260 - lava_callback - INFO - Starting async job submit for node 69c26b564a2c5046eb88089e 2026-03-26 10:46:02,261 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:46:02,282 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c26b564a2c5046eb88089e HTTP/1.1" 200 None 2026-03-26 10:46:02,283 - lava_callback - DEBUG - Processing node 69c26b564a2c5046eb88089e: baseline-arm64-baylibre 2026-03-26 10:46:02,283 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c26b564a2c5046eb88089e 2026-03-26 10:46:02,283 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:46:02,298 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c26b564a2c5046eb88089e HTTP/1.1" 200 None 2026-03-26 10:46:02,300 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:46:02,315 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ba5f7e537ae4d3c8236e55 HTTP/1.1" 200 None 2026-03-26 10:46:02,317 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:46:02,339 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c26b564a2c5046eb88089e HTTP/1.1" 200 None 2026-03-26 10:46:02,346 - lava_callback - ERROR - Error processing callback for node 69c26b564a2c5046eb88089e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:49546 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40590 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40604 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47596 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58612 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51642 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51136 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51578 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51582 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46044 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46046 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43624 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45520 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60082 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59398 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35078 - "GET / HTTP/1.0" 200 OK 2026-03-26 10:48:04,691 - lava_callback - INFO - Received callback for node 69c3cf5ebbc022e792645336 INFO: 172.21.0.1:35084 - "POST /node/69c3cf5ebbc022e792645336 HTTP/1.0" 202 Accepted 2026-03-26 10:48:04,697 - lava_callback - INFO - Starting async job submit for node 69c3cf5ebbc022e792645336 2026-03-26 10:48:04,721 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:48:04,735 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3cf5ebbc022e792645336 HTTP/1.1" 200 None 2026-03-26 10:48:04,736 - lava_callback - DEBUG - Processing node 69c3cf5ebbc022e792645336: baseline-arm64 2026-03-26 10:48:05,229 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 10:48:05,541 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 10:48:05,543 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c3cf5ebbc022e792645336/log.txt.gz 2026-03-26 10:48:05,816 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 10:48:06,007 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 10:48:06,009 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c3cf5ebbc022e792645336/lava-logs.yaml 2026-03-26 10:48:06,020 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 10:48:06,183 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 10:48:06,185 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c3cf5ebbc022e792645336/lava_callback.json.gz 2026-03-26 10:48:06,209 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:48:06,224 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3cf5ebbc022e792645336 HTTP/1.1" 200 None 2026-03-26 10:48:06,226 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:48:06,240 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399cdbbc022e792643ea0 HTTP/1.1" 200 None 2026-03-26 10:48:06,241 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:48:06,263 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3cf5ebbc022e792645336 HTTP/1.1" 200 None 2026-03-26 10:48:06,328 - lava_callback - INFO - Completed processing callback for node 69c3cf5ebbc022e792645336 INFO: 172.21.0.1:35094 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45686 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45690 - "GET / HTTP/1.0" 200 OK 2026-03-26 10:48:16,579 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:48:16,595 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:39822 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33652 - "GET / HTTP/1.0" 200 OK 2026-03-26 10:48:42,179 - lava_callback - INFO - Received callback for node 69c4c1d6590270ad7b6d9f0a INFO: 172.21.0.1:60154 - "POST /node/69c4c1d6590270ad7b6d9f0a HTTP/1.0" 202 Accepted 2026-03-26 10:48:42,185 - lava_callback - INFO - Starting async job submit for node 69c4c1d6590270ad7b6d9f0a 2026-03-26 10:48:42,203 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:48:42,227 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4c1d6590270ad7b6d9f0a HTTP/1.1" 200 None 2026-03-26 10:48:42,228 - lava_callback - DEBUG - Processing node 69c4c1d6590270ad7b6d9f0a: baseline-arm64-broonie 2026-03-26 10:48:42,386 - lava_callback - INFO - Received callback for node 69c3d887bbc022e7926453ea INFO: 172.21.0.1:60160 - "POST /node/69c3d887bbc022e7926453ea HTTP/1.0" 202 Accepted 2026-03-26 10:48:42,398 - lava_callback - INFO - Starting async job submit for node 69c3d887bbc022e7926453ea 2026-03-26 10:48:42,456 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:48:42,528 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3d887bbc022e7926453ea HTTP/1.1" 200 None 2026-03-26 10:48:42,544 - lava_callback - DEBUG - Processing node 69c3d887bbc022e7926453ea: baseline-nfs-arm64 2026-03-26 10:48:42,718 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 10:48:42,944 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 10:48:42,981 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c4c1d6590270ad7b6d9f0a/log.txt.gz 2026-03-26 10:48:43,384 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 10:48:43,390 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 10:48:43,520 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 10:48:43,522 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c3d887bbc022e7926453ea/log.txt.gz 2026-03-26 10:48:43,621 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 10:48:43,672 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c4c1d6590270ad7b6d9f0a/lava-logs.yaml 2026-03-26 10:48:43,768 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 10:48:43,803 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 10:48:43,938 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 10:48:43,939 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c4c1d6590270ad7b6d9f0a/lava_callback.json.gz 2026-03-26 10:48:43,978 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:48:43,992 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4c1d6590270ad7b6d9f0a HTTP/1.1" 200 None 2026-03-26 10:48:43,994 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:48:44,010 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ad9944faaf0e7f5e41d HTTP/1.1" 200 None 2026-03-26 10:48:44,013 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:48:44,038 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4c1d6590270ad7b6d9f0a HTTP/1.1" 200 None 2026-03-26 10:48:44,058 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 10:48:44,076 - lava_callback - INFO - Completed processing callback for node 69c4c1d6590270ad7b6d9f0a 2026-03-26 10:48:44,077 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c3d887bbc022e7926453ea/lava-logs.yaml 2026-03-26 10:48:44,090 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 10:48:44,300 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 10:48:44,303 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c3d887bbc022e7926453ea/lava_callback.json.gz 2026-03-26 10:48:44,344 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:48:44,363 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3d887bbc022e7926453ea HTTP/1.1" 200 None 2026-03-26 10:48:44,364 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:48:44,380 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399cdbbc022e792643ea0 HTTP/1.1" 200 None 2026-03-26 10:48:44,382 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:48:44,407 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3d887bbc022e7926453ea HTTP/1.1" 200 None 2026-03-26 10:48:44,476 - lava_callback - INFO - Received callback for node 69c26bf64a2c5046eb8808c2 2026-03-26 10:48:44,497 - lava_callback - INFO - Completed processing callback for node 69c3d887bbc022e7926453ea INFO: 172.21.0.1:60164 - "POST /node/69c26bf64a2c5046eb8808c2 HTTP/1.0" 202 Accepted 2026-03-26 10:48:44,499 - lava_callback - INFO - Starting async job submit for node 69c26bf64a2c5046eb8808c2 2026-03-26 10:48:44,500 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:48:44,530 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c26bf64a2c5046eb8808c2 HTTP/1.1" 200 None 2026-03-26 10:48:44,532 - lava_callback - DEBUG - Processing node 69c26bf64a2c5046eb8808c2: baseline-arm-clabbe 2026-03-26 10:48:44,532 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c26bf64a2c5046eb8808c2 2026-03-26 10:48:44,533 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:48:44,568 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c26bf64a2c5046eb8808c2 HTTP/1.1" 200 None 2026-03-26 10:48:44,570 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:48:44,584 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c167842b2b51e9ad977943 HTTP/1.1" 200 None 2026-03-26 10:48:44,586 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:48:44,612 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c26bf64a2c5046eb8808c2 HTTP/1.1" 200 None 2026-03-26 10:48:44,614 - lava_callback - ERROR - Error processing callback for node 69c26bf64a2c5046eb8808c2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:60180 - "GET / HTTP/1.0" 200 OK 2026-03-26 10:48:46,597 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:48:46,613 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:51556 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41928 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41944 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33650 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33664 - "GET / HTTP/1.0" 200 OK 2026-03-26 10:49:24,440 - lava_callback - INFO - Received callback for node 69c26c264a2c5046eb8808d4 INFO: 172.21.0.1:34978 - "POST /node/69c26c264a2c5046eb8808d4 HTTP/1.0" 202 Accepted 2026-03-26 10:49:24,446 - lava_callback - INFO - Starting async job submit for node 69c26c264a2c5046eb8808d4 2026-03-26 10:49:24,447 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:49:24,472 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c26c264a2c5046eb8808d4 HTTP/1.1" 200 None 2026-03-26 10:49:24,473 - lava_callback - DEBUG - Processing node 69c26c264a2c5046eb8808d4: baseline-arm-clabbe 2026-03-26 10:49:24,473 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c26c264a2c5046eb8808d4 2026-03-26 10:49:24,474 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:49:24,487 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c26c264a2c5046eb8808d4 HTTP/1.1" 200 None 2026-03-26 10:49:24,488 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:49:24,501 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c167842b2b51e9ad977943 HTTP/1.1" 200 None 2026-03-26 10:49:24,503 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:49:24,524 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c26c264a2c5046eb8808d4 HTTP/1.1" 200 None 2026-03-26 10:49:24,526 - lava_callback - ERROR - Error processing callback for node 69c26c264a2c5046eb8808d4: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:34994 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53702 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46686 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34258 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32868 - "GET / HTTP/1.0" 200 OK 2026-03-26 10:50:04,653 - lava_callback - INFO - Received callback for node 69c26c4b4a2c5046eb8808df INFO: 172.21.0.1:32870 - "POST /node/69c26c4b4a2c5046eb8808df HTTP/1.0" 202 Accepted 2026-03-26 10:50:04,659 - lava_callback - INFO - Starting async job submit for node 69c26c4b4a2c5046eb8808df 2026-03-26 10:50:04,660 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:50:04,674 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c26c4b4a2c5046eb8808df HTTP/1.1" 200 None 2026-03-26 10:50:04,674 - lava_callback - DEBUG - Processing node 69c26c4b4a2c5046eb8808df: baseline-arm-clabbe 2026-03-26 10:50:04,675 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c26c4b4a2c5046eb8808df 2026-03-26 10:50:04,675 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:50:04,688 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c26c4b4a2c5046eb8808df HTTP/1.1" 200 None 2026-03-26 10:50:04,690 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:50:04,703 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c167842b2b51e9ad977943 HTTP/1.1" 200 None 2026-03-26 10:50:04,705 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:50:04,734 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c26c4b4a2c5046eb8808df HTTP/1.1" 200 None 2026-03-26 10:50:04,736 - lava_callback - ERROR - Error processing callback for node 69c26c4b4a2c5046eb8808df: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:32872 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42462 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42478 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56068 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60692 - "GET / HTTP/1.0" 200 OK 2026-03-26 10:50:44,510 - lava_callback - INFO - Received callback for node 69c26c704a2c5046eb8808ed INFO: 172.21.0.1:41906 - "POST /node/69c26c704a2c5046eb8808ed HTTP/1.0" 202 Accepted 2026-03-26 10:50:44,516 - lava_callback - INFO - Starting async job submit for node 69c26c704a2c5046eb8808ed 2026-03-26 10:50:44,516 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:50:44,531 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c26c704a2c5046eb8808ed HTTP/1.1" 200 None 2026-03-26 10:50:44,532 - lava_callback - DEBUG - Processing node 69c26c704a2c5046eb8808ed: baseline-arm-clabbe 2026-03-26 10:50:44,532 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c26c704a2c5046eb8808ed 2026-03-26 10:50:44,533 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:50:44,546 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c26c704a2c5046eb8808ed HTTP/1.1" 200 None 2026-03-26 10:50:44,547 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:50:44,568 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c167842b2b51e9ad977943 HTTP/1.1" 200 None 2026-03-26 10:50:44,570 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:50:44,591 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c26c704a2c5046eb8808ed HTTP/1.1" 200 None 2026-03-26 10:50:44,595 - lava_callback - ERROR - Error processing callback for node 69c26c704a2c5046eb8808ed: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:41914 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36624 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40488 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40494 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60696 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60704 - "GET / HTTP/1.0" 200 OK 2026-03-26 10:51:24,556 - lava_callback - INFO - Received callback for node 69c26ca34a2c5046eb8808fb INFO: 172.21.0.1:37378 - "POST /node/69c26ca34a2c5046eb8808fb HTTP/1.0" 202 Accepted 2026-03-26 10:51:24,562 - lava_callback - INFO - Starting async job submit for node 69c26ca34a2c5046eb8808fb 2026-03-26 10:51:24,562 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:51:24,582 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c26ca34a2c5046eb8808fb HTTP/1.1" 200 None 2026-03-26 10:51:24,583 - lava_callback - DEBUG - Processing node 69c26ca34a2c5046eb8808fb: baseline-arm64-clabbe 2026-03-26 10:51:24,583 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c26ca34a2c5046eb8808fb 2026-03-26 10:51:24,584 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:51:24,599 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c26ca34a2c5046eb8808fb HTTP/1.1" 200 None 2026-03-26 10:51:24,600 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:51:24,614 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ba5f7e537ae4d3c8236e55 HTTP/1.1" 200 None 2026-03-26 10:51:24,616 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:51:24,637 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c26ca34a2c5046eb8808fb HTTP/1.1" 200 None 2026-03-26 10:51:24,638 - lava_callback - ERROR - Error processing callback for node 69c26ca34a2c5046eb8808fb: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:37394 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33432 - "GET / HTTP/1.0" 200 OK 2026-03-26 10:51:42,340 - lava_callback - INFO - Received callback for node 69c4e00e590270ad7b6da4d1 INFO: 172.21.0.1:37150 - "POST /node/69c4e00e590270ad7b6da4d1 HTTP/1.0" 202 Accepted 2026-03-26 10:51:42,346 - lava_callback - INFO - Starting async job submit for node 69c4e00e590270ad7b6da4d1 2026-03-26 10:51:42,365 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:51:42,381 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4e00e590270ad7b6da4d1 HTTP/1.1" 200 None 2026-03-26 10:51:42,381 - lava_callback - DEBUG - Processing node 69c4e00e590270ad7b6da4d1: baseline-arm64-broonie 2026-03-26 10:51:42,733 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 10:51:42,893 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 10:51:42,895 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c4e00e590270ad7b6da4d1/log.txt.gz 2026-03-26 10:51:43,092 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 10:51:43,326 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 10:51:43,329 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c4e00e590270ad7b6da4d1/lava-logs.yaml 2026-03-26 10:51:43,337 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 10:51:43,505 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 10:51:43,506 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c4e00e590270ad7b6da4d1/lava_callback.json.gz 2026-03-26 10:51:43,543 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:51:43,558 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4e00e590270ad7b6da4d1 HTTP/1.1" 200 None 2026-03-26 10:51:43,560 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:51:43,573 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b4a590270ad7b6d699d HTTP/1.1" 200 None 2026-03-26 10:51:43,575 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:51:43,611 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4e00e590270ad7b6da4d1 HTTP/1.1" 200 None 2026-03-26 10:51:43,668 - lava_callback - INFO - Completed processing callback for node 69c4e00e590270ad7b6da4d1 INFO: 172.21.0.1:37156 - "GET / HTTP/1.0" 200 OK 2026-03-26 10:51:46,616 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:51:46,632 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:52428 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35434 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35448 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37404 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37414 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39462 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52850 - "GET / HTTP/1.0" 200 OK 2026-03-26 10:52:44,606 - lava_callback - INFO - Received callback for node 69c26ce64a2c5046eb880905 INFO: 172.21.0.1:38986 - "POST /node/69c26ce64a2c5046eb880905 HTTP/1.0" 202 Accepted 2026-03-26 10:52:44,611 - lava_callback - INFO - Starting async job submit for node 69c26ce64a2c5046eb880905 2026-03-26 10:52:44,612 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:52:44,627 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c26ce64a2c5046eb880905 HTTP/1.1" 200 None 2026-03-26 10:52:44,627 - lava_callback - DEBUG - Processing node 69c26ce64a2c5046eb880905: baseline-arm-clabbe 2026-03-26 10:52:44,628 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c26ce64a2c5046eb880905 2026-03-26 10:52:44,628 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:52:44,641 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c26ce64a2c5046eb880905 HTTP/1.1" 200 None 2026-03-26 10:52:44,643 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:52:44,661 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c167842b2b51e9ad977943 HTTP/1.1" 200 None 2026-03-26 10:52:44,663 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:52:44,684 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c26ce64a2c5046eb880905 HTTP/1.1" 200 None 2026-03-26 10:52:44,693 - lava_callback - ERROR - Error processing callback for node 69c26ce64a2c5046eb880905: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:38994 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54788 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44320 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44336 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36540 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36550 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51098 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39108 - "GET / HTTP/1.0" 200 OK 2026-03-26 10:53:42,328 - lava_callback - INFO - Received callback for node 69c4f4f4f63ce05331e2af18 INFO: 172.21.0.1:35988 - "POST /node/69c4f4f4f63ce05331e2af18 HTTP/1.0" 202 Accepted 2026-03-26 10:53:42,335 - lava_callback - INFO - Starting async job submit for node 69c4f4f4f63ce05331e2af18 2026-03-26 10:53:42,354 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:53:42,368 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f4f4f63ce05331e2af18 HTTP/1.1" 200 None 2026-03-26 10:53:42,369 - lava_callback - DEBUG - Processing node 69c4f4f4f63ce05331e2af18: baseline-arm64-broonie 2026-03-26 10:53:42,700 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 10:53:42,848 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 10:53:42,850 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c4f4f4f63ce05331e2af18/log.txt.gz 2026-03-26 10:53:43,049 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 10:53:43,440 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 10:53:43,442 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c4f4f4f63ce05331e2af18/lava-logs.yaml 2026-03-26 10:53:43,450 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 10:53:43,644 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 10:53:43,646 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c4f4f4f63ce05331e2af18/lava_callback.json.gz 2026-03-26 10:53:43,664 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:53:43,678 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f4f4f63ce05331e2af18 HTTP/1.1" 200 None 2026-03-26 10:53:43,680 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:53:43,700 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebd8f63ce05331e2a492 HTTP/1.1" 200 None 2026-03-26 10:53:43,702 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:53:43,726 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4f4f4f63ce05331e2af18 HTTP/1.1" 200 None 2026-03-26 10:53:43,732 - lava_callback - INFO - Completed processing callback for node 69c4f4f4f63ce05331e2af18 INFO: 172.21.0.1:35990 - "GET / HTTP/1.0" 200 OK 2026-03-26 10:53:46,634 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:53:46,651 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:49584 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36704 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36706 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33378 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33388 - "GET / HTTP/1.0" 200 OK 2026-03-26 10:54:24,716 - lava_callback - INFO - Received callback for node 69c26d5b4a2c5046eb880915 INFO: 172.21.0.1:37610 - "POST /node/69c26d5b4a2c5046eb880915 HTTP/1.0" 202 Accepted 2026-03-26 10:54:24,722 - lava_callback - INFO - Starting async job submit for node 69c26d5b4a2c5046eb880915 2026-03-26 10:54:24,723 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:54:24,737 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c26d5b4a2c5046eb880915 HTTP/1.1" 200 None 2026-03-26 10:54:24,738 - lava_callback - DEBUG - Processing node 69c26d5b4a2c5046eb880915: baseline-arm64-clabbe 2026-03-26 10:54:24,738 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c26d5b4a2c5046eb880915 2026-03-26 10:54:24,739 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:54:24,752 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c26d5b4a2c5046eb880915 HTTP/1.1" 200 None 2026-03-26 10:54:24,753 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:54:24,767 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ba5f7e537ae4d3c8236e55 HTTP/1.1" 200 None 2026-03-26 10:54:24,769 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:54:24,790 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c26d5b4a2c5046eb880915 HTTP/1.1" 200 None 2026-03-26 10:54:24,791 - lava_callback - ERROR - Error processing callback for node 69c26d5b4a2c5046eb880915: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:37620 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49618 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48436 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47624 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43376 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43378 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33342 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33346 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59964 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41362 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56370 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55930 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55496 - "GET / HTTP/1.0" 200 OK 2026-03-26 10:56:04,730 - lava_callback - INFO - Received callback for node 69c26db04a2c5046eb880923 INFO: 172.21.0.1:55498 - "POST /node/69c26db04a2c5046eb880923 HTTP/1.0" 202 Accepted 2026-03-26 10:56:04,736 - lava_callback - INFO - Starting async job submit for node 69c26db04a2c5046eb880923 2026-03-26 10:56:04,736 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:56:04,751 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c26db04a2c5046eb880923 HTTP/1.1" 200 None 2026-03-26 10:56:04,752 - lava_callback - DEBUG - Processing node 69c26db04a2c5046eb880923: baseline-arm64-clabbe 2026-03-26 10:56:04,752 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c26db04a2c5046eb880923 2026-03-26 10:56:04,753 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:56:04,766 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c26db04a2c5046eb880923 HTTP/1.1" 200 None 2026-03-26 10:56:04,768 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:56:04,786 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ba5f7e537ae4d3c8236e55 HTTP/1.1" 200 None 2026-03-26 10:56:04,788 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:56:04,808 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c26db04a2c5046eb880923 HTTP/1.1" 200 None 2026-03-26 10:56:04,813 - lava_callback - ERROR - Error processing callback for node 69c26db04a2c5046eb880923: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:55512 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36048 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36050 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37328 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33442 - "GET / HTTP/1.0" 200 OK 2026-03-26 10:56:42,431 - lava_callback - INFO - Received callback for node 69c50dc0f63ce05331e2b47c INFO: 172.21.0.1:36520 - "POST /node/69c50dc0f63ce05331e2b47c HTTP/1.0" 202 Accepted 2026-03-26 10:56:42,438 - lava_callback - INFO - Starting async job submit for node 69c50dc0f63ce05331e2b47c 2026-03-26 10:56:42,456 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:56:42,479 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c50dc0f63ce05331e2b47c HTTP/1.1" 200 None 2026-03-26 10:56:42,480 - lava_callback - DEBUG - Processing node 69c50dc0f63ce05331e2b47c: baseline-arm64-broonie 2026-03-26 10:56:42,864 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 10:56:43,010 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 10:56:43,012 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c50dc0f63ce05331e2b47c/log.txt.gz 2026-03-26 10:56:43,208 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 10:56:43,587 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 10:56:43,589 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c50dc0f63ce05331e2b47c/lava-logs.yaml 2026-03-26 10:56:43,598 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 10:56:43,756 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 10:56:43,758 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c50dc0f63ce05331e2b47c/lava_callback.json.gz 2026-03-26 10:56:43,794 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:56:43,809 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c50dc0f63ce05331e2b47c HTTP/1.1" 200 None 2026-03-26 10:56:43,810 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:56:43,824 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ad9944faaf0e7f5e41d HTTP/1.1" 200 None 2026-03-26 10:56:43,825 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:56:43,862 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c50dc0f63ce05331e2b47c HTTP/1.1" 200 None 2026-03-26 10:56:43,914 - lava_callback - INFO - Completed processing callback for node 69c50dc0f63ce05331e2b47c INFO: 172.21.0.1:36526 - "GET / HTTP/1.0" 200 OK 2026-03-26 10:56:46,653 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:56:46,670 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:47466 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55632 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55648 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50534 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50546 - "GET / HTTP/1.0" 200 OK 2026-03-26 10:57:24,854 - lava_callback - INFO - Received callback for node 69c26e074a2c5046eb88092d INFO: 172.21.0.1:60770 - "POST /node/69c26e074a2c5046eb88092d HTTP/1.0" 202 Accepted 2026-03-26 10:57:24,859 - lava_callback - INFO - Starting async job submit for node 69c26e074a2c5046eb88092d 2026-03-26 10:57:24,860 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:57:24,879 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c26e074a2c5046eb88092d HTTP/1.1" 200 None 2026-03-26 10:57:24,880 - lava_callback - DEBUG - Processing node 69c26e074a2c5046eb88092d: baseline-arm64-clabbe 2026-03-26 10:57:24,880 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c26e074a2c5046eb88092d 2026-03-26 10:57:24,881 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:57:24,894 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c26e074a2c5046eb88092d HTTP/1.1" 200 None 2026-03-26 10:57:24,896 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:57:24,910 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ba5f7e537ae4d3c8236e55 HTTP/1.1" 200 None 2026-03-26 10:57:24,911 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:57:24,932 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c26e074a2c5046eb88092d HTTP/1.1" 200 None 2026-03-26 10:57:24,938 - lava_callback - ERROR - Error processing callback for node 69c26e074a2c5046eb88092d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:60780 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52302 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47166 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50220 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50054 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50056 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60316 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60328 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34732 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57284 - "GET / HTTP/1.0" 200 OK 2026-03-26 10:58:44,848 - lava_callback - INFO - Received callback for node 69c26e584a2c5046eb880938 INFO: 172.21.0.1:38926 - "POST /node/69c26e584a2c5046eb880938 HTTP/1.0" 202 Accepted 2026-03-26 10:58:44,853 - lava_callback - INFO - Starting async job submit for node 69c26e584a2c5046eb880938 2026-03-26 10:58:44,854 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:58:44,869 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c26e584a2c5046eb880938 HTTP/1.1" 200 None 2026-03-26 10:58:44,869 - lava_callback - DEBUG - Processing node 69c26e584a2c5046eb880938: baseline-arm64-clabbe 2026-03-26 10:58:44,870 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c26e584a2c5046eb880938 2026-03-26 10:58:44,870 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:58:44,883 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c26e584a2c5046eb880938 HTTP/1.1" 200 None 2026-03-26 10:58:44,885 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:58:44,900 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ba5f7e537ae4d3c8236e55 HTTP/1.1" 200 None 2026-03-26 10:58:44,902 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:58:44,927 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c26e584a2c5046eb880938 HTTP/1.1" 200 None 2026-03-26 10:58:44,929 - lava_callback - ERROR - Error processing callback for node 69c26e584a2c5046eb880938: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:38936 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44470 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40498 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40506 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39038 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39050 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52442 - "GET / HTTP/1.0" 200 OK 2026-03-26 10:59:26,879 - lava_callback - INFO - Received callback for node 69c50cd9f63ce05331e2b44e INFO: 172.21.0.1:52446 - "POST /node/69c50cd9f63ce05331e2b44e HTTP/1.0" 202 Accepted 2026-03-26 10:59:26,925 - lava_callback - INFO - Starting async job submit for node 69c50cd9f63ce05331e2b44e 2026-03-26 10:59:26,950 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:59:26,966 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c50cd9f63ce05331e2b44e HTTP/1.1" 200 None 2026-03-26 10:59:26,967 - lava_callback - DEBUG - Processing node 69c50cd9f63ce05331e2b44e: baseline-arm-pengutronix 2026-03-26 10:59:27,074 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 10:59:27,210 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 10:59:27,212 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c50cd9f63ce05331e2b44e/log.txt.gz 2026-03-26 10:59:27,272 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 10:59:27,425 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 10:59:27,427 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c50cd9f63ce05331e2b44e/lava-logs.yaml 2026-03-26 10:59:27,430 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 10:59:27,559 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 10:59:27,561 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c50cd9f63ce05331e2b44e/lava_callback.json.gz DEBUG: Setup failed for 69c50cd9f63ce05331e2b44e. Transitting job node result: incomplete -> fail 2026-03-26 10:59:27,610 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:59:27,624 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c50cd9f63ce05331e2b44e HTTP/1.1" 200 None 2026-03-26 10:59:27,625 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:59:27,638 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebcbf63ce05331e2a47f HTTP/1.1" 200 None 2026-03-26 10:59:27,640 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:59:27,668 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c50cd9f63ce05331e2b44e HTTP/1.1" 200 None 2026-03-26 10:59:27,720 - lava_callback - INFO - Completed processing callback for node 69c50cd9f63ce05331e2b44e INFO: 172.21.0.1:56628 - "GET / HTTP/1.0" 200 OK 2026-03-26 10:59:42,177 - lava_callback - INFO - Received callback for node 69c50e6af63ce05331e2b4a0 INFO: 172.21.0.1:52976 - "POST /node/69c50e6af63ce05331e2b4a0 HTTP/1.0" 202 Accepted 2026-03-26 10:59:42,183 - lava_callback - INFO - Starting async job submit for node 69c50e6af63ce05331e2b4a0 2026-03-26 10:59:42,201 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:59:42,223 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c50e6af63ce05331e2b4a0 HTTP/1.1" 200 None 2026-03-26 10:59:42,224 - lava_callback - DEBUG - Processing node 69c50e6af63ce05331e2b4a0: baseline-arm64-broonie 2026-03-26 10:59:42,543 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 10:59:42,684 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 10:59:42,686 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c50e6af63ce05331e2b4a0/log.txt.gz 2026-03-26 10:59:42,898 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 10:59:43,116 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 10:59:43,118 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c50e6af63ce05331e2b4a0/lava-logs.yaml 2026-03-26 10:59:43,126 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 10:59:43,286 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 10:59:43,288 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c50e6af63ce05331e2b4a0/lava_callback.json.gz 2026-03-26 10:59:43,328 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:59:43,354 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c50e6af63ce05331e2b4a0 HTTP/1.1" 200 None 2026-03-26 10:59:43,356 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:59:43,371 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b4a590270ad7b6d699d HTTP/1.1" 200 None 2026-03-26 10:59:43,373 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:59:43,400 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c50e6af63ce05331e2b4a0 HTTP/1.1" 200 None 2026-03-26 10:59:43,437 - lava_callback - INFO - Completed processing callback for node 69c50e6af63ce05331e2b4a0 2026-03-26 10:59:46,672 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 10:59:46,689 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:52984 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53180 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37316 - "GET / HTTP/1.0" 200 OK 2026-03-26 11:00:05,380 - lava_callback - INFO - Received callback for node 69c26eaa4a2c5046eb88093f INFO: 172.21.0.1:37324 - "POST /node/69c26eaa4a2c5046eb88093f HTTP/1.0" 202 Accepted 2026-03-26 11:00:05,386 - lava_callback - INFO - Starting async job submit for node 69c26eaa4a2c5046eb88093f 2026-03-26 11:00:05,387 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 11:00:05,400 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c26eaa4a2c5046eb88093f HTTP/1.1" 200 None 2026-03-26 11:00:05,401 - lava_callback - DEBUG - Processing node 69c26eaa4a2c5046eb88093f: baseline-arm64-clabbe 2026-03-26 11:00:05,401 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c26eaa4a2c5046eb88093f 2026-03-26 11:00:05,402 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 11:00:05,415 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c26eaa4a2c5046eb88093f HTTP/1.1" 200 None 2026-03-26 11:00:05,417 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 11:00:05,431 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ba5f7e537ae4d3c8236e55 HTTP/1.1" 200 None 2026-03-26 11:00:05,432 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 11:00:05,460 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c26eaa4a2c5046eb88093f HTTP/1.1" 200 None 2026-03-26 11:00:05,461 - lava_callback - ERROR - Error processing callback for node 69c26eaa4a2c5046eb88093f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:37334 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39790 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39804 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36150 - "GET / HTTP/1.0" 200 OK 2026-03-26 11:00:26,536 - lava_callback - INFO - Received callback for node 69c40307bbc022e792645b7e INFO: 172.21.0.1:36160 - "POST /node/69c40307bbc022e792645b7e HTTP/1.0" 202 Accepted 2026-03-26 11:00:26,543 - lava_callback - INFO - Starting async job submit for node 69c40307bbc022e792645b7e 2026-03-26 11:00:26,568 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 11:00:26,587 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40307bbc022e792645b7e HTTP/1.1" 200 None 2026-03-26 11:00:26,588 - lava_callback - DEBUG - Processing node 69c40307bbc022e792645b7e: baseline-arm64 2026-03-26 11:00:27,096 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 11:00:27,230 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 11:00:27,232 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c40307bbc022e792645b7e/log.txt.gz 2026-03-26 11:00:27,505 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 11:00:27,712 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 11:00:27,714 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c40307bbc022e792645b7e/lava-logs.yaml 2026-03-26 11:00:27,726 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 11:00:27,933 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 11:00:27,935 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c40307bbc022e792645b7e/lava_callback.json.gz 2026-03-26 11:00:27,960 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 11:00:27,974 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40307bbc022e792645b7e HTTP/1.1" 200 None 2026-03-26 11:00:27,976 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 11:00:27,989 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3295318a3717aa65d01c3 HTTP/1.1" 200 None 2026-03-26 11:00:27,991 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 11:00:28,015 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c40307bbc022e792645b7e HTTP/1.1" 200 None 2026-03-26 11:00:28,070 - lava_callback - INFO - Completed processing callback for node 69c40307bbc022e792645b7e INFO: 172.21.0.1:46986 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36608 - "GET / HTTP/1.0" 200 OK 2026-03-26 11:00:46,691 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 11:00:46,707 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:54486 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50210 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50218 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33842 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33854 - "GET / HTTP/1.0" 200 OK 2026-03-26 11:01:24,575 - lava_callback - INFO - Received callback for node 69c4070cbbc022e792645c5c INFO: 172.21.0.1:49842 - "POST /node/69c4070cbbc022e792645c5c HTTP/1.0" 202 Accepted 2026-03-26 11:01:24,582 - lava_callback - INFO - Starting async job submit for node 69c4070cbbc022e792645c5c 2026-03-26 11:01:24,608 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 11:01:24,622 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4070cbbc022e792645c5c HTTP/1.1" 200 None 2026-03-26 11:01:24,623 - lava_callback - DEBUG - Processing node 69c4070cbbc022e792645c5c: baseline-nfs-arm64 2026-03-26 11:01:25,140 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 11:01:25,277 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 11:01:25,278 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c4070cbbc022e792645c5c/log.txt.gz 2026-03-26 11:01:25,548 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 11:01:25,775 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 11:01:25,777 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c4070cbbc022e792645c5c/lava-logs.yaml 2026-03-26 11:01:25,789 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 11:01:25,962 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 11:01:25,963 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c4070cbbc022e792645c5c/lava_callback.json.gz 2026-03-26 11:01:26,009 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 11:01:26,023 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4070cbbc022e792645c5c HTTP/1.1" 200 None 2026-03-26 11:01:26,025 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 11:01:26,038 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3295318a3717aa65d01c3 HTTP/1.1" 200 None 2026-03-26 11:01:26,040 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 11:01:26,060 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4070cbbc022e792645c5c HTTP/1.1" 200 None 2026-03-26 11:01:26,127 - lava_callback - INFO - Completed processing callback for node 69c4070cbbc022e792645c5c INFO: 172.21.0.1:49850 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59362 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48252 - "GET / HTTP/1.0" 200 OK 2026-03-26 11:01:46,709 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 11:01:46,726 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-26 11:01:50,505 - lava_callback - INFO - Received callback for node 69c50d68f63ce05331e2b466 INFO: 172.21.0.1:48328 - "POST /node/69c50d68f63ce05331e2b466 HTTP/1.0" 202 Accepted 2026-03-26 11:01:50,513 - lava_callback - INFO - Starting async job submit for node 69c50d68f63ce05331e2b466 2026-03-26 11:01:50,556 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 11:01:50,571 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c50d68f63ce05331e2b466 HTTP/1.1" 200 None 2026-03-26 11:01:50,571 - lava_callback - DEBUG - Processing node 69c50d68f63ce05331e2b466: baseline-nfs-arm64 2026-03-26 11:01:51,594 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 11:01:51,755 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 11:01:51,757 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c50d68f63ce05331e2b466/log.txt.gz 2026-03-26 11:01:52,343 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 11:01:52,573 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 11:01:52,576 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c50d68f63ce05331e2b466/lava-logs.yaml 2026-03-26 11:01:52,596 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 11:01:52,791 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 11:01:52,793 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c50d68f63ce05331e2b466/lava_callback.json.gz DEBUG: Setup failed for 69c50d68f63ce05331e2b466. Transitting job node result: incomplete -> fail 2026-03-26 11:01:52,883 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 11:01:52,898 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c50d68f63ce05331e2b466 HTTP/1.1" 200 None 2026-03-26 11:01:52,900 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 11:01:52,913 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebd8f63ce05331e2a492 HTTP/1.1" 200 None 2026-03-26 11:01:52,915 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 11:01:52,939 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c50d68f63ce05331e2b466 HTTP/1.1" 200 None 2026-03-26 11:01:53,043 - lava_callback - INFO - Completed processing callback for node 69c50d68f63ce05331e2b466 INFO: 172.21.0.1:48338 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33732 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33738 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43686 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43688 - "GET / HTTP/1.0" 200 OK 2026-03-26 11:02:16,728 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 11:02:16,747 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:51460 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41270 - "GET / HTTP/1.0" 200 OK 2026-03-26 11:02:43,864 - lava_callback - INFO - Received callback for node 69c50f52f63ce05331e2b4de INFO: 172.21.0.1:50876 - "POST /node/69c50f52f63ce05331e2b4de HTTP/1.0" 202 Accepted 2026-03-26 11:02:43,870 - lava_callback - INFO - Starting async job submit for node 69c50f52f63ce05331e2b4de 2026-03-26 11:02:43,889 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 11:02:43,903 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c50f52f63ce05331e2b4de HTTP/1.1" 200 None 2026-03-26 11:02:43,904 - lava_callback - DEBUG - Processing node 69c50f52f63ce05331e2b4de: baseline-arm64-broonie 2026-03-26 11:02:44,240 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 11:02:44,394 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 11:02:44,396 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c50f52f63ce05331e2b4de/log.txt.gz 2026-03-26 11:02:44,596 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 11:02:44,826 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 11:02:44,828 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c50f52f63ce05331e2b4de/lava-logs.yaml 2026-03-26 11:02:44,836 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 11:02:44,989 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 11:02:44,991 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c50f52f63ce05331e2b4de/lava_callback.json.gz 2026-03-26 11:02:45,028 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 11:02:45,043 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c50f52f63ce05331e2b4de HTTP/1.1" 200 None 2026-03-26 11:02:45,044 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 11:02:45,057 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ad9944faaf0e7f5e41d HTTP/1.1" 200 None 2026-03-26 11:02:45,059 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 11:02:45,088 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c50f52f63ce05331e2b4de HTTP/1.1" 200 None 2026-03-26 11:02:45,124 - lava_callback - INFO - Completed processing callback for node 69c50f52f63ce05331e2b4de INFO: 172.21.0.1:50888 - "GET / HTTP/1.0" 200 OK 2026-03-26 11:02:46,749 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 11:02:46,766 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:54864 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47928 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47932 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44484 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44496 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49582 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38812 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42394 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51008 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46342 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46344 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56686 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56688 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41516 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47264 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42210 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48302 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35610 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35618 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40584 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40596 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40704 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58356 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39164 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37502 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49956 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49970 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46834 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46836 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33250 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39672 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41654 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39320 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48696 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48708 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44034 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44038 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51550 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59708 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39954 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41020 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55128 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55132 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44572 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44588 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47836 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46240 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36916 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42268 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46682 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46696 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39452 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39466 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47636 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33608 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37396 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60826 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44114 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44128 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39018 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39024 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39212 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37372 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41986 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41466 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35868 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35878 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41672 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41688 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45248 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35990 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38730 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39580 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58624 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58630 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41244 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41256 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38962 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60618 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59018 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43518 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44826 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44830 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56562 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56574 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35584 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59148 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51936 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60504 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55590 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55598 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59716 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59726 - "GET / HTTP/1.0" 200 OK 2026-03-26 11:14:16,941 - lava_callback - INFO - Received callback for node 69c41193944faaf0e7f60df9 INFO: 172.21.0.1:59736 - "POST /node/69c41193944faaf0e7f60df9 HTTP/1.0" 202 Accepted 2026-03-26 11:14:16,952 - lava_callback - INFO - Starting async job submit for node 69c41193944faaf0e7f60df9 2026-03-26 11:14:16,981 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 11:14:16,997 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c41193944faaf0e7f60df9 HTTP/1.1" 200 None 2026-03-26 11:14:16,998 - lava_callback - DEBUG - Processing node 69c41193944faaf0e7f60df9: baseline-nfs-arm64 2026-03-26 11:14:17,532 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 11:14:17,686 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 11:14:17,687 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c41193944faaf0e7f60df9/log.txt.gz 2026-03-26 11:14:17,958 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 11:14:18,198 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 11:14:18,200 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c41193944faaf0e7f60df9/lava-logs.yaml 2026-03-26 11:14:18,213 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 11:14:18,373 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 11:14:18,374 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c41193944faaf0e7f60df9/lava_callback.json.gz 2026-03-26 11:14:18,401 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 11:14:18,415 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c41193944faaf0e7f60df9 HTTP/1.1" 200 None 2026-03-26 11:14:18,417 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 11:14:18,433 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ad9944faaf0e7f5e41d HTTP/1.1" 200 None 2026-03-26 11:14:18,435 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 11:14:18,461 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c41193944faaf0e7f60df9 HTTP/1.1" 200 None 2026-03-26 11:14:18,527 - lava_callback - INFO - Completed processing callback for node 69c41193944faaf0e7f60df9 INFO: 172.21.0.1:48400 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60096 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49076 - "GET / HTTP/1.0" 200 OK 2026-03-26 11:14:46,770 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 11:14:46,786 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:49188 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60982 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60994 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53464 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53468 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35974 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52642 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43902 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60574 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40520 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40526 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41242 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41256 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57550 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34990 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34944 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39746 - "GET / HTTP/1.0" 200 OK 2026-03-26 11:16:59,360 - lava_callback - INFO - Received callback for node 69c41182944faaf0e7f60dcb INFO: 172.21.0.1:39756 - "POST /node/69c41182944faaf0e7f60dcb HTTP/1.0" 202 Accepted 2026-03-26 11:16:59,366 - lava_callback - INFO - Starting async job submit for node 69c41182944faaf0e7f60dcb 2026-03-26 11:16:59,391 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 11:16:59,405 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c41182944faaf0e7f60dcb HTTP/1.1" 200 None 2026-03-26 11:16:59,406 - lava_callback - DEBUG - Processing node 69c41182944faaf0e7f60dcb: baseline-arm64 2026-03-26 11:16:59,780 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 11:16:59,972 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 11:16:59,974 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c41182944faaf0e7f60dcb/log.txt.gz 2026-03-26 11:17:00,183 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 11:17:00,429 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 11:17:00,431 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c41182944faaf0e7f60dcb/lava-logs.yaml 2026-03-26 11:17:00,440 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 11:17:00,609 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 11:17:00,611 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c41182944faaf0e7f60dcb/lava_callback.json.gz 2026-03-26 11:17:00,640 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 11:17:00,825 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c41182944faaf0e7f60dcb HTTP/1.1" 200 None 2026-03-26 11:17:00,827 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 11:17:00,840 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ad9944faaf0e7f5e41d HTTP/1.1" 200 None 2026-03-26 11:17:00,841 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 11:17:00,863 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c41182944faaf0e7f60dcb HTTP/1.1" 200 None 2026-03-26 11:17:00,929 - lava_callback - INFO - Completed processing callback for node 69c41182944faaf0e7f60dcb INFO: 172.21.0.1:33248 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33260 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59274 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59278 - "GET / HTTP/1.0" 200 OK 2026-03-26 11:17:16,788 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 11:17:16,807 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:54720 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51716 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33166 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57646 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46682 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46684 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39180 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39188 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53292 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58982 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49110 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51006 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44918 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44926 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59702 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59710 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38006 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58212 - "GET / HTTP/1.0" 200 OK 2026-03-26 11:19:42,972 - lava_callback - INFO - Received callback for node 69c51192f63ce05331e2b5a5 INFO: 172.21.0.1:41694 - "POST /node/69c51192f63ce05331e2b5a5 HTTP/1.0" 202 Accepted 2026-03-26 11:19:42,977 - lava_callback - INFO - Starting async job submit for node 69c51192f63ce05331e2b5a5 2026-03-26 11:19:42,993 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 11:19:43,008 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c51192f63ce05331e2b5a5 HTTP/1.1" 200 None 2026-03-26 11:19:43,008 - lava_callback - DEBUG - Processing node 69c51192f63ce05331e2b5a5: baseline-arm64-broonie 2026-03-26 11:19:43,277 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 11:19:43,436 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 11:19:43,437 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c51192f63ce05331e2b5a5/log.txt.gz 2026-03-26 11:19:43,599 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 11:19:43,815 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 11:19:43,817 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c51192f63ce05331e2b5a5/lava-logs.yaml 2026-03-26 11:19:43,824 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 11:19:43,952 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 11:19:43,954 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c51192f63ce05331e2b5a5/lava_callback.json.gz 2026-03-26 11:19:43,970 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 11:19:43,996 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c51192f63ce05331e2b5a5 HTTP/1.1" 200 None 2026-03-26 11:19:43,998 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 11:19:44,011 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ad9944faaf0e7f5e41d HTTP/1.1" 200 None 2026-03-26 11:19:44,013 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 11:19:44,057 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c51192f63ce05331e2b5a5 HTTP/1.1" 200 None 2026-03-26 11:19:44,059 - lava_callback - INFO - Completed processing callback for node 69c51192f63ce05331e2b5a5 INFO: 172.21.0.1:41704 - "GET / HTTP/1.0" 200 OK 2026-03-26 11:19:46,809 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 11:19:46,825 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:52112 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41326 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41340 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44252 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44258 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40354 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53856 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33768 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49190 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48286 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48294 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45478 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45486 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41384 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57408 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55480 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60104 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49604 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49612 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57848 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57862 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59620 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36882 - "GET / HTTP/1.0" 200 OK 2026-03-26 11:22:43,028 - lava_callback - INFO - Received callback for node 69c5122cf63ce05331e2b5cc INFO: 172.21.0.1:58664 - "POST /node/69c5122cf63ce05331e2b5cc HTTP/1.0" 202 Accepted 2026-03-26 11:22:43,034 - lava_callback - INFO - Starting async job submit for node 69c5122cf63ce05331e2b5cc 2026-03-26 11:22:43,052 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 11:22:43,067 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5122cf63ce05331e2b5cc HTTP/1.1" 200 None 2026-03-26 11:22:43,068 - lava_callback - DEBUG - Processing node 69c5122cf63ce05331e2b5cc: baseline-arm64-broonie 2026-03-26 11:22:43,410 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 11:22:43,586 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 11:22:43,588 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c5122cf63ce05331e2b5cc/log.txt.gz 2026-03-26 11:22:43,785 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 11:22:44,035 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 11:22:44,038 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c5122cf63ce05331e2b5cc/lava-logs.yaml 2026-03-26 11:22:44,046 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 11:22:44,345 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 11:22:44,346 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c5122cf63ce05331e2b5cc/lava_callback.json.gz 2026-03-26 11:22:44,383 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 11:22:44,397 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5122cf63ce05331e2b5cc HTTP/1.1" 200 None 2026-03-26 11:22:44,398 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 11:22:44,411 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b4a590270ad7b6d699d HTTP/1.1" 200 None 2026-03-26 11:22:44,413 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 11:22:44,436 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5122cf63ce05331e2b5cc HTTP/1.1" 200 None 2026-03-26 11:22:44,469 - lava_callback - INFO - Completed processing callback for node 69c5122cf63ce05331e2b5cc INFO: 172.21.0.1:58674 - "GET / HTTP/1.0" 200 OK 2026-03-26 11:22:46,828 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 11:22:46,844 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:35950 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58922 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58936 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40654 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40658 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46892 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60752 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47226 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59142 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44368 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44374 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36142 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36156 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54146 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45428 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59260 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33410 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53194 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53206 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60294 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60296 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40594 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50458 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36186 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40896 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38532 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38534 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57878 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57890 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45590 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51450 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37114 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36920 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40942 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40946 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58804 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58820 - "GET / HTTP/1.0" 200 OK 2026-03-26 11:27:23,356 - lava_callback - INFO - Received callback for node 69c5175df63ce05331e2b677 INFO: 172.21.0.1:53790 - "POST /node/69c5175df63ce05331e2b677 HTTP/1.0" 202 Accepted 2026-03-26 11:27:23,362 - lava_callback - INFO - Starting async job submit for node 69c5175df63ce05331e2b677 2026-03-26 11:27:23,380 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 11:27:23,400 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5175df63ce05331e2b677 HTTP/1.1" 200 None 2026-03-26 11:27:23,400 - lava_callback - DEBUG - Processing node 69c5175df63ce05331e2b677: baseline-arm64-broonie 2026-03-26 11:27:23,712 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 11:27:23,846 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 11:27:23,848 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c5175df63ce05331e2b677/log.txt.gz 2026-03-26 11:27:24,042 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 11:27:24,268 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 11:27:24,270 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c5175df63ce05331e2b677/lava-logs.yaml 2026-03-26 11:27:24,278 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 11:27:24,462 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 11:27:24,464 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c5175df63ce05331e2b677/lava_callback.json.gz 2026-03-26 11:27:24,502 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 11:27:24,515 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5175df63ce05331e2b677 HTTP/1.1" 200 None 2026-03-26 11:27:24,517 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 11:27:24,530 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b4a590270ad7b6d699d HTTP/1.1" 200 None 2026-03-26 11:27:24,531 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 11:27:24,555 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5175df63ce05331e2b677 HTTP/1.1" 200 None 2026-03-26 11:27:24,590 - lava_callback - INFO - Completed processing callback for node 69c5175df63ce05331e2b677 INFO: 172.21.0.1:53804 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45984 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41184 - "GET / HTTP/1.0" 200 OK 2026-03-26 11:27:46,848 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 11:27:46,873 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:54414 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60766 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60768 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60534 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60548 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45902 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50464 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55820 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49896 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49638 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49654 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56866 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56878 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46564 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41022 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40932 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60928 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51160 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51172 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60402 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60414 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42378 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46106 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35986 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52398 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46782 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46794 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39422 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39436 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49288 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34526 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51170 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59900 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50794 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50802 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34552 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34556 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56452 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54834 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56120 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46144 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55230 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55246 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43502 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43506 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45152 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48150 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56256 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35750 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34014 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34022 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47556 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47566 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36658 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59634 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38354 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56622 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55868 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55872 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51744 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51754 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42526 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55052 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51992 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50812 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50440 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50444 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33320 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33328 - "GET / HTTP/1.0" 200 OK 2026-03-26 11:36:23,724 - lava_callback - INFO - Received callback for node 69c51039f63ce05331e2b521 INFO: 172.21.0.1:40692 - "POST /node/69c51039f63ce05331e2b521 HTTP/1.0" 202 Accepted 2026-03-26 11:36:23,731 - lava_callback - INFO - Starting async job submit for node 69c51039f63ce05331e2b521 2026-03-26 11:36:23,750 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 11:36:23,768 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c51039f63ce05331e2b521 HTTP/1.1" 200 None 2026-03-26 11:36:23,769 - lava_callback - DEBUG - Processing node 69c51039f63ce05331e2b521: baseline-arm64-broonie 2026-03-26 11:36:23,969 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 11:36:24,120 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 11:36:24,122 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c51039f63ce05331e2b521/log.txt.gz 2026-03-26 11:36:24,273 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 11:36:24,517 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 11:36:24,519 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c51039f63ce05331e2b521/lava-logs.yaml 2026-03-26 11:36:24,523 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 11:36:24,659 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 11:36:24,661 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c51039f63ce05331e2b521/lava_callback.json.gz 2026-03-26 11:36:24,681 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 11:36:24,696 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c51039f63ce05331e2b521 HTTP/1.1" 200 None 2026-03-26 11:36:24,698 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 11:36:24,712 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b4a590270ad7b6d699d HTTP/1.1" 200 None 2026-03-26 11:36:24,714 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 11:36:24,757 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c51039f63ce05331e2b521 HTTP/1.1" 200 None 2026-03-26 11:36:24,804 - lava_callback - INFO - Completed processing callback for node 69c51039f63ce05331e2b521 INFO: 172.21.0.1:40702 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33048 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42780 - "GET / HTTP/1.0" 200 OK 2026-03-26 11:36:46,877 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 11:36:46,893 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:49292 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35042 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35056 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44732 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44748 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47288 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43842 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54152 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48286 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57834 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57842 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40550 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40564 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48772 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51518 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39462 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43116 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45390 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45394 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32898 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:32910 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40064 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55236 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43392 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58078 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48074 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48086 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36422 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36432 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54906 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59156 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49722 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44454 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36002 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36018 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40198 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40206 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53900 - "GET / HTTP/1.0" 200 OK 2026-03-26 11:41:27,191 - lava_callback - INFO - Received callback for node 69c42a37944faaf0e7f61511 INFO: 172.21.0.1:53904 - "POST /node/69c42a37944faaf0e7f61511 HTTP/1.0" 202 Accepted 2026-03-26 11:41:27,197 - lava_callback - INFO - Starting async job submit for node 69c42a37944faaf0e7f61511 2026-03-26 11:41:27,222 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 11:41:27,263 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c42a37944faaf0e7f61511 HTTP/1.1" 200 None 2026-03-26 11:41:27,264 - lava_callback - DEBUG - Processing node 69c42a37944faaf0e7f61511: baseline-arm64 2026-03-26 11:41:27,787 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 11:41:27,937 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 11:41:27,940 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c42a37944faaf0e7f61511/log.txt.gz 2026-03-26 11:41:28,204 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 11:41:28,428 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 11:41:28,430 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c42a37944faaf0e7f61511/lava-logs.yaml 2026-03-26 11:41:28,441 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 11:41:28,607 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 11:41:28,609 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c42a37944faaf0e7f61511/lava_callback.json.gz 2026-03-26 11:41:28,634 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 11:41:28,648 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c42a37944faaf0e7f61511 HTTP/1.1" 200 None 2026-03-26 11:41:28,650 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 11:41:28,687 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399cdbbc022e792643ea0 HTTP/1.1" 200 None 2026-03-26 11:41:28,689 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 11:41:28,715 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c42a37944faaf0e7f61511 HTTP/1.1" 200 None 2026-03-26 11:41:28,763 - lava_callback - INFO - Completed processing callback for node 69c42a37944faaf0e7f61511 INFO: 172.21.0.1:43434 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57752 - "GET / HTTP/1.0" 200 OK 2026-03-26 11:41:46,896 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 11:41:46,936 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:49416 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38244 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38254 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56652 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56654 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43788 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47770 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39088 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38684 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47330 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47342 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57340 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57356 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55872 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44566 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53384 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51450 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56882 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56884 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41192 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41208 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55876 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58948 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50306 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43272 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43294 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43298 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33874 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33890 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49738 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60308 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57194 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35874 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43776 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43780 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60726 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60732 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60488 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41118 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56182 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55030 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45864 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45868 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57882 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57890 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49536 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47860 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36270 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40978 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40078 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40080 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51546 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51552 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43424 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48616 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45504 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51696 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60282 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60288 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54830 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54846 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46340 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34440 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37926 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38476 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50812 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50824 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42576 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42582 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41952 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52254 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53012 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60970 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58644 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58652 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34770 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34774 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42694 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39854 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51078 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49290 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52230 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52232 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36854 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36866 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52416 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59446 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50096 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60384 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44560 - "GET / HTTP/1.0" 200 OK 2026-03-26 11:53:04,063 - lava_callback - INFO - Received callback for node 69c42d70944faaf0e7f615e3 INFO: 172.21.0.1:44570 - "POST /node/69c42d70944faaf0e7f615e3 HTTP/1.0" 202 Accepted 2026-03-26 11:53:04,070 - lava_callback - INFO - Starting async job submit for node 69c42d70944faaf0e7f615e3 2026-03-26 11:53:04,096 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 11:53:04,136 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c42d70944faaf0e7f615e3 HTTP/1.1" 200 None 2026-03-26 11:53:04,137 - lava_callback - DEBUG - Processing node 69c42d70944faaf0e7f615e3: baseline-nfs-arm64 2026-03-26 11:53:04,686 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 11:53:04,854 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 11:53:04,855 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c42d70944faaf0e7f615e3/log.txt.gz 2026-03-26 11:53:05,120 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 11:53:05,331 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 11:53:05,334 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c42d70944faaf0e7f615e3/lava-logs.yaml 2026-03-26 11:53:05,345 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 11:53:05,507 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 11:53:05,509 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c42d70944faaf0e7f615e3/lava_callback.json.gz 2026-03-26 11:53:05,536 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 11:53:05,572 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c42d70944faaf0e7f615e3 HTTP/1.1" 200 None 2026-03-26 11:53:05,573 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 11:53:05,598 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399cdbbc022e792643ea0 HTTP/1.1" 200 None 2026-03-26 11:53:05,600 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 11:53:05,629 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c42d70944faaf0e7f615e3 HTTP/1.1" 200 None 2026-03-26 11:53:05,674 - lava_callback - INFO - Completed processing callback for node 69c42d70944faaf0e7f615e3 INFO: 172.21.0.1:44574 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59536 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59548 - "GET / HTTP/1.0" 200 OK 2026-03-26 11:53:16,941 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 11:53:16,972 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:49880 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43152 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38038 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57998 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35342 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35346 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55016 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55026 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48048 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58410 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41564 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39138 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40922 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40930 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47050 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47066 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48176 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60068 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54028 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33318 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47666 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47670 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33216 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33226 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37116 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36730 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48114 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43192 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47616 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47626 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39340 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39348 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51358 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35256 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52200 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57360 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51038 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51044 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57948 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57964 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57320 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45576 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48892 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45284 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41414 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41422 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55000 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55012 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36062 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34226 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54008 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34874 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57792 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57804 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37896 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37906 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50718 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51614 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48732 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39994 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42494 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42496 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38062 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38070 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58304 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58860 - "GET / HTTP/1.0" 200 OK 2026-03-26 12:01:45,653 - lava_callback - INFO - Received callback for node 69c51a4cf63ce05331e2b6cd INFO: 172.21.0.1:49392 - "POST /node/69c51a4cf63ce05331e2b6cd HTTP/1.0" 202 Accepted 2026-03-26 12:01:45,659 - lava_callback - INFO - Starting async job submit for node 69c51a4cf63ce05331e2b6cd 2026-03-26 12:01:45,683 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 12:01:45,717 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c51a4cf63ce05331e2b6cd HTTP/1.1" 200 None 2026-03-26 12:01:45,719 - lava_callback - DEBUG - Processing node 69c51a4cf63ce05331e2b6cd: baseline-arm64-broonie 2026-03-26 12:01:46,074 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 12:01:46,230 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 12:01:46,232 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c51a4cf63ce05331e2b6cd/log.txt.gz 2026-03-26 12:01:46,430 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.21.0.1:49398 - "GET / HTTP/1.0" 200 OK 2026-03-26 12:01:46,810 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 12:01:46,813 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c51a4cf63ce05331e2b6cd/lava-logs.yaml 2026-03-26 12:01:46,822 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 12:01:46,982 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 12:01:46,984 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c51a4cf63ce05331e2b6cd/lava_callback.json.gz 2026-03-26 12:01:47,031 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 12:01:47,048 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c51a4cf63ce05331e2b6cd HTTP/1.1" 200 None 2026-03-26 12:01:47,050 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 12:01:47,073 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b4a590270ad7b6d699d HTTP/1.1" 200 None 2026-03-26 12:01:47,075 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 12:01:47,100 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c51a4cf63ce05331e2b6cd HTTP/1.1" 200 None 2026-03-26 12:01:47,143 - lava_callback - INFO - Completed processing callback for node 69c51a4cf63ce05331e2b6cd INFO: 172.21.0.1:49938 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37020 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37028 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57966 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57976 - "GET / HTTP/1.0" 200 OK 2026-03-26 12:02:16,976 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 12:02:16,993 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:38118 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36710 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41802 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34596 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46268 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46272 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48356 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48370 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49348 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58314 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58590 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58892 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41244 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41256 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32938 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:32950 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41064 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39588 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50396 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59952 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51958 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51972 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51928 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51930 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52124 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52658 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59552 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40562 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51706 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51710 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47886 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47898 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50144 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57264 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43512 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39550 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48336 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48346 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33884 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33894 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48146 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44932 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33220 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35374 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50564 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50572 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47368 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47384 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39380 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39318 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43088 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53722 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39786 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39802 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37250 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37262 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36910 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47512 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54242 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55042 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43672 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43686 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56970 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56978 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52708 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39700 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39528 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39598 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39154 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39162 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34488 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34496 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33742 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48290 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37146 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33664 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54246 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54258 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51878 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51880 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34626 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43314 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40434 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43502 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37310 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37326 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49908 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49922 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44592 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46552 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51818 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50170 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43790 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43792 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36926 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36936 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41050 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51824 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42074 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59006 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44702 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44708 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41100 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41106 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60288 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55160 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42900 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52860 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49818 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49820 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42914 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42926 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57180 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58648 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56028 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44406 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38866 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38870 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46186 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46202 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34124 - "GET / HTTP/1.0" 200 OK 2026-03-26 12:17:27,167 - lava_callback - INFO - Received callback for node 69c280d44a2c5046eb880a20 INFO: 172.21.0.1:34140 - "POST /node/69c280d44a2c5046eb880a20 HTTP/1.0" 202 Accepted 2026-03-26 12:17:27,172 - lava_callback - INFO - Starting async job submit for node 69c280d44a2c5046eb880a20 2026-03-26 12:17:27,173 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 12:17:27,188 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c280d44a2c5046eb880a20 HTTP/1.1" 200 None 2026-03-26 12:17:27,188 - lava_callback - DEBUG - Processing node 69c280d44a2c5046eb880a20: baseline-arm64-baylibre 2026-03-26 12:17:27,189 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c280d44a2c5046eb880a20 2026-03-26 12:17:27,189 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 12:17:27,203 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c280d44a2c5046eb880a20 HTTP/1.1" 200 None 2026-03-26 12:17:27,204 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 12:17:27,218 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1679b2b2b51e9ad977955 HTTP/1.1" 200 None 2026-03-26 12:17:27,219 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 12:17:27,240 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c280d44a2c5046eb880a20 HTTP/1.1" 200 None 2026-03-26 12:17:27,246 - lava_callback - ERROR - Error processing callback for node 69c280d44a2c5046eb880a20: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:37316 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55512 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49964 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51652 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51662 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36112 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36124 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46480 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32968 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33820 - "GET / HTTP/1.0" 200 OK 2026-03-26 12:18:47,546 - lava_callback - INFO - Received callback for node 69c281234a2c5046eb880a28 INFO: 172.21.0.1:33826 - "POST /node/69c281234a2c5046eb880a28 HTTP/1.0" 202 Accepted 2026-03-26 12:18:47,552 - lava_callback - INFO - Starting async job submit for node 69c281234a2c5046eb880a28 2026-03-26 12:18:47,552 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 12:18:47,588 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c281234a2c5046eb880a28 HTTP/1.1" 200 None 2026-03-26 12:18:47,589 - lava_callback - DEBUG - Processing node 69c281234a2c5046eb880a28: baseline-arm64-baylibre 2026-03-26 12:18:47,589 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c281234a2c5046eb880a28 2026-03-26 12:18:47,590 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 12:18:47,604 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c281234a2c5046eb880a28 HTTP/1.1" 200 None 2026-03-26 12:18:47,605 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 12:18:47,623 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1679b2b2b51e9ad977955 HTTP/1.1" 200 None 2026-03-26 12:18:47,625 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 12:18:47,646 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c281234a2c5046eb880a28 HTTP/1.1" 200 None 2026-03-26 12:18:47,647 - lava_callback - ERROR - Error processing callback for node 69c281234a2c5046eb880a28: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:54958 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39608 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39618 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52114 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52120 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53540 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36750 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57502 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55334 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33298 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33300 - "GET / HTTP/1.0" 200 OK 2026-03-26 12:20:07,326 - lava_callback - INFO - Received callback for node 69c281734a2c5046eb880a35 INFO: 172.21.0.1:33314 - "POST /node/69c281734a2c5046eb880a35 HTTP/1.0" 202 Accepted 2026-03-26 12:20:07,332 - lava_callback - INFO - Starting async job submit for node 69c281734a2c5046eb880a35 2026-03-26 12:20:07,333 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 12:20:07,347 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c281734a2c5046eb880a35 HTTP/1.1" 200 None 2026-03-26 12:20:07,348 - lava_callback - DEBUG - Processing node 69c281734a2c5046eb880a35: baseline-arm64-baylibre 2026-03-26 12:20:07,349 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c281734a2c5046eb880a35 2026-03-26 12:20:07,349 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 12:20:07,383 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c281734a2c5046eb880a35 HTTP/1.1" 200 None 2026-03-26 12:20:07,384 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 12:20:07,408 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1679b2b2b51e9ad977955 HTTP/1.1" 200 None 2026-03-26 12:20:07,409 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 12:20:07,430 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c281734a2c5046eb880a35 HTTP/1.1" 200 None 2026-03-26 12:20:07,432 - lava_callback - ERROR - Error processing callback for node 69c281734a2c5046eb880a35: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:59220 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59232 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47474 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45048 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41716 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35134 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55188 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55204 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50922 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50936 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39030 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56752 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60504 - "GET / HTTP/1.0" 200 OK 2026-03-26 12:21:47,336 - lava_callback - INFO - Received callback for node 69c281c64a2c5046eb880a41 INFO: 172.21.0.1:60516 - "POST /node/69c281c64a2c5046eb880a41 HTTP/1.0" 202 Accepted 2026-03-26 12:21:47,341 - lava_callback - INFO - Starting async job submit for node 69c281c64a2c5046eb880a41 2026-03-26 12:21:47,342 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 12:21:47,357 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c281c64a2c5046eb880a41 HTTP/1.1" 200 None 2026-03-26 12:21:47,358 - lava_callback - DEBUG - Processing node 69c281c64a2c5046eb880a41: baseline-arm64-baylibre 2026-03-26 12:21:47,358 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c281c64a2c5046eb880a41 2026-03-26 12:21:47,359 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 12:21:47,379 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c281c64a2c5046eb880a41 HTTP/1.1" 200 None 2026-03-26 12:21:47,381 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 12:21:47,395 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1679b2b2b51e9ad977955 HTTP/1.1" 200 None 2026-03-26 12:21:47,396 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 12:21:47,417 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c281c64a2c5046eb880a41 HTTP/1.1" 200 None 2026-03-26 12:21:47,422 - lava_callback - ERROR - Error processing callback for node 69c281c64a2c5046eb880a41: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:47402 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60074 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60080 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34940 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34952 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56434 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55344 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57262 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43512 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52016 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52022 - "GET / HTTP/1.0" 200 OK 2026-03-26 12:23:07,786 - lava_callback - INFO - Received callback for node 69c282164a2c5046eb880a49 INFO: 172.21.0.1:52034 - "POST /node/69c282164a2c5046eb880a49 HTTP/1.0" 202 Accepted 2026-03-26 12:23:07,791 - lava_callback - INFO - Starting async job submit for node 69c282164a2c5046eb880a49 2026-03-26 12:23:07,792 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 12:23:07,831 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c282164a2c5046eb880a49 HTTP/1.1" 200 None 2026-03-26 12:23:07,832 - lava_callback - DEBUG - Processing node 69c282164a2c5046eb880a49: baseline-arm64-baylibre 2026-03-26 12:23:07,833 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c282164a2c5046eb880a49 2026-03-26 12:23:07,833 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 12:23:07,851 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c282164a2c5046eb880a49 HTTP/1.1" 200 None 2026-03-26 12:23:07,853 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 12:23:07,868 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1679b2b2b51e9ad977955 HTTP/1.1" 200 None 2026-03-26 12:23:07,869 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 12:23:07,889 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c282164a2c5046eb880a49 HTTP/1.1" 200 None 2026-03-26 12:23:07,891 - lava_callback - ERROR - Error processing callback for node 69c282164a2c5046eb880a49: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:43472 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43478 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55080 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35122 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54010 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34188 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54904 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54920 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39838 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39842 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45084 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40996 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60942 - "GET / HTTP/1.0" 200 OK 2026-03-26 12:24:47,506 - lava_callback - INFO - Received callback for node 69c2827b4a2c5046eb880a58 INFO: 172.21.0.1:60956 - "POST /node/69c2827b4a2c5046eb880a58 HTTP/1.0" 202 Accepted 2026-03-26 12:24:47,512 - lava_callback - INFO - Starting async job submit for node 69c2827b4a2c5046eb880a58 2026-03-26 12:24:47,513 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 12:24:47,536 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2827b4a2c5046eb880a58 HTTP/1.1" 200 None 2026-03-26 12:24:47,537 - lava_callback - DEBUG - Processing node 69c2827b4a2c5046eb880a58: baseline-arm64-baylibre 2026-03-26 12:24:47,537 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2827b4a2c5046eb880a58 2026-03-26 12:24:47,538 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 12:24:47,559 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2827b4a2c5046eb880a58 HTTP/1.1" 200 None 2026-03-26 12:24:47,561 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 12:24:47,574 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1679b2b2b51e9ad977955 HTTP/1.1" 200 None 2026-03-26 12:24:47,576 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 12:24:47,768 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2827b4a2c5046eb880a58 HTTP/1.1" 200 None 2026-03-26 12:24:47,769 - lava_callback - ERROR - Error processing callback for node 69c2827b4a2c5046eb880a58: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:45002 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34210 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34216 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59740 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59744 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59034 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46030 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36402 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49068 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50702 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50716 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55846 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55860 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51146 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42366 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50038 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53150 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39628 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39630 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36654 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36666 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48924 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44476 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51170 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37386 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35350 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35364 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46534 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46544 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50504 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48202 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34554 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57270 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54972 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54980 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57404 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57412 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57762 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54552 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38132 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44212 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52830 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52834 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60562 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60568 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49468 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36566 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51918 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41298 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38638 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38642 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57558 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57566 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34646 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39428 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54408 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48024 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35488 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35496 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44074 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44090 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57266 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39060 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41506 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53184 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42358 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42368 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52520 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52524 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42612 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59184 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41526 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43444 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56014 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56028 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34700 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34716 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47038 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41356 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33096 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44788 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46756 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46770 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48438 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48446 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37084 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55688 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34956 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58526 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40450 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40462 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45436 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45450 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34486 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51904 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44696 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40970 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47242 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47252 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34210 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34222 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56810 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59534 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51108 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49378 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47306 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47308 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42836 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42840 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41116 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32868 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57136 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44684 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42784 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42788 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39908 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39912 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42578 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37644 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44518 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34866 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54456 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54462 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52124 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52140 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56856 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39486 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45592 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39490 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33246 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33254 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40006 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40014 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50718 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44734 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59254 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51364 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59858 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59864 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57088 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57096 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56540 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39774 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60486 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46432 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57940 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57944 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51054 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51070 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58728 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37648 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48628 - "GET / HTTP/1.0" 200 OK 2026-03-26 12:43:49,464 - lava_callback - INFO - Received callback for node 69c286f44a2c5046eb880a91 INFO: 172.21.0.1:48636 - "POST /node/69c286f44a2c5046eb880a91 HTTP/1.0" 202 Accepted 2026-03-26 12:43:49,469 - lava_callback - INFO - Starting async job submit for node 69c286f44a2c5046eb880a91 2026-03-26 12:43:49,470 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 12:43:49,484 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c286f44a2c5046eb880a91 HTTP/1.1" 200 None 2026-03-26 12:43:49,485 - lava_callback - DEBUG - Processing node 69c286f44a2c5046eb880a91: baseline-arm-clabbe 2026-03-26 12:43:49,485 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c286f44a2c5046eb880a91 2026-03-26 12:43:49,486 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 12:43:49,499 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c286f44a2c5046eb880a91 HTTP/1.1" 200 None 2026-03-26 12:43:49,500 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 12:43:49,513 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d82b0efaf21d93103543 HTTP/1.1" 200 None 2026-03-26 12:43:49,514 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 12:43:49,534 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c286f44a2c5046eb880a91 HTTP/1.1" 200 None 2026-03-26 12:43:49,544 - lava_callback - ERROR - Error processing callback for node 69c286f44a2c5046eb880a91: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:49364 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47676 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47690 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34572 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34574 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48358 - "GET / HTTP/1.0" 200 OK 2026-03-26 12:44:29,400 - lava_callback - INFO - Received callback for node 69c287244a2c5046eb880a9f INFO: 172.21.0.1:48368 - "POST /node/69c287244a2c5046eb880a9f HTTP/1.0" 202 Accepted 2026-03-26 12:44:29,406 - lava_callback - INFO - Starting async job submit for node 69c287244a2c5046eb880a9f 2026-03-26 12:44:29,406 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 12:44:29,420 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c287244a2c5046eb880a9f HTTP/1.1" 200 None 2026-03-26 12:44:29,421 - lava_callback - DEBUG - Processing node 69c287244a2c5046eb880a9f: baseline-arm-clabbe 2026-03-26 12:44:29,421 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c287244a2c5046eb880a9f 2026-03-26 12:44:29,422 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 12:44:29,434 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c287244a2c5046eb880a9f HTTP/1.1" 200 None 2026-03-26 12:44:29,436 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 12:44:29,449 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d82b0efaf21d93103543 HTTP/1.1" 200 None 2026-03-26 12:44:29,451 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 12:44:29,483 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c287244a2c5046eb880a9f HTTP/1.1" 200 None 2026-03-26 12:44:29,489 - lava_callback - ERROR - Error processing callback for node 69c287244a2c5046eb880a9f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:36276 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32950 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41602 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57956 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57966 - "GET / HTTP/1.0" 200 OK 2026-03-26 12:45:09,548 - lava_callback - INFO - Received callback for node 69c287494a2c5046eb880aaa INFO: 172.21.0.1:57982 - "POST /node/69c287494a2c5046eb880aaa HTTP/1.0" 202 Accepted 2026-03-26 12:45:09,553 - lava_callback - INFO - Starting async job submit for node 69c287494a2c5046eb880aaa 2026-03-26 12:45:09,554 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 12:45:09,568 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c287494a2c5046eb880aaa HTTP/1.1" 200 None 2026-03-26 12:45:09,568 - lava_callback - DEBUG - Processing node 69c287494a2c5046eb880aaa: baseline-arm-clabbe 2026-03-26 12:45:09,569 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c287494a2c5046eb880aaa 2026-03-26 12:45:09,569 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 12:45:09,583 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c287494a2c5046eb880aaa HTTP/1.1" 200 None 2026-03-26 12:45:09,584 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 12:45:09,597 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d82b0efaf21d93103543 HTTP/1.1" 200 None 2026-03-26 12:45:09,599 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 12:45:09,626 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c287494a2c5046eb880aaa HTTP/1.1" 200 None 2026-03-26 12:45:09,627 - lava_callback - ERROR - Error processing callback for node 69c287494a2c5046eb880aaa: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:37132 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37138 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34662 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43180 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40996 - "GET / HTTP/1.0" 200 OK 2026-03-26 12:45:49,461 - lava_callback - INFO - Received callback for node 69c2876d4a2c5046eb880ab7 INFO: 172.21.0.1:41012 - "POST /node/69c2876d4a2c5046eb880ab7 HTTP/1.0" 202 Accepted 2026-03-26 12:45:49,467 - lava_callback - INFO - Starting async job submit for node 69c2876d4a2c5046eb880ab7 2026-03-26 12:45:49,468 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 12:45:49,482 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2876d4a2c5046eb880ab7 HTTP/1.1" 200 None 2026-03-26 12:45:49,483 - lava_callback - DEBUG - Processing node 69c2876d4a2c5046eb880ab7: baseline-arm-clabbe 2026-03-26 12:45:49,483 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2876d4a2c5046eb880ab7 2026-03-26 12:45:49,484 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 12:45:49,498 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2876d4a2c5046eb880ab7 HTTP/1.1" 200 None 2026-03-26 12:45:49,499 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 12:45:49,517 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d82b0efaf21d93103543 HTTP/1.1" 200 None 2026-03-26 12:45:49,518 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 12:45:49,540 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2876d4a2c5046eb880ab7 HTTP/1.1" 200 None 2026-03-26 12:45:49,547 - lava_callback - ERROR - Error processing callback for node 69c2876d4a2c5046eb880ab7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:59828 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35648 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35652 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36058 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36060 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42178 - "GET / HTTP/1.0" 200 OK 2026-03-26 12:46:29,559 - lava_callback - INFO - Received callback for node 69c287934a2c5046eb880ac0 INFO: 172.21.0.1:42194 - "POST /node/69c287934a2c5046eb880ac0 HTTP/1.0" 202 Accepted 2026-03-26 12:46:29,565 - lava_callback - INFO - Starting async job submit for node 69c287934a2c5046eb880ac0 2026-03-26 12:46:29,566 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 12:46:29,580 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c287934a2c5046eb880ac0 HTTP/1.1" 200 None 2026-03-26 12:46:29,581 - lava_callback - DEBUG - Processing node 69c287934a2c5046eb880ac0: baseline-arm-clabbe 2026-03-26 12:46:29,582 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c287934a2c5046eb880ac0 2026-03-26 12:46:29,582 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 12:46:29,595 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c287934a2c5046eb880ac0 HTTP/1.1" 200 None 2026-03-26 12:46:29,597 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 12:46:29,611 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d82b0efaf21d93103543 HTTP/1.1" 200 None 2026-03-26 12:46:29,612 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 12:46:29,633 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c287934a2c5046eb880ac0 HTTP/1.1" 200 None 2026-03-26 12:46:29,638 - lava_callback - ERROR - Error processing callback for node 69c287934a2c5046eb880ac0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:56996 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37502 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59034 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47360 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47366 - "GET / HTTP/1.0" 200 OK 2026-03-26 12:47:09,504 - lava_callback - INFO - Received callback for node 69c287b74a2c5046eb880ace INFO: 172.21.0.1:47370 - "POST /node/69c287b74a2c5046eb880ace HTTP/1.0" 202 Accepted 2026-03-26 12:47:09,509 - lava_callback - INFO - Starting async job submit for node 69c287b74a2c5046eb880ace 2026-03-26 12:47:09,510 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 12:47:09,525 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c287b74a2c5046eb880ace HTTP/1.1" 200 None 2026-03-26 12:47:09,525 - lava_callback - DEBUG - Processing node 69c287b74a2c5046eb880ace: baseline-arm-clabbe 2026-03-26 12:47:09,526 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c287b74a2c5046eb880ace 2026-03-26 12:47:09,526 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 12:47:09,540 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c287b74a2c5046eb880ace HTTP/1.1" 200 None 2026-03-26 12:47:09,542 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 12:47:09,555 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf353af1004e82292e0db7 HTTP/1.1" 200 None 2026-03-26 12:47:09,557 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 12:47:09,580 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c287b74a2c5046eb880ace HTTP/1.1" 200 None 2026-03-26 12:47:09,581 - lava_callback - ERROR - Error processing callback for node 69c287b74a2c5046eb880ace: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:52890 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52906 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59040 - "GET / HTTP/1.0" 200 OK 2026-03-26 12:47:29,528 - lava_callback - INFO - Received callback for node 69c287dc4a2c5046eb880ada INFO: 172.21.0.1:59050 - "POST /node/69c287dc4a2c5046eb880ada HTTP/1.0" 202 Accepted 2026-03-26 12:47:29,533 - lava_callback - INFO - Starting async job submit for node 69c287dc4a2c5046eb880ada 2026-03-26 12:47:29,534 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 12:47:29,549 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c287dc4a2c5046eb880ada HTTP/1.1" 200 None 2026-03-26 12:47:29,550 - lava_callback - DEBUG - Processing node 69c287dc4a2c5046eb880ada: baseline-arm-clabbe 2026-03-26 12:47:29,550 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c287dc4a2c5046eb880ada 2026-03-26 12:47:29,551 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 12:47:29,564 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c287dc4a2c5046eb880ada HTTP/1.1" 200 None 2026-03-26 12:47:29,566 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 12:47:29,579 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf353af1004e82292e0db7 HTTP/1.1" 200 None 2026-03-26 12:47:29,581 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 12:47:29,609 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c287dc4a2c5046eb880ada HTTP/1.1" 200 None 2026-03-26 12:47:29,610 - lava_callback - ERROR - Error processing callback for node 69c287dc4a2c5046eb880ada: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:58104 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55812 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53710 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58076 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58090 - "GET / HTTP/1.0" 200 OK 2026-03-26 12:48:09,580 - lava_callback - INFO - Received callback for node 69c288024a2c5046eb880ae7 INFO: 172.21.0.1:58096 - "POST /node/69c288024a2c5046eb880ae7 HTTP/1.0" 202 Accepted 2026-03-26 12:48:09,586 - lava_callback - INFO - Starting async job submit for node 69c288024a2c5046eb880ae7 2026-03-26 12:48:09,587 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 12:48:09,603 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c288024a2c5046eb880ae7 HTTP/1.1" 200 None 2026-03-26 12:48:09,604 - lava_callback - DEBUG - Processing node 69c288024a2c5046eb880ae7: baseline-arm-clabbe 2026-03-26 12:48:09,604 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c288024a2c5046eb880ae7 2026-03-26 12:48:09,605 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 12:48:09,619 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c288024a2c5046eb880ae7 HTTP/1.1" 200 None 2026-03-26 12:48:09,621 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 12:48:09,634 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf353af1004e82292e0db7 HTTP/1.1" 200 None 2026-03-26 12:48:09,636 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 12:48:09,668 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c288024a2c5046eb880ae7 HTTP/1.1" 200 None 2026-03-26 12:48:09,670 - lava_callback - ERROR - Error processing callback for node 69c288024a2c5046eb880ae7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:46300 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46310 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42154 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57778 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43514 - "GET / HTTP/1.0" 200 OK 2026-03-26 12:48:49,615 - lava_callback - INFO - Received callback for node 69c288254a2c5046eb880af6 INFO: 172.21.0.1:43522 - "POST /node/69c288254a2c5046eb880af6 HTTP/1.0" 202 Accepted 2026-03-26 12:48:49,621 - lava_callback - INFO - Starting async job submit for node 69c288254a2c5046eb880af6 2026-03-26 12:48:49,622 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 12:48:49,636 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c288254a2c5046eb880af6 HTTP/1.1" 200 None 2026-03-26 12:48:49,637 - lava_callback - DEBUG - Processing node 69c288254a2c5046eb880af6: baseline-arm-clabbe 2026-03-26 12:48:49,637 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c288254a2c5046eb880af6 2026-03-26 12:48:49,638 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 12:48:49,654 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c288254a2c5046eb880af6 HTTP/1.1" 200 None 2026-03-26 12:48:49,655 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 12:48:49,669 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf353af1004e82292e0db7 HTTP/1.1" 200 None 2026-03-26 12:48:49,671 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 12:48:49,698 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c288254a2c5046eb880af6 HTTP/1.1" 200 None 2026-03-26 12:48:49,700 - lava_callback - ERROR - Error processing callback for node 69c288254a2c5046eb880af6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:42318 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40598 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40606 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43970 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43986 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51536 - "GET / HTTP/1.0" 200 OK 2026-03-26 12:49:29,616 - lava_callback - INFO - Received callback for node 69c2884b4a2c5046eb880b05 INFO: 172.21.0.1:51550 - "POST /node/69c2884b4a2c5046eb880b05 HTTP/1.0" 202 Accepted 2026-03-26 12:49:29,622 - lava_callback - INFO - Starting async job submit for node 69c2884b4a2c5046eb880b05 2026-03-26 12:49:29,623 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 12:49:29,722 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2884b4a2c5046eb880b05 HTTP/1.1" 200 None 2026-03-26 12:49:29,722 - lava_callback - DEBUG - Processing node 69c2884b4a2c5046eb880b05: baseline-arm-clabbe 2026-03-26 12:49:29,723 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2884b4a2c5046eb880b05 2026-03-26 12:49:29,723 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 12:49:29,737 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2884b4a2c5046eb880b05 HTTP/1.1" 200 None 2026-03-26 12:49:29,738 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 12:49:29,751 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf353af1004e82292e0db7 HTTP/1.1" 200 None 2026-03-26 12:49:29,753 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 12:49:29,780 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2884b4a2c5046eb880b05 HTTP/1.1" 200 None 2026-03-26 12:49:29,781 - lava_callback - ERROR - Error processing callback for node 69c2884b4a2c5046eb880b05: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:42856 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57480 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38628 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47706 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47710 - "GET / HTTP/1.0" 200 OK 2026-03-26 12:50:09,742 - lava_callback - INFO - Received callback for node 69c288704a2c5046eb880b13 INFO: 172.21.0.1:48626 - "POST /node/69c288704a2c5046eb880b13 HTTP/1.0" 202 Accepted 2026-03-26 12:50:09,748 - lava_callback - INFO - Starting async job submit for node 69c288704a2c5046eb880b13 2026-03-26 12:50:09,749 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 12:50:09,764 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c288704a2c5046eb880b13 HTTP/1.1" 200 None 2026-03-26 12:50:09,764 - lava_callback - DEBUG - Processing node 69c288704a2c5046eb880b13: baseline-arm-clabbe 2026-03-26 12:50:09,765 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c288704a2c5046eb880b13 2026-03-26 12:50:09,765 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 12:50:09,779 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c288704a2c5046eb880b13 HTTP/1.1" 200 None 2026-03-26 12:50:09,780 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 12:50:09,801 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf353af1004e82292e0db7 HTTP/1.1" 200 None 2026-03-26 12:50:09,803 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 12:50:09,828 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c288704a2c5046eb880b13 HTTP/1.1" 200 None 2026-03-26 12:50:09,829 - lava_callback - ERROR - Error processing callback for node 69c288704a2c5046eb880b13: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:48638 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48644 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60672 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50352 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40940 - "GET / HTTP/1.0" 200 OK 2026-03-26 12:50:49,700 - lava_callback - INFO - Received callback for node 69c288954a2c5046eb880b21 INFO: 172.21.0.1:53116 - "POST /node/69c288954a2c5046eb880b21 HTTP/1.0" 202 Accepted 2026-03-26 12:50:49,706 - lava_callback - INFO - Starting async job submit for node 69c288954a2c5046eb880b21 2026-03-26 12:50:49,707 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 12:50:49,722 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c288954a2c5046eb880b21 HTTP/1.1" 200 None 2026-03-26 12:50:49,722 - lava_callback - DEBUG - Processing node 69c288954a2c5046eb880b21: baseline-arm-clabbe 2026-03-26 12:50:49,723 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c288954a2c5046eb880b21 2026-03-26 12:50:49,723 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 12:50:49,738 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c288954a2c5046eb880b21 HTTP/1.1" 200 None 2026-03-26 12:50:49,740 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 12:50:49,753 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf353af1004e82292e0db7 HTTP/1.1" 200 None 2026-03-26 12:50:49,754 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 12:50:49,775 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c288954a2c5046eb880b21 HTTP/1.1" 200 None 2026-03-26 12:50:49,781 - lava_callback - ERROR - Error processing callback for node 69c288954a2c5046eb880b21: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:53118 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51500 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51506 - "GET / HTTP/1.0" 200 OK 2026-03-26 12:51:09,710 - lava_callback - INFO - Received callback for node 69c288ba4a2c5046eb880b29 INFO: 172.21.0.1:49822 - "POST /node/69c288ba4a2c5046eb880b29 HTTP/1.0" 202 Accepted 2026-03-26 12:51:09,716 - lava_callback - INFO - Starting async job submit for node 69c288ba4a2c5046eb880b29 2026-03-26 12:51:09,717 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 12:51:09,731 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c288ba4a2c5046eb880b29 HTTP/1.1" 200 None 2026-03-26 12:51:09,732 - lava_callback - DEBUG - Processing node 69c288ba4a2c5046eb880b29: baseline-arm-clabbe 2026-03-26 12:51:09,732 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c288ba4a2c5046eb880b29 2026-03-26 12:51:09,733 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 12:51:09,747 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c288ba4a2c5046eb880b29 HTTP/1.1" 200 None 2026-03-26 12:51:09,748 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 12:51:09,762 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf353af1004e82292e0db7 HTTP/1.1" 200 None 2026-03-26 12:51:09,764 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 12:51:09,785 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c288ba4a2c5046eb880b29 HTTP/1.1" 200 None 2026-03-26 12:51:09,786 - lava_callback - ERROR - Error processing callback for node 69c288ba4a2c5046eb880b29: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:49828 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49830 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58878 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39054 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35152 - "GET / HTTP/1.0" 200 OK 2026-03-26 12:51:49,730 - lava_callback - INFO - Received callback for node 69c288de4a2c5046eb880b37 INFO: 172.21.0.1:51640 - "POST /node/69c288de4a2c5046eb880b37 HTTP/1.0" 202 Accepted 2026-03-26 12:51:49,735 - lava_callback - INFO - Starting async job submit for node 69c288de4a2c5046eb880b37 2026-03-26 12:51:49,736 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 12:51:49,750 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c288de4a2c5046eb880b37 HTTP/1.1" 200 None 2026-03-26 12:51:49,751 - lava_callback - DEBUG - Processing node 69c288de4a2c5046eb880b37: baseline-arm-clabbe 2026-03-26 12:51:49,751 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c288de4a2c5046eb880b37 2026-03-26 12:51:49,752 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 12:51:49,774 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c288de4a2c5046eb880b37 HTTP/1.1" 200 None 2026-03-26 12:51:49,776 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 12:51:49,789 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf353af1004e82292e0db7 HTTP/1.1" 200 None 2026-03-26 12:51:49,791 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 12:51:49,815 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c288de4a2c5046eb880b37 HTTP/1.1" 200 None 2026-03-26 12:51:49,817 - lava_callback - ERROR - Error processing callback for node 69c288de4a2c5046eb880b37: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:51646 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34324 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34328 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52304 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52308 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44616 - "GET / HTTP/1.0" 200 OK 2026-03-26 12:52:29,760 - lava_callback - INFO - Received callback for node 69c289034a2c5046eb880b42 INFO: 172.21.0.1:59940 - "POST /node/69c289034a2c5046eb880b42 HTTP/1.0" 202 Accepted 2026-03-26 12:52:29,765 - lava_callback - INFO - Starting async job submit for node 69c289034a2c5046eb880b42 2026-03-26 12:52:29,766 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 12:52:29,785 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c289034a2c5046eb880b42 HTTP/1.1" 200 None 2026-03-26 12:52:29,787 - lava_callback - DEBUG - Processing node 69c289034a2c5046eb880b42: baseline-arm-clabbe 2026-03-26 12:52:29,787 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c289034a2c5046eb880b42 2026-03-26 12:52:29,788 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 12:52:29,805 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c289034a2c5046eb880b42 HTTP/1.1" 200 None 2026-03-26 12:52:29,806 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 12:52:29,820 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf353af1004e82292e0db7 HTTP/1.1" 200 None 2026-03-26 12:52:29,821 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 12:52:29,842 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c289034a2c5046eb880b42 HTTP/1.1" 200 None 2026-03-26 12:52:29,851 - lava_callback - ERROR - Error processing callback for node 69c289034a2c5046eb880b42: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:59950 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35414 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58622 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57072 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57088 - "GET / HTTP/1.0" 200 OK 2026-03-26 12:53:09,731 - lava_callback - INFO - Received callback for node 69c289284a2c5046eb880b50 INFO: 172.21.0.1:37306 - "POST /node/69c289284a2c5046eb880b50 HTTP/1.0" 202 Accepted 2026-03-26 12:53:09,738 - lava_callback - INFO - Starting async job submit for node 69c289284a2c5046eb880b50 2026-03-26 12:53:09,739 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 12:53:09,756 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c289284a2c5046eb880b50 HTTP/1.1" 200 None 2026-03-26 12:53:09,757 - lava_callback - DEBUG - Processing node 69c289284a2c5046eb880b50: baseline-arm-clabbe 2026-03-26 12:53:09,757 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c289284a2c5046eb880b50 2026-03-26 12:53:09,758 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 12:53:09,773 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c289284a2c5046eb880b50 HTTP/1.1" 200 None 2026-03-26 12:53:09,774 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 12:53:09,792 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf353af1004e82292e0db7 HTTP/1.1" 200 None 2026-03-26 12:53:09,794 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 12:53:09,815 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c289284a2c5046eb880b50 HTTP/1.1" 200 None 2026-03-26 12:53:09,821 - lava_callback - ERROR - Error processing callback for node 69c289284a2c5046eb880b50: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:37318 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37324 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58860 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33036 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47152 - "GET / HTTP/1.0" 200 OK 2026-03-26 12:53:49,902 - lava_callback - INFO - Received callback for node 69c2894b4a2c5046eb880b60 INFO: 172.21.0.1:60118 - "POST /node/69c2894b4a2c5046eb880b60 HTTP/1.0" 202 Accepted 2026-03-26 12:53:49,908 - lava_callback - INFO - Starting async job submit for node 69c2894b4a2c5046eb880b60 2026-03-26 12:53:49,909 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 12:53:49,923 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2894b4a2c5046eb880b60 HTTP/1.1" 200 None 2026-03-26 12:53:49,924 - lava_callback - DEBUG - Processing node 69c2894b4a2c5046eb880b60: baseline-arm-clabbe 2026-03-26 12:53:49,925 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2894b4a2c5046eb880b60 2026-03-26 12:53:49,925 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 12:53:49,938 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2894b4a2c5046eb880b60 HTTP/1.1" 200 None 2026-03-26 12:53:49,939 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 12:53:49,953 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf353af1004e82292e0db7 HTTP/1.1" 200 None 2026-03-26 12:53:49,954 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 12:53:49,980 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2894b4a2c5046eb880b60 HTTP/1.1" 200 None 2026-03-26 12:53:49,987 - lava_callback - ERROR - Error processing callback for node 69c2894b4a2c5046eb880b60: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:60130 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34122 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34132 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52350 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52356 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36264 - "GET / HTTP/1.0" 200 OK 2026-03-26 12:54:29,762 - lava_callback - INFO - Received callback for node 69c289704a2c5046eb880b68 INFO: 172.21.0.1:40636 - "POST /node/69c289704a2c5046eb880b68 HTTP/1.0" 202 Accepted 2026-03-26 12:54:29,768 - lava_callback - INFO - Starting async job submit for node 69c289704a2c5046eb880b68 2026-03-26 12:54:29,769 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 12:54:29,783 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c289704a2c5046eb880b68 HTTP/1.1" 200 None 2026-03-26 12:54:29,784 - lava_callback - DEBUG - Processing node 69c289704a2c5046eb880b68: baseline-arm-clabbe 2026-03-26 12:54:29,784 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c289704a2c5046eb880b68 2026-03-26 12:54:29,785 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 12:54:29,798 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c289704a2c5046eb880b68 HTTP/1.1" 200 None 2026-03-26 12:54:29,799 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 12:54:29,812 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf353af1004e82292e0db7 HTTP/1.1" 200 None 2026-03-26 12:54:29,814 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 12:54:29,844 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c289704a2c5046eb880b68 HTTP/1.1" 200 None 2026-03-26 12:54:29,851 - lava_callback - ERROR - Error processing callback for node 69c289704a2c5046eb880b68: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:40638 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44178 - "GET / HTTP/1.0" 200 OK 2026-03-26 12:54:49,807 - lava_callback - INFO - Received callback for node 69c289944a2c5046eb880b76 INFO: 172.21.0.1:43168 - "POST /node/69c289944a2c5046eb880b76 HTTP/1.0" 202 Accepted 2026-03-26 12:54:49,813 - lava_callback - INFO - Starting async job submit for node 69c289944a2c5046eb880b76 2026-03-26 12:54:49,814 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 12:54:49,830 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c289944a2c5046eb880b76 HTTP/1.1" 200 None 2026-03-26 12:54:49,831 - lava_callback - DEBUG - Processing node 69c289944a2c5046eb880b76: baseline-arm-clabbe 2026-03-26 12:54:49,831 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c289944a2c5046eb880b76 2026-03-26 12:54:49,831 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 12:54:49,848 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c289944a2c5046eb880b76 HTTP/1.1" 200 None 2026-03-26 12:54:49,849 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 12:54:49,862 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf353af1004e82292e0db7 HTTP/1.1" 200 None 2026-03-26 12:54:49,864 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 12:54:49,887 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c289944a2c5046eb880b76 HTTP/1.1" 200 None 2026-03-26 12:54:49,888 - lava_callback - ERROR - Error processing callback for node 69c289944a2c5046eb880b76: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:43174 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45738 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45754 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40462 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40470 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60150 - "GET / HTTP/1.0" 200 OK 2026-03-26 12:55:29,811 - lava_callback - INFO - Received callback for node 69c289ba4a2c5046eb880b82 INFO: 172.21.0.1:46872 - "POST /node/69c289ba4a2c5046eb880b82 HTTP/1.0" 202 Accepted 2026-03-26 12:55:29,817 - lava_callback - INFO - Starting async job submit for node 69c289ba4a2c5046eb880b82 2026-03-26 12:55:29,817 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 12:55:29,837 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c289ba4a2c5046eb880b82 HTTP/1.1" 200 None 2026-03-26 12:55:29,838 - lava_callback - DEBUG - Processing node 69c289ba4a2c5046eb880b82: baseline-arm-clabbe 2026-03-26 12:55:29,838 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c289ba4a2c5046eb880b82 2026-03-26 12:55:29,839 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 12:55:29,852 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c289ba4a2c5046eb880b82 HTTP/1.1" 200 None 2026-03-26 12:55:29,854 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 12:55:29,867 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf353af1004e82292e0db7 HTTP/1.1" 200 None 2026-03-26 12:55:29,869 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 12:55:29,889 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c289ba4a2c5046eb880b82 HTTP/1.1" 200 None 2026-03-26 12:55:29,895 - lava_callback - ERROR - Error processing callback for node 69c289ba4a2c5046eb880b82: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:46888 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41238 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53228 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44262 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44278 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48744 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48750 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39806 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39572 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48912 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57792 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51130 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51140 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59310 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59320 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49662 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49150 - "GET /robots.txt HTTP/1.0" 404 Not Found INFO: 172.21.0.1:49164 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38660 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54038 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34010 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34016 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50650 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50658 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52688 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45002 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48576 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34144 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51092 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51096 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54678 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54686 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42956 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57414 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38664 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50208 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35560 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35562 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60190 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60196 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50350 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45402 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37308 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52852 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57902 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57912 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57066 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57076 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56008 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56982 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53030 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49842 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47248 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47262 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52314 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52324 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50998 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39474 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43326 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57232 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52366 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52380 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35598 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35604 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38644 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43162 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54484 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35450 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49216 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49222 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53552 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53556 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59024 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51644 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56762 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33288 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56054 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56062 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38492 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38500 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33800 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46142 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45628 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58218 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54948 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54962 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58798 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58800 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56600 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41866 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57598 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55330 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41104 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41110 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55570 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55574 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57522 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38916 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52762 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46858 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52406 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52418 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55748 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55762 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34896 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47822 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50448 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33744 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53716 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53728 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49822 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49830 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36170 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35272 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45640 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60368 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50352 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50362 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40070 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40076 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36536 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59124 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55980 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39272 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43910 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43912 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57570 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57586 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47242 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45074 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54696 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60704 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38898 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38906 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43748 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43762 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57218 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44354 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52154 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58116 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54734 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54748 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32796 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:32808 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53592 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51680 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53792 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51174 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55988 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56002 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45052 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45066 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45802 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44766 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37196 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49702 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47822 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47824 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47042 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47044 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33572 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47048 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56738 - "GET / HTTP/1.0" 200 OK 2026-03-26 13:15:50,579 - lava_callback - INFO - Received callback for node 69c28e714a2c5046eb880c08 INFO: 172.21.0.1:35390 - "POST /node/69c28e714a2c5046eb880c08 HTTP/1.0" 202 Accepted 2026-03-26 13:15:50,585 - lava_callback - INFO - Starting async job submit for node 69c28e714a2c5046eb880c08 2026-03-26 13:15:50,586 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 13:15:50,601 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c28e714a2c5046eb880c08 HTTP/1.1" 200 None 2026-03-26 13:15:50,602 - lava_callback - DEBUG - Processing node 69c28e714a2c5046eb880c08: baseline-arm64-baylibre 2026-03-26 13:15:50,602 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c28e714a2c5046eb880c08 2026-03-26 13:15:50,603 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 13:15:50,616 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c28e714a2c5046eb880c08 HTTP/1.1" 200 None 2026-03-26 13:15:50,617 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 13:15:50,631 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-26 13:15:50,633 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 13:15:50,654 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c28e714a2c5046eb880c08 HTTP/1.1" 200 None 2026-03-26 13:15:50,656 - lava_callback - ERROR - Error processing callback for node 69c28e714a2c5046eb880c08: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:35400 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56130 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56132 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46416 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46428 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57356 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57248 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38462 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54928 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37096 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37110 - "GET / HTTP/1.0" 200 OK 2026-03-26 13:17:10,380 - lava_callback - INFO - Received callback for node 69c28ec24a2c5046eb880c10 INFO: 172.21.0.1:33390 - "POST /node/69c28ec24a2c5046eb880c10 HTTP/1.0" 202 Accepted 2026-03-26 13:17:10,385 - lava_callback - INFO - Starting async job submit for node 69c28ec24a2c5046eb880c10 2026-03-26 13:17:10,386 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 13:17:10,401 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c28ec24a2c5046eb880c10 HTTP/1.1" 200 None 2026-03-26 13:17:10,402 - lava_callback - DEBUG - Processing node 69c28ec24a2c5046eb880c10: baseline-arm64-baylibre 2026-03-26 13:17:10,402 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c28ec24a2c5046eb880c10 2026-03-26 13:17:10,403 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 13:17:10,416 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c28ec24a2c5046eb880c10 HTTP/1.1" 200 None 2026-03-26 13:17:10,418 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 13:17:10,431 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-26 13:17:10,432 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 13:17:10,452 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c28ec24a2c5046eb880c10 HTTP/1.1" 200 None 2026-03-26 13:17:10,458 - lava_callback - ERROR - Error processing callback for node 69c28ec24a2c5046eb880c10: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:33394 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33404 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45912 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33104 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38508 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45876 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36810 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36826 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35298 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35304 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37226 - "GET / HTTP/1.0" 200 OK 2026-03-26 13:18:30,385 - lava_callback - INFO - Received callback for node 69c28f134a2c5046eb880c1d INFO: 172.21.0.1:51032 - "POST /node/69c28f134a2c5046eb880c1d HTTP/1.0" 202 Accepted 2026-03-26 13:18:30,390 - lava_callback - INFO - Starting async job submit for node 69c28f134a2c5046eb880c1d 2026-03-26 13:18:30,391 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 13:18:30,406 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c28f134a2c5046eb880c1d HTTP/1.1" 200 None 2026-03-26 13:18:30,407 - lava_callback - DEBUG - Processing node 69c28f134a2c5046eb880c1d: baseline-arm64-baylibre 2026-03-26 13:18:30,407 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c28f134a2c5046eb880c1d 2026-03-26 13:18:30,408 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 13:18:30,421 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c28f134a2c5046eb880c1d HTTP/1.1" 200 None 2026-03-26 13:18:30,422 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 13:18:30,436 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-26 13:18:30,437 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 13:18:30,471 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c28f134a2c5046eb880c1d HTTP/1.1" 200 None 2026-03-26 13:18:30,473 - lava_callback - ERROR - Error processing callback for node 69c28f134a2c5046eb880c1d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:51044 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46532 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47866 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33582 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33596 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36522 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36524 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40018 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48706 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35078 - "GET / HTTP/1.0" 200 OK 2026-03-26 13:19:50,418 - lava_callback - INFO - Received callback for node 69c28f654a2c5046eb880c2b INFO: 172.21.0.1:38484 - "POST /node/69c28f654a2c5046eb880c2b HTTP/1.0" 202 Accepted 2026-03-26 13:19:50,423 - lava_callback - INFO - Starting async job submit for node 69c28f654a2c5046eb880c2b 2026-03-26 13:19:50,424 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 13:19:50,438 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c28f654a2c5046eb880c2b HTTP/1.1" 200 None 2026-03-26 13:19:50,439 - lava_callback - DEBUG - Processing node 69c28f654a2c5046eb880c2b: baseline-arm64-baylibre 2026-03-26 13:19:50,440 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c28f654a2c5046eb880c2b 2026-03-26 13:19:50,440 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 13:19:50,455 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c28f654a2c5046eb880c2b HTTP/1.1" 200 None 2026-03-26 13:19:50,456 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 13:19:50,470 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-26 13:19:50,471 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 13:19:50,500 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c28f654a2c5046eb880c2b HTTP/1.1" 200 None 2026-03-26 13:19:50,502 - lava_callback - ERROR - Error processing callback for node 69c28f654a2c5046eb880c2b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:38488 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51018 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51028 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54338 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54344 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60122 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60532 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56956 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42948 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56932 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56940 - "GET / HTTP/1.0" 200 OK 2026-03-26 13:21:10,750 - lava_callback - INFO - Received callback for node 69c28fb64a2c5046eb880c34 INFO: 172.21.0.1:55650 - "POST /node/69c28fb64a2c5046eb880c34 HTTP/1.0" 202 Accepted 2026-03-26 13:21:10,756 - lava_callback - INFO - Starting async job submit for node 69c28fb64a2c5046eb880c34 2026-03-26 13:21:10,756 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 13:21:10,771 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c28fb64a2c5046eb880c34 HTTP/1.1" 200 None 2026-03-26 13:21:10,772 - lava_callback - DEBUG - Processing node 69c28fb64a2c5046eb880c34: baseline-arm64-baylibre 2026-03-26 13:21:10,772 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c28fb64a2c5046eb880c34 2026-03-26 13:21:10,773 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 13:21:10,786 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c28fb64a2c5046eb880c34 HTTP/1.1" 200 None 2026-03-26 13:21:10,788 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 13:21:10,809 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-26 13:21:10,810 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 13:21:10,831 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c28fb64a2c5046eb880c34 HTTP/1.1" 200 None 2026-03-26 13:21:10,837 - lava_callback - ERROR - Error processing callback for node 69c28fb64a2c5046eb880c34: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:55656 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55672 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44602 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60760 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53296 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53034 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58912 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58920 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45554 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45570 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39874 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37862 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37704 - "GET / HTTP/1.0" 200 OK 2026-03-26 13:22:50,657 - lava_callback - INFO - Received callback for node 69c290194a2c5046eb880c43 INFO: 172.21.0.1:36042 - "POST /node/69c290194a2c5046eb880c43 HTTP/1.0" 202 Accepted 2026-03-26 13:22:50,663 - lava_callback - INFO - Starting async job submit for node 69c290194a2c5046eb880c43 2026-03-26 13:22:50,664 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 13:22:50,678 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c290194a2c5046eb880c43 HTTP/1.1" 200 None 2026-03-26 13:22:50,679 - lava_callback - DEBUG - Processing node 69c290194a2c5046eb880c43: baseline-arm64-baylibre 2026-03-26 13:22:50,679 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c290194a2c5046eb880c43 2026-03-26 13:22:50,679 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 13:22:50,693 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c290194a2c5046eb880c43 HTTP/1.1" 200 None 2026-03-26 13:22:50,694 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 13:22:50,727 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-26 13:22:50,729 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 13:22:50,759 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c290194a2c5046eb880c43 HTTP/1.1" 200 None 2026-03-26 13:22:50,760 - lava_callback - ERROR - Error processing callback for node 69c290194a2c5046eb880c43: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:36050 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52394 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52402 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36604 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36612 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38062 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35244 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56546 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34364 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58246 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58262 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59462 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59476 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39360 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51638 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38204 - "GET / HTTP/1.0" 200 OK 2026-03-26 13:24:50,771 - lava_callback - INFO - Received callback for node 69c2908e4a2c5046eb880c4f INFO: 172.21.0.1:33272 - "POST /node/69c2908e4a2c5046eb880c4f HTTP/1.0" 202 Accepted 2026-03-26 13:24:50,777 - lava_callback - INFO - Starting async job submit for node 69c2908e4a2c5046eb880c4f 2026-03-26 13:24:50,777 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 13:24:50,793 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2908e4a2c5046eb880c4f HTTP/1.1" 200 None 2026-03-26 13:24:50,793 - lava_callback - DEBUG - Processing node 69c2908e4a2c5046eb880c4f: baseline-arm64-baylibre 2026-03-26 13:24:50,794 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2908e4a2c5046eb880c4f 2026-03-26 13:24:50,794 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 13:24:50,808 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2908e4a2c5046eb880c4f HTTP/1.1" 200 None 2026-03-26 13:24:50,809 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 13:24:50,823 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf3545f1004e82292e0dc6 HTTP/1.1" 200 None 2026-03-26 13:24:50,824 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 13:24:50,854 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2908e4a2c5046eb880c4f HTTP/1.1" 200 None 2026-03-26 13:24:50,867 - lava_callback - ERROR - Error processing callback for node 69c2908e4a2c5046eb880c4f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:33288 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40208 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40220 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43566 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43568 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42750 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50684 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37974 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57966 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59436 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59450 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41304 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41308 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60958 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41212 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49654 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38746 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44754 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44766 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54970 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54976 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46082 - "GET / HTTP/1.0" 200 OK 2026-03-26 13:27:31,350 - lava_callback - INFO - Received callback for node 69c291324a2c5046eb880c66 INFO: 172.21.0.1:50800 - "POST /node/69c291324a2c5046eb880c66 HTTP/1.0" 202 Accepted 2026-03-26 13:27:31,356 - lava_callback - INFO - Starting async job submit for node 69c291324a2c5046eb880c66 2026-03-26 13:27:31,357 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 13:27:31,371 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c291324a2c5046eb880c66 HTTP/1.1" 200 None 2026-03-26 13:27:31,372 - lava_callback - DEBUG - Processing node 69c291324a2c5046eb880c66: baseline-arm64-clabbe 2026-03-26 13:27:31,372 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c291324a2c5046eb880c66 2026-03-26 13:27:31,373 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 13:27:31,386 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c291324a2c5046eb880c66 HTTP/1.1" 200 None 2026-03-26 13:27:31,388 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 13:27:31,401 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf3545f1004e82292e0dc6 HTTP/1.1" 200 None 2026-03-26 13:27:31,403 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 13:27:31,426 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c291324a2c5046eb880c66 HTTP/1.1" 200 None 2026-03-26 13:27:31,428 - lava_callback - ERROR - Error processing callback for node 69c291324a2c5046eb880c66: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:50810 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59170 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35740 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54552 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54566 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38988 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39002 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32812 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38350 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36758 - "GET / HTTP/1.0" 200 OK 2026-03-26 13:28:51,624 - lava_callback - INFO - Received callback for node 69c291824a2c5046eb880c77 INFO: 172.21.0.1:49544 - "POST /node/69c291824a2c5046eb880c77 HTTP/1.0" 202 Accepted 2026-03-26 13:28:51,630 - lava_callback - INFO - Starting async job submit for node 69c291824a2c5046eb880c77 2026-03-26 13:28:51,631 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 13:28:51,678 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c291824a2c5046eb880c77 HTTP/1.1" 200 None 2026-03-26 13:28:51,679 - lava_callback - DEBUG - Processing node 69c291824a2c5046eb880c77: baseline-arm64-clabbe 2026-03-26 13:28:51,679 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c291824a2c5046eb880c77 2026-03-26 13:28:51,680 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 13:28:51,695 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c291824a2c5046eb880c77 HTTP/1.1" 200 None 2026-03-26 13:28:51,697 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 13:28:51,710 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf3545f1004e82292e0dc6 HTTP/1.1" 200 None 2026-03-26 13:28:51,711 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 13:28:51,741 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c291824a2c5046eb880c77 HTTP/1.1" 200 None 2026-03-26 13:28:51,743 - lava_callback - ERROR - Error processing callback for node 69c291824a2c5046eb880c77: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:49546 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38038 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38040 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53922 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53932 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56124 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48072 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42850 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53380 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58796 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58810 - "GET / HTTP/1.0" 200 OK 2026-03-26 13:30:11,452 - lava_callback - INFO - Received callback for node 69c291d24a2c5046eb880c81 INFO: 172.21.0.1:53870 - "POST /node/69c291d24a2c5046eb880c81 HTTP/1.0" 202 Accepted 2026-03-26 13:30:11,458 - lava_callback - INFO - Starting async job submit for node 69c291d24a2c5046eb880c81 2026-03-26 13:30:11,459 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 13:30:11,485 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c291d24a2c5046eb880c81 HTTP/1.1" 200 None 2026-03-26 13:30:11,486 - lava_callback - DEBUG - Processing node 69c291d24a2c5046eb880c81: baseline-arm64-clabbe 2026-03-26 13:30:11,486 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c291d24a2c5046eb880c81 2026-03-26 13:30:11,487 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 13:30:11,505 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c291d24a2c5046eb880c81 HTTP/1.1" 200 None 2026-03-26 13:30:11,507 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 13:30:11,520 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf3545f1004e82292e0dc6 HTTP/1.1" 200 None 2026-03-26 13:30:11,522 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 13:30:11,543 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c291d24a2c5046eb880c81 HTTP/1.1" 200 None 2026-03-26 13:30:11,545 - lava_callback - ERROR - Error processing callback for node 69c291d24a2c5046eb880c81: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:53880 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53890 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35064 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52610 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53228 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50946 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34630 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34642 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47080 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47092 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34262 - "GET / HTTP/1.0" 200 OK 2026-03-26 13:31:31,460 - lava_callback - INFO - Received callback for node 69c292224a2c5046eb880c91 INFO: 172.21.0.1:51616 - "POST /node/69c292224a2c5046eb880c91 HTTP/1.0" 202 Accepted 2026-03-26 13:31:31,465 - lava_callback - INFO - Starting async job submit for node 69c292224a2c5046eb880c91 2026-03-26 13:31:31,466 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 13:31:31,481 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c292224a2c5046eb880c91 HTTP/1.1" 200 None 2026-03-26 13:31:31,482 - lava_callback - DEBUG - Processing node 69c292224a2c5046eb880c91: baseline-arm64-clabbe 2026-03-26 13:31:31,482 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c292224a2c5046eb880c91 2026-03-26 13:31:31,482 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 13:31:31,496 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c292224a2c5046eb880c91 HTTP/1.1" 200 None 2026-03-26 13:31:31,497 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 13:31:31,514 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-26 13:31:31,516 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 13:31:31,538 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c292224a2c5046eb880c91 HTTP/1.1" 200 None 2026-03-26 13:31:31,540 - lava_callback - ERROR - Error processing callback for node 69c292224a2c5046eb880c91: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:51630 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33444 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49720 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38076 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38088 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50154 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50162 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50776 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39024 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43866 - "GET / HTTP/1.0" 200 OK 2026-03-26 13:32:51,568 - lava_callback - INFO - Received callback for node 69c292724a2c5046eb880ca1 INFO: 172.21.0.1:54318 - "POST /node/69c292724a2c5046eb880ca1 HTTP/1.0" 202 Accepted 2026-03-26 13:32:51,574 - lava_callback - INFO - Starting async job submit for node 69c292724a2c5046eb880ca1 2026-03-26 13:32:51,575 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 13:32:51,589 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c292724a2c5046eb880ca1 HTTP/1.1" 200 None 2026-03-26 13:32:51,590 - lava_callback - DEBUG - Processing node 69c292724a2c5046eb880ca1: baseline-arm64-clabbe 2026-03-26 13:32:51,590 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c292724a2c5046eb880ca1 2026-03-26 13:32:51,591 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 13:32:51,607 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c292724a2c5046eb880ca1 HTTP/1.1" 200 None 2026-03-26 13:32:51,608 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 13:32:51,623 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-26 13:32:51,625 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 13:32:51,645 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c292724a2c5046eb880ca1 HTTP/1.1" 200 None 2026-03-26 13:32:51,646 - lava_callback - ERROR - Error processing callback for node 69c292724a2c5046eb880ca1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:54332 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52256 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52268 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52246 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52252 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59716 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44794 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56636 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46068 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59210 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59212 - "GET / HTTP/1.0" 200 OK 2026-03-26 13:34:11,623 - lava_callback - INFO - Received callback for node 69c292c24a2c5046eb880ca9 INFO: 172.21.0.1:56268 - "POST /node/69c292c24a2c5046eb880ca9 HTTP/1.0" 202 Accepted 2026-03-26 13:34:11,628 - lava_callback - INFO - Starting async job submit for node 69c292c24a2c5046eb880ca9 2026-03-26 13:34:11,629 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 13:34:11,650 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c292c24a2c5046eb880ca9 HTTP/1.1" 200 None 2026-03-26 13:34:11,651 - lava_callback - DEBUG - Processing node 69c292c24a2c5046eb880ca9: baseline-arm64-clabbe 2026-03-26 13:34:11,652 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c292c24a2c5046eb880ca9 2026-03-26 13:34:11,652 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 13:34:11,666 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c292c24a2c5046eb880ca9 HTTP/1.1" 200 None 2026-03-26 13:34:11,668 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 13:34:11,697 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf3545f1004e82292e0dc6 HTTP/1.1" 200 None 2026-03-26 13:34:11,699 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 13:34:11,721 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c292c24a2c5046eb880ca9 HTTP/1.1" 200 None 2026-03-26 13:34:11,728 - lava_callback - ERROR - Error processing callback for node 69c292c24a2c5046eb880ca9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:56272 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56278 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34002 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56754 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39982 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49936 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53142 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53148 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42508 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42518 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43244 - "GET / HTTP/1.0" 200 OK 2026-03-26 13:35:31,671 - lava_callback - INFO - Received callback for node 69c293134a2c5046eb880cb4 INFO: 172.21.0.1:32810 - "POST /node/69c293134a2c5046eb880cb4 HTTP/1.0" 202 Accepted 2026-03-26 13:35:31,676 - lava_callback - INFO - Starting async job submit for node 69c293134a2c5046eb880cb4 2026-03-26 13:35:31,677 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 13:35:31,692 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c293134a2c5046eb880cb4 HTTP/1.1" 200 None 2026-03-26 13:35:31,693 - lava_callback - DEBUG - Processing node 69c293134a2c5046eb880cb4: baseline-arm64-clabbe 2026-03-26 13:35:31,693 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c293134a2c5046eb880cb4 2026-03-26 13:35:31,694 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 13:35:31,734 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c293134a2c5046eb880cb4 HTTP/1.1" 200 None 2026-03-26 13:35:31,735 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 13:35:31,757 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-26 13:35:31,758 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 13:35:31,779 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c293134a2c5046eb880cb4 HTTP/1.1" 200 None 2026-03-26 13:35:31,785 - lava_callback - ERROR - Error processing callback for node 69c293134a2c5046eb880cb4: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:32816 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46980 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45028 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49410 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49418 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35278 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35288 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33892 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47406 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39526 - "GET / HTTP/1.0" 200 OK 2026-03-26 13:36:51,582 - lava_callback - INFO - Received callback for node 69c293634a2c5046eb880cc5 INFO: 172.21.0.1:36372 - "POST /node/69c293634a2c5046eb880cc5 HTTP/1.0" 202 Accepted 2026-03-26 13:36:51,588 - lava_callback - INFO - Starting async job submit for node 69c293634a2c5046eb880cc5 2026-03-26 13:36:51,588 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 13:36:51,602 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c293634a2c5046eb880cc5 HTTP/1.1" 200 None 2026-03-26 13:36:51,603 - lava_callback - DEBUG - Processing node 69c293634a2c5046eb880cc5: baseline-arm64-clabbe 2026-03-26 13:36:51,603 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c293634a2c5046eb880cc5 2026-03-26 13:36:51,604 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 13:36:51,617 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c293634a2c5046eb880cc5 HTTP/1.1" 200 None 2026-03-26 13:36:51,619 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 13:36:51,631 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf3545f1004e82292e0dc6 HTTP/1.1" 200 None 2026-03-26 13:36:51,633 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 13:36:51,653 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c293634a2c5046eb880cc5 HTTP/1.1" 200 None 2026-03-26 13:36:51,654 - lava_callback - ERROR - Error processing callback for node 69c293634a2c5046eb880cc5: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:36376 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36068 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36072 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35058 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35066 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60572 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54876 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52830 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53614 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37100 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37116 - "GET / HTTP/1.0" 200 OK 2026-03-26 13:38:11,709 - lava_callback - INFO - Received callback for node 69c293bd4a2c5046eb880cd1 INFO: 172.21.0.1:59956 - "POST /node/69c293bd4a2c5046eb880cd1 HTTP/1.0" 202 Accepted 2026-03-26 13:38:11,714 - lava_callback - INFO - Starting async job submit for node 69c293bd4a2c5046eb880cd1 2026-03-26 13:38:11,715 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 13:38:11,747 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c293bd4a2c5046eb880cd1 HTTP/1.1" 200 None 2026-03-26 13:38:11,747 - lava_callback - DEBUG - Processing node 69c293bd4a2c5046eb880cd1: baseline-arm64-clabbe 2026-03-26 13:38:11,748 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c293bd4a2c5046eb880cd1 2026-03-26 13:38:11,748 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 13:38:11,761 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c293bd4a2c5046eb880cd1 HTTP/1.1" 200 None 2026-03-26 13:38:11,763 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 13:38:11,775 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf3545f1004e82292e0dc6 HTTP/1.1" 200 None 2026-03-26 13:38:11,777 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 13:38:11,797 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c293bd4a2c5046eb880cd1 HTTP/1.1" 200 None 2026-03-26 13:38:11,803 - lava_callback - ERROR - Error processing callback for node 69c293bd4a2c5046eb880cd1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:59966 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59974 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57582 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53110 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54868 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50508 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40866 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40880 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60024 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60028 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55724 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33204 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58156 - "GET / HTTP/1.0" 200 OK 2026-03-26 13:39:51,845 - lava_callback - INFO - Received callback for node 69c294134a2c5046eb880cdd INFO: 172.21.0.1:42588 - "POST /node/69c294134a2c5046eb880cdd HTTP/1.0" 202 Accepted 2026-03-26 13:39:51,851 - lava_callback - INFO - Starting async job submit for node 69c294134a2c5046eb880cdd 2026-03-26 13:39:51,852 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 13:39:51,866 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c294134a2c5046eb880cdd HTTP/1.1" 200 None 2026-03-26 13:39:51,867 - lava_callback - DEBUG - Processing node 69c294134a2c5046eb880cdd: baseline-arm64-clabbe 2026-03-26 13:39:51,867 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c294134a2c5046eb880cdd 2026-03-26 13:39:51,868 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 13:39:51,882 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c294134a2c5046eb880cdd HTTP/1.1" 200 None 2026-03-26 13:39:51,884 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 13:39:51,898 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-26 13:39:51,899 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 13:39:51,920 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c294134a2c5046eb880cdd HTTP/1.1" 200 None 2026-03-26 13:39:51,921 - lava_callback - ERROR - Error processing callback for node 69c294134a2c5046eb880cdd: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:42596 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58832 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58844 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36796 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36810 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51778 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42594 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37078 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53728 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43902 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43906 - "GET / HTTP/1.0" 200 OK 2026-03-26 13:41:11,902 - lava_callback - INFO - Received callback for node 69c294644a2c5046eb880ce5 INFO: 172.21.0.1:56990 - "POST /node/69c294644a2c5046eb880ce5 HTTP/1.0" 202 Accepted 2026-03-26 13:41:11,908 - lava_callback - INFO - Starting async job submit for node 69c294644a2c5046eb880ce5 2026-03-26 13:41:11,909 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 13:41:11,925 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c294644a2c5046eb880ce5 HTTP/1.1" 200 None 2026-03-26 13:41:11,926 - lava_callback - DEBUG - Processing node 69c294644a2c5046eb880ce5: baseline-arm64-clabbe 2026-03-26 13:41:11,926 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c294644a2c5046eb880ce5 2026-03-26 13:41:11,927 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 13:41:11,941 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c294644a2c5046eb880ce5 HTTP/1.1" 200 None 2026-03-26 13:41:11,943 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 13:41:11,957 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-26 13:41:11,959 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 13:41:11,996 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c294644a2c5046eb880ce5 HTTP/1.1" 200 None 2026-03-26 13:41:11,998 - lava_callback - ERROR - Error processing callback for node 69c294644a2c5046eb880ce5: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:56992 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57000 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59068 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56850 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48674 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33348 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45460 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45466 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44192 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44202 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54168 - "GET / HTTP/1.0" 200 OK 2026-03-26 13:42:31,943 - lava_callback - INFO - Received callback for node 69c294b44a2c5046eb880cf4 INFO: 172.21.0.1:35268 - "POST /node/69c294b44a2c5046eb880cf4 HTTP/1.0" 202 Accepted 2026-03-26 13:42:31,949 - lava_callback - INFO - Starting async job submit for node 69c294b44a2c5046eb880cf4 2026-03-26 13:42:31,950 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 13:42:31,964 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c294b44a2c5046eb880cf4 HTTP/1.1" 200 None 2026-03-26 13:42:31,965 - lava_callback - DEBUG - Processing node 69c294b44a2c5046eb880cf4: baseline-arm64-clabbe 2026-03-26 13:42:31,965 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c294b44a2c5046eb880cf4 2026-03-26 13:42:31,966 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 13:42:31,980 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c294b44a2c5046eb880cf4 HTTP/1.1" 200 None 2026-03-26 13:42:31,982 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 13:42:32,004 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-26 13:42:32,006 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 13:42:32,037 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c294b44a2c5046eb880cf4 HTTP/1.1" 200 None 2026-03-26 13:42:32,040 - lava_callback - ERROR - Error processing callback for node 69c294b44a2c5046eb880cf4: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:35270 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56370 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49132 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52862 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52870 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54260 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54270 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42152 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51008 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48804 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56332 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46648 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46658 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33976 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33990 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52538 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42076 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48912 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50464 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35298 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35312 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53558 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53570 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36882 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36684 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40626 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35096 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44970 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44984 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33024 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33040 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52654 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54342 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45808 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48526 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40352 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40356 - "GET / HTTP/1.0" 200 OK 2026-03-26 13:47:12,260 - lava_callback - INFO - Received callback for node 69c295db4a2c5046eb880d18 INFO: 172.21.0.1:49848 - "POST /node/69c295db4a2c5046eb880d18 HTTP/1.0" 202 Accepted 2026-03-26 13:47:12,265 - lava_callback - INFO - Starting async job submit for node 69c295db4a2c5046eb880d18 2026-03-26 13:47:12,266 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 13:47:12,282 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c295db4a2c5046eb880d18 HTTP/1.1" 200 None 2026-03-26 13:47:12,282 - lava_callback - DEBUG - Processing node 69c295db4a2c5046eb880d18: baseline-arm64-baylibre 2026-03-26 13:47:12,283 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c295db4a2c5046eb880d18 2026-03-26 13:47:12,283 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 13:47:12,298 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c295db4a2c5046eb880d18 HTTP/1.1" 200 None 2026-03-26 13:47:12,300 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 13:47:12,313 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf3545f1004e82292e0dc6 HTTP/1.1" 200 None 2026-03-26 13:47:12,314 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 13:47:12,335 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c295db4a2c5046eb880d18 HTTP/1.1" 200 None 2026-03-26 13:47:12,341 - lava_callback - ERROR - Error processing callback for node 69c295db4a2c5046eb880d18: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:49860 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49876 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49474 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52830 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48694 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35696 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41852 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41856 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42332 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42344 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49336 - "GET / HTTP/1.0" 200 OK 2026-03-26 13:48:32,130 - lava_callback - INFO - Received callback for node 69c296284a2c5046eb880d25 INFO: 172.21.0.1:51210 - "POST /node/69c296284a2c5046eb880d25 HTTP/1.0" 202 Accepted 2026-03-26 13:48:32,136 - lava_callback - INFO - Starting async job submit for node 69c296284a2c5046eb880d25 2026-03-26 13:48:32,137 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 13:48:32,151 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c296284a2c5046eb880d25 HTTP/1.1" 200 None 2026-03-26 13:48:32,152 - lava_callback - DEBUG - Processing node 69c296284a2c5046eb880d25: baseline-arm64-baylibre 2026-03-26 13:48:32,153 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c296284a2c5046eb880d25 2026-03-26 13:48:32,153 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 13:48:32,167 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c296284a2c5046eb880d25 HTTP/1.1" 200 None 2026-03-26 13:48:32,168 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 13:48:32,181 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf3545f1004e82292e0dc6 HTTP/1.1" 200 None 2026-03-26 13:48:32,183 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 13:48:32,204 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c296284a2c5046eb880d25 HTTP/1.1" 200 None 2026-03-26 13:48:32,211 - lava_callback - ERROR - Error processing callback for node 69c296284a2c5046eb880d25: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:51216 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58706 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39612 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42696 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42706 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34104 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34116 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41850 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54740 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34944 - "GET / HTTP/1.0" 200 OK 2026-03-26 13:49:52,116 - lava_callback - INFO - Received callback for node 69c296794a2c5046eb880d30 INFO: 172.21.0.1:35878 - "POST /node/69c296794a2c5046eb880d30 HTTP/1.0" 202 Accepted 2026-03-26 13:49:52,122 - lava_callback - INFO - Starting async job submit for node 69c296794a2c5046eb880d30 2026-03-26 13:49:52,122 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 13:49:52,139 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c296794a2c5046eb880d30 HTTP/1.1" 200 None 2026-03-26 13:49:52,140 - lava_callback - DEBUG - Processing node 69c296794a2c5046eb880d30: baseline-arm64-baylibre 2026-03-26 13:49:52,140 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c296794a2c5046eb880d30 2026-03-26 13:49:52,141 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 13:49:52,155 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c296794a2c5046eb880d30 HTTP/1.1" 200 None 2026-03-26 13:49:52,156 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 13:49:52,169 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf3545f1004e82292e0dc6 HTTP/1.1" 200 None 2026-03-26 13:49:52,170 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 13:49:52,192 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c296794a2c5046eb880d30 HTTP/1.1" 200 None 2026-03-26 13:49:52,198 - lava_callback - ERROR - Error processing callback for node 69c296794a2c5046eb880d30: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:35892 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41442 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41452 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50910 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50916 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40384 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36726 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55788 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39766 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56734 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56748 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59088 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59098 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35464 - "GET / HTTP/1.0" 200 OK 2026-03-26 13:51:32,385 - lava_callback - INFO - Received callback for node 69c296ce4a2c5046eb880d3f INFO: 172.21.0.1:47072 - "POST /node/69c296ce4a2c5046eb880d3f HTTP/1.0" 202 Accepted 2026-03-26 13:51:32,393 - lava_callback - INFO - Starting async job submit for node 69c296ce4a2c5046eb880d3f 2026-03-26 13:51:32,394 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 13:51:32,408 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c296ce4a2c5046eb880d3f HTTP/1.1" 200 None 2026-03-26 13:51:32,409 - lava_callback - DEBUG - Processing node 69c296ce4a2c5046eb880d3f: baseline-arm64-clabbe 2026-03-26 13:51:32,409 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c296ce4a2c5046eb880d3f 2026-03-26 13:51:32,410 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 13:51:32,423 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c296ce4a2c5046eb880d3f HTTP/1.1" 200 None 2026-03-26 13:51:32,424 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 13:51:32,439 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf3545f1004e82292e0dc6 HTTP/1.1" 200 None 2026-03-26 13:51:32,440 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 13:51:32,462 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c296ce4a2c5046eb880d3f HTTP/1.1" 200 None 2026-03-26 13:51:32,463 - lava_callback - ERROR - Error processing callback for node 69c296ce4a2c5046eb880d3f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:47078 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37502 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59292 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60096 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60110 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43086 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43092 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36048 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60772 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37562 - "GET / HTTP/1.0" 200 OK 2026-03-26 13:52:52,422 - lava_callback - INFO - Received callback for node 69c2971e4a2c5046eb880d4e INFO: 172.21.0.1:56926 - "POST /node/69c2971e4a2c5046eb880d4e HTTP/1.0" 202 Accepted 2026-03-26 13:52:52,428 - lava_callback - INFO - Starting async job submit for node 69c2971e4a2c5046eb880d4e 2026-03-26 13:52:52,429 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 13:52:52,444 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2971e4a2c5046eb880d4e HTTP/1.1" 200 None 2026-03-26 13:52:52,445 - lava_callback - DEBUG - Processing node 69c2971e4a2c5046eb880d4e: baseline-arm64-clabbe 2026-03-26 13:52:52,445 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2971e4a2c5046eb880d4e 2026-03-26 13:52:52,446 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 13:52:52,471 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2971e4a2c5046eb880d4e HTTP/1.1" 200 None 2026-03-26 13:52:52,472 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 13:52:52,485 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf3545f1004e82292e0dc6 HTTP/1.1" 200 None 2026-03-26 13:52:52,487 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 13:52:52,509 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2971e4a2c5046eb880d4e HTTP/1.1" 200 None 2026-03-26 13:52:52,511 - lava_callback - ERROR - Error processing callback for node 69c2971e4a2c5046eb880d4e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:56930 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58676 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58680 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58466 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58470 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41012 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58142 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38242 - "GET / HTTP/1.0" 200 OK 2026-03-26 13:53:52,361 - lava_callback - INFO - Received callback for node 69c2976d4a2c5046eb880d57 INFO: 172.21.0.1:49258 - "POST /node/69c2976d4a2c5046eb880d57 HTTP/1.0" 202 Accepted 2026-03-26 13:53:52,367 - lava_callback - INFO - Starting async job submit for node 69c2976d4a2c5046eb880d57 2026-03-26 13:53:52,368 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 13:53:52,388 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2976d4a2c5046eb880d57 HTTP/1.1" 200 None 2026-03-26 13:53:52,389 - lava_callback - DEBUG - Processing node 69c2976d4a2c5046eb880d57: baseline-arm64-clabbe 2026-03-26 13:53:52,389 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2976d4a2c5046eb880d57 2026-03-26 13:53:52,390 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 13:53:52,405 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2976d4a2c5046eb880d57 HTTP/1.1" 200 None 2026-03-26 13:53:52,407 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 13:53:52,422 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf3545f1004e82292e0dc6 HTTP/1.1" 200 None 2026-03-26 13:53:52,424 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 13:53:52,445 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2976d4a2c5046eb880d57 HTTP/1.1" 200 None 2026-03-26 13:53:52,447 - lava_callback - ERROR - Error processing callback for node 69c2976d4a2c5046eb880d57: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:49272 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57254 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57264 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46420 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46430 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42904 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48874 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41188 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51442 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35492 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35500 - "GET / HTTP/1.0" 200 OK 2026-03-26 13:55:12,515 - lava_callback - INFO - Received callback for node 69c297bc4a2c5046eb880d67 INFO: 172.21.0.1:39468 - "POST /node/69c297bc4a2c5046eb880d67 HTTP/1.0" 202 Accepted 2026-03-26 13:55:12,521 - lava_callback - INFO - Starting async job submit for node 69c297bc4a2c5046eb880d67 2026-03-26 13:55:12,522 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 13:55:12,536 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c297bc4a2c5046eb880d67 HTTP/1.1" 200 None 2026-03-26 13:55:12,537 - lava_callback - DEBUG - Processing node 69c297bc4a2c5046eb880d67: baseline-arm64-clabbe 2026-03-26 13:55:12,537 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c297bc4a2c5046eb880d67 2026-03-26 13:55:12,538 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 13:55:12,556 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c297bc4a2c5046eb880d67 HTTP/1.1" 200 None 2026-03-26 13:55:12,557 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 13:55:12,582 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf3545f1004e82292e0dc6 HTTP/1.1" 200 None 2026-03-26 13:55:12,584 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 13:55:12,604 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c297bc4a2c5046eb880d67 HTTP/1.1" 200 None 2026-03-26 13:55:12,605 - lava_callback - ERROR - Error processing callback for node 69c297bc4a2c5046eb880d67: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:39478 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39484 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47998 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56322 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47722 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56254 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44196 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44208 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43874 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43882 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38520 - "GET / HTTP/1.0" 200 OK 2026-03-26 13:56:32,584 - lava_callback - INFO - Received callback for node 69c2980d4a2c5046eb880d75 INFO: 172.21.0.1:45062 - "POST /node/69c2980d4a2c5046eb880d75 HTTP/1.0" 202 Accepted 2026-03-26 13:56:32,590 - lava_callback - INFO - Starting async job submit for node 69c2980d4a2c5046eb880d75 2026-03-26 13:56:32,590 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 13:56:32,607 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2980d4a2c5046eb880d75 HTTP/1.1" 200 None 2026-03-26 13:56:32,608 - lava_callback - DEBUG - Processing node 69c2980d4a2c5046eb880d75: baseline-arm64-clabbe 2026-03-26 13:56:32,608 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2980d4a2c5046eb880d75 2026-03-26 13:56:32,609 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 13:56:32,630 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2980d4a2c5046eb880d75 HTTP/1.1" 200 None 2026-03-26 13:56:32,632 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 13:56:32,647 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf3545f1004e82292e0dc6 HTTP/1.1" 200 None 2026-03-26 13:56:32,649 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 13:56:32,671 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2980d4a2c5046eb880d75 HTTP/1.1" 200 None 2026-03-26 13:56:32,672 - lava_callback - ERROR - Error processing callback for node 69c2980d4a2c5046eb880d75: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:45064 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59360 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59198 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38050 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38052 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52284 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52296 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45192 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46886 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42444 - "GET / HTTP/1.0" 200 OK 2026-03-26 13:57:53,019 - lava_callback - INFO - Received callback for node 69c2985e4a2c5046eb880d7f INFO: 172.21.0.1:60398 - "POST /node/69c2985e4a2c5046eb880d7f HTTP/1.0" 202 Accepted 2026-03-26 13:57:53,025 - lava_callback - INFO - Starting async job submit for node 69c2985e4a2c5046eb880d7f 2026-03-26 13:57:53,026 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 13:57:53,058 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2985e4a2c5046eb880d7f HTTP/1.1" 200 None 2026-03-26 13:57:53,059 - lava_callback - DEBUG - Processing node 69c2985e4a2c5046eb880d7f: baseline-arm64-clabbe 2026-03-26 13:57:53,060 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2985e4a2c5046eb880d7f 2026-03-26 13:57:53,060 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 13:57:53,085 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2985e4a2c5046eb880d7f HTTP/1.1" 200 None 2026-03-26 13:57:53,087 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 13:57:53,101 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf3545f1004e82292e0dc6 HTTP/1.1" 200 None 2026-03-26 13:57:53,102 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 13:57:53,124 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2985e4a2c5046eb880d7f HTTP/1.1" 200 None 2026-03-26 13:57:53,126 - lava_callback - ERROR - Error processing callback for node 69c2985e4a2c5046eb880d7f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:60402 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37200 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37204 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34138 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34154 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41016 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56904 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41212 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59210 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54880 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54882 - "GET / HTTP/1.0" 200 OK 2026-03-26 13:59:12,728 - lava_callback - INFO - Received callback for node 69c298ad4a2c5046eb880d8e INFO: 172.21.0.1:57974 - "POST /node/69c298ad4a2c5046eb880d8e HTTP/1.0" 202 Accepted 2026-03-26 13:59:12,734 - lava_callback - INFO - Starting async job submit for node 69c298ad4a2c5046eb880d8e 2026-03-26 13:59:12,735 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 13:59:12,750 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c298ad4a2c5046eb880d8e HTTP/1.1" 200 None 2026-03-26 13:59:12,751 - lava_callback - DEBUG - Processing node 69c298ad4a2c5046eb880d8e: baseline-arm64-clabbe 2026-03-26 13:59:12,751 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c298ad4a2c5046eb880d8e 2026-03-26 13:59:12,752 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 13:59:12,766 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c298ad4a2c5046eb880d8e HTTP/1.1" 200 None 2026-03-26 13:59:12,768 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 13:59:12,781 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf3545f1004e82292e0dc6 HTTP/1.1" 200 None 2026-03-26 13:59:12,782 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 13:59:12,803 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c298ad4a2c5046eb880d8e HTTP/1.1" 200 None 2026-03-26 13:59:12,809 - lava_callback - ERROR - Error processing callback for node 69c298ad4a2c5046eb880d8e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:57988 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57990 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60678 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48930 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53034 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38654 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55442 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55458 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33330 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33342 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51746 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52852 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49178 - "GET / HTTP/1.0" 200 OK 2026-03-26 14:00:52,764 - lava_callback - INFO - Received callback for node 69c298ff4a2c5046eb880d9b INFO: 172.21.0.1:46484 - "POST /node/69c298ff4a2c5046eb880d9b HTTP/1.0" 202 Accepted 2026-03-26 14:00:52,770 - lava_callback - INFO - Starting async job submit for node 69c298ff4a2c5046eb880d9b 2026-03-26 14:00:52,771 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 14:00:52,799 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c298ff4a2c5046eb880d9b HTTP/1.1" 200 None 2026-03-26 14:00:52,800 - lava_callback - DEBUG - Processing node 69c298ff4a2c5046eb880d9b: baseline-arm64-clabbe 2026-03-26 14:00:52,800 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c298ff4a2c5046eb880d9b 2026-03-26 14:00:52,800 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 14:00:52,814 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c298ff4a2c5046eb880d9b HTTP/1.1" 200 None 2026-03-26 14:00:52,816 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 14:00:52,828 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf3545f1004e82292e0dc6 HTTP/1.1" 200 None 2026-03-26 14:00:52,830 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 14:00:52,850 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c298ff4a2c5046eb880d9b HTTP/1.1" 200 None 2026-03-26 14:00:52,852 - lava_callback - ERROR - Error processing callback for node 69c298ff4a2c5046eb880d9b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:46496 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38594 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38604 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44484 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44494 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46816 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41874 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47886 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58036 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35520 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35536 - "GET / HTTP/1.0" 200 OK 2026-03-26 14:02:12,917 - lava_callback - INFO - Received callback for node 69c2994f4a2c5046eb880da2 INFO: 172.21.0.1:53498 - "POST /node/69c2994f4a2c5046eb880da2 HTTP/1.0" 202 Accepted 2026-03-26 14:02:12,922 - lava_callback - INFO - Starting async job submit for node 69c2994f4a2c5046eb880da2 2026-03-26 14:02:12,923 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 14:02:12,937 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2994f4a2c5046eb880da2 HTTP/1.1" 200 None 2026-03-26 14:02:12,938 - lava_callback - DEBUG - Processing node 69c2994f4a2c5046eb880da2: baseline-arm64-clabbe 2026-03-26 14:02:12,938 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2994f4a2c5046eb880da2 2026-03-26 14:02:12,939 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 14:02:12,956 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2994f4a2c5046eb880da2 HTTP/1.1" 200 None 2026-03-26 14:02:12,957 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 14:02:12,971 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf3545f1004e82292e0dc6 HTTP/1.1" 200 None 2026-03-26 14:02:12,973 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 14:02:12,993 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2994f4a2c5046eb880da2 HTTP/1.1" 200 None 2026-03-26 14:02:13,003 - lava_callback - ERROR - Error processing callback for node 69c2994f4a2c5046eb880da2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:53510 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53522 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36100 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34940 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54598 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53896 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54422 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54436 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49246 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49260 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59314 - "GET / HTTP/1.0" 200 OK 2026-03-26 14:03:32,974 - lava_callback - INFO - Received callback for node 69c2999e4a2c5046eb880dae INFO: 172.21.0.1:52254 - "POST /node/69c2999e4a2c5046eb880dae HTTP/1.0" 202 Accepted 2026-03-26 14:03:32,979 - lava_callback - INFO - Starting async job submit for node 69c2999e4a2c5046eb880dae 2026-03-26 14:03:32,980 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 14:03:32,995 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2999e4a2c5046eb880dae HTTP/1.1" 200 None 2026-03-26 14:03:32,996 - lava_callback - DEBUG - Processing node 69c2999e4a2c5046eb880dae: baseline-arm64-clabbe 2026-03-26 14:03:32,996 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2999e4a2c5046eb880dae 2026-03-26 14:03:32,997 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 14:03:33,010 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2999e4a2c5046eb880dae HTTP/1.1" 200 None 2026-03-26 14:03:33,011 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 14:03:33,024 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf3545f1004e82292e0dc6 HTTP/1.1" 200 None 2026-03-26 14:03:33,025 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 14:03:33,045 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2999e4a2c5046eb880dae HTTP/1.1" 200 None 2026-03-26 14:03:33,047 - lava_callback - ERROR - Error processing callback for node 69c2999e4a2c5046eb880dae: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:52268 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60864 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44126 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37508 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37518 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40404 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40410 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37906 - "GET / HTTP/1.0" 200 OK 2026-03-26 14:04:32,935 - lava_callback - INFO - Received callback for node 69c299ed4a2c5046eb880dbc INFO: 172.21.0.1:59008 - "POST /node/69c299ed4a2c5046eb880dbc HTTP/1.0" 202 Accepted 2026-03-26 14:04:32,941 - lava_callback - INFO - Starting async job submit for node 69c299ed4a2c5046eb880dbc 2026-03-26 14:04:32,941 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 14:04:32,956 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c299ed4a2c5046eb880dbc HTTP/1.1" 200 None 2026-03-26 14:04:32,957 - lava_callback - DEBUG - Processing node 69c299ed4a2c5046eb880dbc: baseline-arm64-clabbe 2026-03-26 14:04:32,957 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c299ed4a2c5046eb880dbc 2026-03-26 14:04:32,958 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 14:04:32,971 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c299ed4a2c5046eb880dbc HTTP/1.1" 200 None 2026-03-26 14:04:32,973 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 14:04:32,987 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf3545f1004e82292e0dc6 HTTP/1.1" 200 None 2026-03-26 14:04:32,989 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 14:04:33,010 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c299ed4a2c5046eb880dbc HTTP/1.1" 200 None 2026-03-26 14:04:33,011 - lava_callback - ERROR - Error processing callback for node 69c299ed4a2c5046eb880dbc: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:59010 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44904 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50392 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33374 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33386 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52076 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52090 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54888 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49802 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43116 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57544 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58246 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58262 - "GET / HTTP/1.0" 200 OK 2026-03-26 14:06:13,047 - lava_callback - INFO - Received callback for node 69c29a3f4a2c5046eb880dc1 INFO: 172.21.0.1:54896 - "POST /node/69c29a3f4a2c5046eb880dc1 HTTP/1.0" 202 Accepted 2026-03-26 14:06:13,052 - lava_callback - INFO - Starting async job submit for node 69c29a3f4a2c5046eb880dc1 2026-03-26 14:06:13,053 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 14:06:13,068 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c29a3f4a2c5046eb880dc1 HTTP/1.1" 200 None 2026-03-26 14:06:13,069 - lava_callback - DEBUG - Processing node 69c29a3f4a2c5046eb880dc1: baseline-arm64-clabbe 2026-03-26 14:06:13,069 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c29a3f4a2c5046eb880dc1 2026-03-26 14:06:13,070 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 14:06:13,083 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c29a3f4a2c5046eb880dc1 HTTP/1.1" 200 None 2026-03-26 14:06:13,084 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 14:06:13,098 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf3545f1004e82292e0dc6 HTTP/1.1" 200 None 2026-03-26 14:06:13,100 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 14:06:13,122 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c29a3f4a2c5046eb880dc1 HTTP/1.1" 200 None 2026-03-26 14:06:13,123 - lava_callback - ERROR - Error processing callback for node 69c29a3f4a2c5046eb880dc1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:54910 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54912 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39144 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55722 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36432 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48646 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48840 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48854 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47724 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47738 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54666 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56564 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52618 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49704 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39100 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39104 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36706 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36710 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40602 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60024 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56850 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35314 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46982 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46988 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34800 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34816 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52284 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60368 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52716 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56076 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44236 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44250 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36294 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36306 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52702 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35282 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60126 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38500 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45454 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45468 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41680 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41694 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50882 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56800 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59494 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52114 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51620 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51624 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50282 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50290 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45268 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37922 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37594 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34916 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49450 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49462 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55058 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55072 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41198 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45206 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50730 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37382 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37234 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37246 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43326 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43332 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60416 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52870 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57802 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59944 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35272 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35274 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39704 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39712 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40044 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49490 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38066 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35426 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33062 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33076 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60684 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60686 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39074 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56422 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50832 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54594 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52132 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52140 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55632 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55644 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37474 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43550 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59100 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48336 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45536 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45542 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48370 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48386 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38814 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55660 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36868 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55112 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37188 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37198 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46966 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46978 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48440 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37048 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57044 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49460 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48262 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48264 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37452 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37468 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59542 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33506 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50260 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51414 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49332 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49348 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35412 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35418 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58054 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39838 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56374 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36654 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52030 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52038 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34838 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34842 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38810 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51544 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47948 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42212 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38118 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38122 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54708 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54714 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55170 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59518 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41302 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44512 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47736 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47752 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53658 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53672 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53898 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45754 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36902 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36602 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33936 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33942 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57428 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57438 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56370 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51134 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58626 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41932 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60214 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60222 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40378 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40380 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52728 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45902 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54926 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51070 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53706 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53714 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48432 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48436 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52990 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55444 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34104 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54444 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45472 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45484 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58472 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58478 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55242 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37050 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56868 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46832 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57862 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57872 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46464 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46468 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55474 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55988 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58196 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37708 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37048 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37062 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43426 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43434 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51376 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39936 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45988 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45988 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52998 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53006 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45066 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45070 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50046 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52512 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38938 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37816 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58468 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58484 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38244 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38254 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51850 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45230 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36980 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33340 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57072 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57086 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42208 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42214 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38010 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60598 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59950 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42570 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38750 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38760 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53206 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53212 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60330 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53456 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55364 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46136 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53752 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53768 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49982 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49984 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48920 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55040 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33320 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42812 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54916 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54928 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59382 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59398 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59822 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47712 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37924 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33166 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36132 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36148 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48220 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48236 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60336 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36490 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44684 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47506 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39010 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39022 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57940 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57946 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59666 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60010 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54766 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53326 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58866 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58882 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37410 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37426 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40420 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39836 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32950 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34422 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44884 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44898 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57110 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57124 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55730 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40880 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57814 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54632 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34764 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34770 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44562 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44568 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42706 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51802 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37084 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41172 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51894 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51902 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33658 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33674 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60674 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32934 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38760 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49294 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55676 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55680 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34550 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34560 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42710 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47788 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55794 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51502 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56580 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56586 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52488 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52494 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54416 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34118 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47620 - "GET / HTTP/1.0" 200 OK 2026-03-26 14:44:55,024 - lava_callback - INFO - Received callback for node 69c2a3584a2c5046eb880e47 INFO: 172.21.0.1:50168 - "POST /node/69c2a3584a2c5046eb880e47 HTTP/1.0" 202 Accepted 2026-03-26 14:44:55,032 - lava_callback - INFO - Starting async job submit for node 69c2a3584a2c5046eb880e47 2026-03-26 14:44:55,034 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 14:44:55,049 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2a3584a2c5046eb880e47 HTTP/1.1" 200 None 2026-03-26 14:44:55,049 - lava_callback - DEBUG - Processing node 69c2a3584a2c5046eb880e47: baseline-arm-baylibre 2026-03-26 14:44:55,050 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2a3584a2c5046eb880e47 2026-03-26 14:44:55,050 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 14:44:55,069 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2a3584a2c5046eb880e47 HTTP/1.1" 200 None 2026-03-26 14:44:55,071 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 14:44:55,087 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248d74a2c5046eb87f890 HTTP/1.1" 200 None 2026-03-26 14:44:55,088 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 14:44:55,119 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2a3584a2c5046eb880e47 HTTP/1.1" 200 None 2026-03-26 14:44:55,130 - lava_callback - ERROR - Error processing callback for node 69c2a3584a2c5046eb880e47: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:50184 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55164 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55166 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33634 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33636 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34808 - "GET / HTTP/1.0" 200 OK 2026-03-26 14:45:34,856 - lava_callback - INFO - Received callback for node 69c2a37b4a2c5046eb880e60 INFO: 172.21.0.1:34766 - "POST /node/69c2a37b4a2c5046eb880e60 HTTP/1.0" 202 Accepted 2026-03-26 14:45:34,862 - lava_callback - INFO - Starting async job submit for node 69c2a37b4a2c5046eb880e60 2026-03-26 14:45:34,863 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 14:45:34,878 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2a37b4a2c5046eb880e60 HTTP/1.1" 200 None 2026-03-26 14:45:34,879 - lava_callback - DEBUG - Processing node 69c2a37b4a2c5046eb880e60: baseline-arm-baylibre 2026-03-26 14:45:34,879 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2a37b4a2c5046eb880e60 2026-03-26 14:45:34,880 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 14:45:34,893 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2a37b4a2c5046eb880e60 HTTP/1.1" 200 None 2026-03-26 14:45:34,895 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 14:45:34,911 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248d74a2c5046eb87f890 HTTP/1.1" 200 None 2026-03-26 14:45:34,912 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 14:45:34,934 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2a37b4a2c5046eb880e60 HTTP/1.1" 200 None 2026-03-26 14:45:34,935 - lava_callback - ERROR - Error processing callback for node 69c2a37b4a2c5046eb880e60: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:34768 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56672 - "GET / HTTP/1.0" 200 OK 2026-03-26 14:45:54,942 - lava_callback - INFO - Received callback for node 69c2a39f4a2c5046eb880e70 INFO: 172.21.0.1:56566 - "POST /node/69c2a39f4a2c5046eb880e70 HTTP/1.0" 202 Accepted 2026-03-26 14:45:54,947 - lava_callback - INFO - Starting async job submit for node 69c2a39f4a2c5046eb880e70 2026-03-26 14:45:54,948 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 14:45:54,962 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2a39f4a2c5046eb880e70 HTTP/1.1" 200 None 2026-03-26 14:45:54,963 - lava_callback - DEBUG - Processing node 69c2a39f4a2c5046eb880e70: baseline-arm-baylibre 2026-03-26 14:45:54,963 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2a39f4a2c5046eb880e70 2026-03-26 14:45:54,964 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 14:45:54,979 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2a39f4a2c5046eb880e70 HTTP/1.1" 200 None 2026-03-26 14:45:54,981 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 14:45:55,013 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248d74a2c5046eb87f890 HTTP/1.1" 200 None 2026-03-26 14:45:55,015 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 14:45:55,038 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2a39f4a2c5046eb880e70 HTTP/1.1" 200 None 2026-03-26 14:45:55,039 - lava_callback - ERROR - Error processing callback for node 69c2a39f4a2c5046eb880e70: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:56568 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53760 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53772 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37340 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37350 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33458 - "GET / HTTP/1.0" 200 OK 2026-03-26 14:46:34,980 - lava_callback - INFO - Received callback for node 69c2a3c34a2c5046eb880e7d INFO: 172.21.0.1:40564 - "POST /node/69c2a3c34a2c5046eb880e7d HTTP/1.0" 202 Accepted 2026-03-26 14:46:34,985 - lava_callback - INFO - Starting async job submit for node 69c2a3c34a2c5046eb880e7d 2026-03-26 14:46:34,986 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 14:46:35,011 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2a3c34a2c5046eb880e7d HTTP/1.1" 200 None 2026-03-26 14:46:35,013 - lava_callback - DEBUG - Processing node 69c2a3c34a2c5046eb880e7d: baseline-arm-baylibre 2026-03-26 14:46:35,013 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2a3c34a2c5046eb880e7d 2026-03-26 14:46:35,014 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 14:46:35,027 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2a3c34a2c5046eb880e7d HTTP/1.1" 200 None 2026-03-26 14:46:35,028 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 14:46:35,041 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248d74a2c5046eb87f890 HTTP/1.1" 200 None 2026-03-26 14:46:35,043 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 14:46:35,063 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2a3c34a2c5046eb880e7d HTTP/1.1" 200 None 2026-03-26 14:46:35,064 - lava_callback - ERROR - Error processing callback for node 69c2a3c34a2c5046eb880e7d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:40566 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40578 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40590 - "GET /favicon.ico HTTP/1.0" 404 Not Found INFO: 172.21.0.1:56514 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55262 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38726 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38734 - "GET / HTTP/1.0" 200 OK 2026-03-26 14:47:15,024 - lava_callback - INFO - Received callback for node 69c2a3e84a2c5046eb880e8b INFO: 172.21.0.1:56446 - "POST /node/69c2a3e84a2c5046eb880e8b HTTP/1.0" 202 Accepted 2026-03-26 14:47:15,030 - lava_callback - INFO - Starting async job submit for node 69c2a3e84a2c5046eb880e8b 2026-03-26 14:47:15,031 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 14:47:15,045 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2a3e84a2c5046eb880e8b HTTP/1.1" 200 None 2026-03-26 14:47:15,046 - lava_callback - DEBUG - Processing node 69c2a3e84a2c5046eb880e8b: baseline-arm-baylibre 2026-03-26 14:47:15,047 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2a3e84a2c5046eb880e8b 2026-03-26 14:47:15,047 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 14:47:15,061 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2a3e84a2c5046eb880e8b HTTP/1.1" 200 None 2026-03-26 14:47:15,062 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 14:47:15,076 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248d74a2c5046eb87f890 HTTP/1.1" 200 None 2026-03-26 14:47:15,077 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 14:47:15,101 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2a3e84a2c5046eb880e8b HTTP/1.1" 200 None 2026-03-26 14:47:15,103 - lava_callback - ERROR - Error processing callback for node 69c2a3e84a2c5046eb880e8b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:56454 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56458 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60110 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39334 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36356 - "GET / HTTP/1.0" 200 OK 2026-03-26 14:47:55,060 - lava_callback - INFO - Received callback for node 69c2a40c4a2c5046eb880e95 INFO: 172.21.0.1:51776 - "POST /node/69c2a40c4a2c5046eb880e95 HTTP/1.0" 202 Accepted 2026-03-26 14:47:55,066 - lava_callback - INFO - Starting async job submit for node 69c2a40c4a2c5046eb880e95 2026-03-26 14:47:55,067 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 14:47:55,102 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2a40c4a2c5046eb880e95 HTTP/1.1" 200 None 2026-03-26 14:47:55,103 - lava_callback - DEBUG - Processing node 69c2a40c4a2c5046eb880e95: baseline-arm-baylibre 2026-03-26 14:47:55,103 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2a40c4a2c5046eb880e95 2026-03-26 14:47:55,104 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 14:47:55,134 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2a40c4a2c5046eb880e95 HTTP/1.1" 200 None 2026-03-26 14:47:55,135 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 14:47:55,151 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248d74a2c5046eb87f890 HTTP/1.1" 200 None 2026-03-26 14:47:55,153 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 14:47:55,174 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2a40c4a2c5046eb880e95 HTTP/1.1" 200 None 2026-03-26 14:47:55,184 - lava_callback - ERROR - Error processing callback for node 69c2a40c4a2c5046eb880e95: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:51792 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59448 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59454 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39422 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39426 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50944 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35918 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57216 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34438 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51930 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51936 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36204 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36216 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45972 - "GET / HTTP/1.0" 200 OK 2026-03-26 14:49:35,139 - lava_callback - INFO - Received callback for node 69c2a4774a2c5046eb880ebb INFO: 172.21.0.1:47874 - "POST /node/69c2a4774a2c5046eb880ebb HTTP/1.0" 202 Accepted 2026-03-26 14:49:35,145 - lava_callback - INFO - Starting async job submit for node 69c2a4774a2c5046eb880ebb 2026-03-26 14:49:35,146 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 14:49:35,172 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2a4774a2c5046eb880ebb HTTP/1.1" 200 None 2026-03-26 14:49:35,173 - lava_callback - DEBUG - Processing node 69c2a4774a2c5046eb880ebb: baseline-arm-clabbe 2026-03-26 14:49:35,173 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2a4774a2c5046eb880ebb 2026-03-26 14:49:35,174 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 14:49:35,190 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2a4774a2c5046eb880ebb HTTP/1.1" 200 None 2026-03-26 14:49:35,192 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 14:49:35,205 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248d74a2c5046eb87f890 HTTP/1.1" 200 None 2026-03-26 14:49:35,206 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 14:49:35,232 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2a4774a2c5046eb880ebb HTTP/1.1" 200 None 2026-03-26 14:49:35,233 - lava_callback - ERROR - Error processing callback for node 69c2a4774a2c5046eb880ebb: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:47884 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55046 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44758 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38822 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38832 - "GET / HTTP/1.0" 200 OK 2026-03-26 14:50:14,818 - lava_callback - INFO - Received callback for node 69c2a49b4a2c5046eb880ec7 INFO: 172.21.0.1:36730 - "POST /node/69c2a49b4a2c5046eb880ec7 HTTP/1.0" 202 Accepted 2026-03-26 14:50:14,824 - lava_callback - INFO - Starting async job submit for node 69c2a49b4a2c5046eb880ec7 2026-03-26 14:50:14,825 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 14:50:14,839 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2a49b4a2c5046eb880ec7 HTTP/1.1" 200 None 2026-03-26 14:50:14,840 - lava_callback - DEBUG - Processing node 69c2a49b4a2c5046eb880ec7: baseline-arm-clabbe 2026-03-26 14:50:14,841 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2a49b4a2c5046eb880ec7 2026-03-26 14:50:14,841 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 14:50:14,854 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2a49b4a2c5046eb880ec7 HTTP/1.1" 200 None 2026-03-26 14:50:14,856 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 14:50:14,870 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248d74a2c5046eb87f890 HTTP/1.1" 200 None 2026-03-26 14:50:14,871 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 14:50:14,894 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2a49b4a2c5046eb880ec7 HTTP/1.1" 200 None 2026-03-26 14:50:14,903 - lava_callback - ERROR - Error processing callback for node 69c2a49b4a2c5046eb880ec7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:36736 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36752 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60814 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58092 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59414 - "GET / HTTP/1.0" 200 OK 2026-03-26 14:50:54,958 - lava_callback - INFO - Received callback for node 69c2a4c04a2c5046eb880ecf INFO: 172.21.0.1:37652 - "POST /node/69c2a4c04a2c5046eb880ecf HTTP/1.0" 202 Accepted 2026-03-26 14:50:54,964 - lava_callback - INFO - Starting async job submit for node 69c2a4c04a2c5046eb880ecf 2026-03-26 14:50:54,964 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 14:50:54,985 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2a4c04a2c5046eb880ecf HTTP/1.1" 200 None 2026-03-26 14:50:54,986 - lava_callback - DEBUG - Processing node 69c2a4c04a2c5046eb880ecf: baseline-arm-clabbe 2026-03-26 14:50:54,986 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2a4c04a2c5046eb880ecf 2026-03-26 14:50:54,987 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 14:50:55,000 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2a4c04a2c5046eb880ecf HTTP/1.1" 200 None 2026-03-26 14:50:55,001 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 14:50:55,033 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248d74a2c5046eb87f890 HTTP/1.1" 200 None 2026-03-26 14:50:55,034 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 14:50:55,056 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2a4c04a2c5046eb880ecf HTTP/1.1" 200 None 2026-03-26 14:50:55,058 - lava_callback - ERROR - Error processing callback for node 69c2a4c04a2c5046eb880ecf: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:37666 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43770 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43780 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35010 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35018 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60098 - "GET / HTTP/1.0" 200 OK 2026-03-26 14:51:34,946 - lava_callback - INFO - Received callback for node 69c2a4e54a2c5046eb880eda INFO: 172.21.0.1:45862 - "POST /node/69c2a4e54a2c5046eb880eda HTTP/1.0" 202 Accepted 2026-03-26 14:51:34,951 - lava_callback - INFO - Starting async job submit for node 69c2a4e54a2c5046eb880eda 2026-03-26 14:51:34,952 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 14:51:34,966 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2a4e54a2c5046eb880eda HTTP/1.1" 200 None 2026-03-26 14:51:34,967 - lava_callback - DEBUG - Processing node 69c2a4e54a2c5046eb880eda: baseline-arm-clabbe 2026-03-26 14:51:34,967 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2a4e54a2c5046eb880eda 2026-03-26 14:51:34,968 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 14:51:35,021 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2a4e54a2c5046eb880eda HTTP/1.1" 200 None 2026-03-26 14:51:35,023 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 14:51:35,036 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248d74a2c5046eb87f890 HTTP/1.1" 200 None 2026-03-26 14:51:35,038 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 14:51:35,059 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2a4e54a2c5046eb880eda HTTP/1.1" 200 None 2026-03-26 14:51:35,060 - lava_callback - ERROR - Error processing callback for node 69c2a4e54a2c5046eb880eda: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:45874 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41062 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45792 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59272 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59284 - "GET / HTTP/1.0" 200 OK 2026-03-26 14:52:15,064 - lava_callback - INFO - Received callback for node 69c2a50c4a2c5046eb880ee5 INFO: 172.21.0.1:40270 - "POST /node/69c2a50c4a2c5046eb880ee5 HTTP/1.0" 202 Accepted 2026-03-26 14:52:15,072 - lava_callback - INFO - Starting async job submit for node 69c2a50c4a2c5046eb880ee5 2026-03-26 14:52:15,072 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 14:52:15,089 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2a50c4a2c5046eb880ee5 HTTP/1.1" 200 None 2026-03-26 14:52:15,090 - lava_callback - DEBUG - Processing node 69c2a50c4a2c5046eb880ee5: baseline-arm-clabbe 2026-03-26 14:52:15,090 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2a50c4a2c5046eb880ee5 2026-03-26 14:52:15,091 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 14:52:15,105 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2a50c4a2c5046eb880ee5 HTTP/1.1" 200 None 2026-03-26 14:52:15,107 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 14:52:15,132 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248d74a2c5046eb87f890 HTTP/1.1" 200 None 2026-03-26 14:52:15,133 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 14:52:15,155 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2a50c4a2c5046eb880ee5 HTTP/1.1" 200 None 2026-03-26 14:52:15,157 - lava_callback - ERROR - Error processing callback for node 69c2a50c4a2c5046eb880ee5: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:40276 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40284 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58474 - "GET / HTTP/1.0" 200 OK 2026-03-26 14:52:35,379 - lava_callback - INFO - Received callback for node 69c2a52f4a2c5046eb880ef2 INFO: 172.21.0.1:43050 - "POST /node/69c2a52f4a2c5046eb880ef2 HTTP/1.0" 202 Accepted 2026-03-26 14:52:35,384 - lava_callback - INFO - Starting async job submit for node 69c2a52f4a2c5046eb880ef2 2026-03-26 14:52:35,385 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 14:52:35,409 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2a52f4a2c5046eb880ef2 HTTP/1.1" 200 None 2026-03-26 14:52:35,410 - lava_callback - DEBUG - Processing node 69c2a52f4a2c5046eb880ef2: baseline-arm-baylibre 2026-03-26 14:52:35,410 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2a52f4a2c5046eb880ef2 2026-03-26 14:52:35,411 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 14:52:35,424 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2a52f4a2c5046eb880ef2 HTTP/1.1" 200 None 2026-03-26 14:52:35,426 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 14:52:35,443 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd026d05e4bbd6ba634c57 HTTP/1.1" 200 None 2026-03-26 14:52:35,445 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 14:52:35,468 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2a52f4a2c5046eb880ef2 HTTP/1.1" 200 None 2026-03-26 14:52:35,473 - lava_callback - ERROR - Error processing callback for node 69c2a52f4a2c5046eb880ef2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:43062 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54738 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60326 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46212 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46222 - "GET / HTTP/1.0" 200 OK 2026-03-26 14:53:15,048 - lava_callback - INFO - Received callback for node 69c2a5554a2c5046eb880efa INFO: 172.21.0.1:52408 - "POST /node/69c2a5554a2c5046eb880efa HTTP/1.0" 202 Accepted 2026-03-26 14:53:15,054 - lava_callback - INFO - Starting async job submit for node 69c2a5554a2c5046eb880efa 2026-03-26 14:53:15,055 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 14:53:15,089 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2a5554a2c5046eb880efa HTTP/1.1" 200 None 2026-03-26 14:53:15,090 - lava_callback - DEBUG - Processing node 69c2a5554a2c5046eb880efa: baseline-arm-clabbe 2026-03-26 14:53:15,090 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2a5554a2c5046eb880efa 2026-03-26 14:53:15,090 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 14:53:15,104 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2a5554a2c5046eb880efa HTTP/1.1" 200 None 2026-03-26 14:53:15,106 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 14:53:15,119 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd026d05e4bbd6ba634c57 HTTP/1.1" 200 None 2026-03-26 14:53:15,121 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.21.0.1:52422 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-26 14:53:15,172 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2a5554a2c5046eb880efa HTTP/1.1" 200 None 2026-03-26 14:53:15,174 - lava_callback - ERROR - Error processing callback for node 69c2a5554a2c5046eb880efa: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:52428 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55714 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58210 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47048 - "GET / HTTP/1.0" 200 OK 2026-03-26 14:53:55,054 - lava_callback - INFO - Received callback for node 69c2a57a4a2c5046eb880f0d INFO: 172.21.0.1:41106 - "POST /node/69c2a57a4a2c5046eb880f0d HTTP/1.0" 202 Accepted 2026-03-26 14:53:55,059 - lava_callback - INFO - Starting async job submit for node 69c2a57a4a2c5046eb880f0d 2026-03-26 14:53:55,060 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 14:53:55,075 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2a57a4a2c5046eb880f0d HTTP/1.1" 200 None 2026-03-26 14:53:55,076 - lava_callback - DEBUG - Processing node 69c2a57a4a2c5046eb880f0d: baseline-arm-clabbe 2026-03-26 14:53:55,076 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2a57a4a2c5046eb880f0d 2026-03-26 14:53:55,077 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 14:53:55,090 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2a57a4a2c5046eb880f0d HTTP/1.1" 200 None 2026-03-26 14:53:55,091 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 14:53:55,104 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd026d05e4bbd6ba634c57 HTTP/1.1" 200 None 2026-03-26 14:53:55,105 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 14:53:55,125 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2a57a4a2c5046eb880f0d HTTP/1.1" 200 None 2026-03-26 14:53:55,127 - lava_callback - ERROR - Error processing callback for node 69c2a57a4a2c5046eb880f0d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:41118 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60430 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60438 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39226 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39238 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34430 - "GET / HTTP/1.0" 200 OK 2026-03-26 14:54:35,440 - lava_callback - INFO - Received callback for node 69c2a59d4a2c5046eb880f1a INFO: 172.21.0.1:47938 - "POST /node/69c2a59d4a2c5046eb880f1a HTTP/1.0" 202 Accepted 2026-03-26 14:54:35,446 - lava_callback - INFO - Starting async job submit for node 69c2a59d4a2c5046eb880f1a 2026-03-26 14:54:35,447 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 14:54:35,461 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2a59d4a2c5046eb880f1a HTTP/1.1" 200 None 2026-03-26 14:54:35,462 - lava_callback - DEBUG - Processing node 69c2a59d4a2c5046eb880f1a: baseline-arm-baylibre 2026-03-26 14:54:35,462 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2a59d4a2c5046eb880f1a 2026-03-26 14:54:35,462 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 14:54:35,476 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2a59d4a2c5046eb880f1a HTTP/1.1" 200 None 2026-03-26 14:54:35,477 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 14:54:35,494 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd026d05e4bbd6ba634c57 HTTP/1.1" 200 None 2026-03-26 14:54:35,496 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 14:54:35,528 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2a59d4a2c5046eb880f1a HTTP/1.1" 200 None 2026-03-26 14:54:35,533 - lava_callback - ERROR - Error processing callback for node 69c2a59d4a2c5046eb880f1a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:47948 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32906 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41728 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53858 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53872 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39946 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-26 14:55:15,256 - lava_callback - INFO - Received callback for node 69c2a5c34a2c5046eb880f25 INFO: 172.21.0.1:39956 - "POST /node/69c2a5c34a2c5046eb880f25 HTTP/1.0" 202 Accepted 2026-03-26 14:55:15,262 - lava_callback - INFO - Starting async job submit for node 69c2a5c34a2c5046eb880f25 2026-03-26 14:55:15,263 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 14:55:15,277 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2a5c34a2c5046eb880f25 HTTP/1.1" 200 None 2026-03-26 14:55:15,278 - lava_callback - DEBUG - Processing node 69c2a5c34a2c5046eb880f25: baseline-arm-clabbe 2026-03-26 14:55:15,278 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2a5c34a2c5046eb880f25 2026-03-26 14:55:15,279 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 14:55:15,292 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2a5c34a2c5046eb880f25 HTTP/1.1" 200 None 2026-03-26 14:55:15,293 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 14:55:15,306 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd026d05e4bbd6ba634c57 HTTP/1.1" 200 None 2026-03-26 14:55:15,307 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 14:55:15,327 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2a5c34a2c5046eb880f25 HTTP/1.1" 200 None 2026-03-26 14:55:15,328 - lava_callback - ERROR - Error processing callback for node 69c2a5c34a2c5046eb880f25: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:39972 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51426 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44436 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59360 - "GET / HTTP/1.0" 200 OK 2026-03-26 14:55:55,105 - lava_callback - INFO - Received callback for node 69c2a5e74a2c5046eb880f3e INFO: 172.21.0.1:37908 - "POST /node/69c2a5e74a2c5046eb880f3e HTTP/1.0" 202 Accepted 2026-03-26 14:55:55,110 - lava_callback - INFO - Starting async job submit for node 69c2a5e74a2c5046eb880f3e 2026-03-26 14:55:55,111 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 14:55:55,135 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2a5e74a2c5046eb880f3e HTTP/1.1" 200 None 2026-03-26 14:55:55,135 - lava_callback - DEBUG - Processing node 69c2a5e74a2c5046eb880f3e: baseline-arm-clabbe 2026-03-26 14:55:55,136 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2a5e74a2c5046eb880f3e 2026-03-26 14:55:55,136 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 14:55:55,149 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2a5e74a2c5046eb880f3e HTTP/1.1" 200 None 2026-03-26 14:55:55,151 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 14:55:55,164 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd026d05e4bbd6ba634c57 HTTP/1.1" 200 None 2026-03-26 14:55:55,165 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 14:55:55,185 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2a5e74a2c5046eb880f3e HTTP/1.1" 200 None 2026-03-26 14:55:55,186 - lava_callback - ERROR - Error processing callback for node 69c2a5e74a2c5046eb880f3e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:37916 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55280 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55292 - "GET / HTTP/1.0" 200 OK 2026-03-26 14:56:15,142 - lava_callback - INFO - Received callback for node 69c2a60c4a2c5046eb880f4f INFO: 172.21.0.1:47150 - "POST /node/69c2a60c4a2c5046eb880f4f HTTP/1.0" 202 Accepted 2026-03-26 14:56:15,147 - lava_callback - INFO - Starting async job submit for node 69c2a60c4a2c5046eb880f4f 2026-03-26 14:56:15,148 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.21.0.1:47158 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-26 14:56:15,163 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2a60c4a2c5046eb880f4f HTTP/1.1" 200 None 2026-03-26 14:56:15,164 - lava_callback - DEBUG - Processing node 69c2a60c4a2c5046eb880f4f: baseline-arm-clabbe 2026-03-26 14:56:15,164 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2a60c4a2c5046eb880f4f 2026-03-26 14:56:15,165 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 14:56:15,179 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2a60c4a2c5046eb880f4f HTTP/1.1" 200 None 2026-03-26 14:56:15,181 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 14:56:15,194 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd026d05e4bbd6ba634c57 HTTP/1.1" 200 None 2026-03-26 14:56:15,195 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 14:56:15,216 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2a60c4a2c5046eb880f4f HTTP/1.1" 200 None 2026-03-26 14:56:15,225 - lava_callback - ERROR - Error processing callback for node 69c2a60c4a2c5046eb880f4f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:47170 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49098 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39488 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34098 - "GET / HTTP/1.0" 200 OK 2026-03-26 14:56:55,157 - lava_callback - INFO - Received callback for node 69c2a6304a2c5046eb880f6b INFO: 172.21.0.1:55416 - "POST /node/69c2a6304a2c5046eb880f6b HTTP/1.0" 202 Accepted 2026-03-26 14:56:55,162 - lava_callback - INFO - Starting async job submit for node 69c2a6304a2c5046eb880f6b 2026-03-26 14:56:55,163 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 14:56:55,178 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2a6304a2c5046eb880f6b HTTP/1.1" 200 None 2026-03-26 14:56:55,178 - lava_callback - DEBUG - Processing node 69c2a6304a2c5046eb880f6b: baseline-arm-clabbe 2026-03-26 14:56:55,179 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2a6304a2c5046eb880f6b 2026-03-26 14:56:55,179 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 14:56:55,193 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2a6304a2c5046eb880f6b HTTP/1.1" 200 None 2026-03-26 14:56:55,194 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 14:56:55,221 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd026d05e4bbd6ba634c57 HTTP/1.1" 200 None 2026-03-26 14:56:55,222 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 14:56:55,251 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2a6304a2c5046eb880f6b HTTP/1.1" 200 None 2026-03-26 14:56:55,253 - lava_callback - ERROR - Error processing callback for node 69c2a6304a2c5046eb880f6b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:55426 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44750 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44752 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51866 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51882 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49320 - "GET / HTTP/1.0" 200 OK 2026-03-26 14:57:35,612 - lava_callback - INFO - Received callback for node 69c2a6534a2c5046eb880f7e INFO: 172.21.0.1:41578 - "POST /node/69c2a6534a2c5046eb880f7e HTTP/1.0" 202 Accepted 2026-03-26 14:57:35,617 - lava_callback - INFO - Starting async job submit for node 69c2a6534a2c5046eb880f7e 2026-03-26 14:57:35,618 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 14:57:35,652 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2a6534a2c5046eb880f7e HTTP/1.1" 200 None 2026-03-26 14:57:35,653 - lava_callback - DEBUG - Processing node 69c2a6534a2c5046eb880f7e: baseline-arm-baylibre 2026-03-26 14:57:35,653 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2a6534a2c5046eb880f7e 2026-03-26 14:57:35,654 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 14:57:35,667 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2a6534a2c5046eb880f7e HTTP/1.1" 200 None 2026-03-26 14:57:35,669 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 14:57:35,682 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd026d05e4bbd6ba634c57 HTTP/1.1" 200 None 2026-03-26 14:57:35,683 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 14:57:35,722 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2a6534a2c5046eb880f7e HTTP/1.1" 200 None 2026-03-26 14:57:35,729 - lava_callback - ERROR - Error processing callback for node 69c2a6534a2c5046eb880f7e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:41582 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51994 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53640 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44278 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44294 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44718 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-26 14:58:15,280 - lava_callback - INFO - Received callback for node 69c2a6794a2c5046eb880f8b INFO: 172.21.0.1:44720 - "POST /node/69c2a6794a2c5046eb880f8b HTTP/1.0" 202 Accepted 2026-03-26 14:58:15,285 - lava_callback - INFO - Starting async job submit for node 69c2a6794a2c5046eb880f8b 2026-03-26 14:58:15,286 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 14:58:15,354 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2a6794a2c5046eb880f8b HTTP/1.1" 200 None 2026-03-26 14:58:15,355 - lava_callback - DEBUG - Processing node 69c2a6794a2c5046eb880f8b: baseline-arm-clabbe 2026-03-26 14:58:15,355 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2a6794a2c5046eb880f8b 2026-03-26 14:58:15,355 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 14:58:15,368 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2a6794a2c5046eb880f8b HTTP/1.1" 200 None 2026-03-26 14:58:15,370 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 14:58:15,383 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd026d05e4bbd6ba634c57 HTTP/1.1" 200 None 2026-03-26 14:58:15,385 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 14:58:15,407 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2a6794a2c5046eb880f8b HTTP/1.1" 200 None 2026-03-26 14:58:15,412 - lava_callback - ERROR - Error processing callback for node 69c2a6794a2c5046eb880f8b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:44726 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33486 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44810 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51204 - "GET / HTTP/1.0" 200 OK 2026-03-26 14:58:55,249 - lava_callback - INFO - Received callback for node 69c2a69e4a2c5046eb880f94 INFO: 172.21.0.1:34996 - "POST /node/69c2a69e4a2c5046eb880f94 HTTP/1.0" 202 Accepted 2026-03-26 14:58:55,255 - lava_callback - INFO - Starting async job submit for node 69c2a69e4a2c5046eb880f94 2026-03-26 14:58:55,255 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 14:58:55,275 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2a69e4a2c5046eb880f94 HTTP/1.1" 200 None 2026-03-26 14:58:55,276 - lava_callback - DEBUG - Processing node 69c2a69e4a2c5046eb880f94: baseline-arm-clabbe 2026-03-26 14:58:55,276 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2a69e4a2c5046eb880f94 2026-03-26 14:58:55,277 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 14:58:55,290 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2a69e4a2c5046eb880f94 HTTP/1.1" 200 None 2026-03-26 14:58:55,292 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 14:58:55,304 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd026d05e4bbd6ba634c57 HTTP/1.1" 200 None 2026-03-26 14:58:55,306 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 14:58:55,349 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2a69e4a2c5046eb880f94 HTTP/1.1" 200 None 2026-03-26 14:58:55,355 - lava_callback - ERROR - Error processing callback for node 69c2a69e4a2c5046eb880f94: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:35010 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36082 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36088 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39198 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39204 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47740 - "GET / HTTP/1.0" 200 OK 2026-03-26 14:59:35,321 - lava_callback - INFO - Received callback for node 69c2a6c34a2c5046eb880fa0 INFO: 172.21.0.1:54478 - "POST /node/69c2a6c34a2c5046eb880fa0 HTTP/1.0" 202 Accepted 2026-03-26 14:59:35,327 - lava_callback - INFO - Starting async job submit for node 69c2a6c34a2c5046eb880fa0 2026-03-26 14:59:35,328 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 14:59:35,342 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2a6c34a2c5046eb880fa0 HTTP/1.1" 200 None 2026-03-26 14:59:35,342 - lava_callback - DEBUG - Processing node 69c2a6c34a2c5046eb880fa0: baseline-arm-clabbe 2026-03-26 14:59:35,343 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2a6c34a2c5046eb880fa0 2026-03-26 14:59:35,343 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 14:59:35,356 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2a6c34a2c5046eb880fa0 HTTP/1.1" 200 None 2026-03-26 14:59:35,357 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 14:59:35,371 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd026d05e4bbd6ba634c57 HTTP/1.1" 200 None 2026-03-26 14:59:35,372 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 14:59:35,401 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2a6c34a2c5046eb880fa0 HTTP/1.1" 200 None 2026-03-26 14:59:35,403 - lava_callback - ERROR - Error processing callback for node 69c2a6c34a2c5046eb880fa0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:54482 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53938 - "GET / HTTP/1.0" 200 OK 2026-03-26 14:59:55,317 - lava_callback - INFO - Received callback for node 69c2a6e94a2c5046eb880fac INFO: 172.21.0.1:34152 - "POST /node/69c2a6e94a2c5046eb880fac HTTP/1.0" 202 Accepted 2026-03-26 14:59:55,322 - lava_callback - INFO - Starting async job submit for node 69c2a6e94a2c5046eb880fac 2026-03-26 14:59:55,323 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 14:59:55,337 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2a6e94a2c5046eb880fac HTTP/1.1" 200 None 2026-03-26 14:59:55,338 - lava_callback - DEBUG - Processing node 69c2a6e94a2c5046eb880fac: baseline-arm-clabbe 2026-03-26 14:59:55,338 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2a6e94a2c5046eb880fac 2026-03-26 14:59:55,339 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 14:59:55,352 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2a6e94a2c5046eb880fac HTTP/1.1" 200 None 2026-03-26 14:59:55,353 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 14:59:55,366 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd026d05e4bbd6ba634c57 HTTP/1.1" 200 None 2026-03-26 14:59:55,368 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 14:59:55,393 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2a6e94a2c5046eb880fac HTTP/1.1" 200 None 2026-03-26 14:59:55,394 - lava_callback - ERROR - Error processing callback for node 69c2a6e94a2c5046eb880fac: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:34164 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43092 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43108 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45724 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45740 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33996 - "GET / HTTP/1.0" 200 OK 2026-03-26 15:00:35,391 - lava_callback - INFO - Received callback for node 69c2a70e4a2c5046eb880fba INFO: 172.21.0.1:36264 - "POST /node/69c2a70e4a2c5046eb880fba HTTP/1.0" 202 Accepted 2026-03-26 15:00:35,396 - lava_callback - INFO - Starting async job submit for node 69c2a70e4a2c5046eb880fba 2026-03-26 15:00:35,397 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 15:00:35,411 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2a70e4a2c5046eb880fba HTTP/1.1" 200 None 2026-03-26 15:00:35,411 - lava_callback - DEBUG - Processing node 69c2a70e4a2c5046eb880fba: baseline-arm-clabbe 2026-03-26 15:00:35,412 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2a70e4a2c5046eb880fba 2026-03-26 15:00:35,412 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 15:00:35,425 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2a70e4a2c5046eb880fba HTTP/1.1" 200 None 2026-03-26 15:00:35,427 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 15:00:35,440 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd026d05e4bbd6ba634c57 HTTP/1.1" 200 None 2026-03-26 15:00:35,442 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 15:00:35,462 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2a70e4a2c5046eb880fba HTTP/1.1" 200 None 2026-03-26 15:00:35,468 - lava_callback - ERROR - Error processing callback for node 69c2a70e4a2c5046eb880fba: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:36270 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42140 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34692 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35182 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35198 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44788 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44792 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36088 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34566 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45912 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32834 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39460 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39472 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35850 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35864 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45494 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38334 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46586 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47062 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46148 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46164 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33158 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33168 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35470 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36040 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50188 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43282 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40012 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40016 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54664 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54678 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46360 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34952 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46520 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35592 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60690 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60704 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35356 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35364 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41724 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53926 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34570 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34046 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32814 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32830 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47534 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47548 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50614 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59346 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45688 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33586 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33956 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33962 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34936 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34938 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40166 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58534 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50842 - "GET / HTTP/1.0" 200 OK 2026-03-26 15:07:56,083 - lava_callback - INFO - Received callback for node 69c2a8c14a2c5046eb881009 INFO: 172.21.0.1:38846 - "POST /node/69c2a8c14a2c5046eb881009 HTTP/1.0" 202 Accepted 2026-03-26 15:07:56,091 - lava_callback - INFO - Starting async job submit for node 69c2a8c14a2c5046eb881009 2026-03-26 15:07:56,092 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 15:07:56,108 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2a8c14a2c5046eb881009 HTTP/1.1" 200 None 2026-03-26 15:07:56,109 - lava_callback - DEBUG - Processing node 69c2a8c14a2c5046eb881009: baseline-arm64-baylibre 2026-03-26 15:07:56,109 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2a8c14a2c5046eb881009 2026-03-26 15:07:56,110 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 15:07:56,127 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2a8c14a2c5046eb881009 HTTP/1.1" 200 None 2026-03-26 15:07:56,129 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 15:07:56,142 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248e54a2c5046eb87f89f HTTP/1.1" 200 None 2026-03-26 15:07:56,144 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 15:07:56,164 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2a8c14a2c5046eb881009 HTTP/1.1" 200 None 2026-03-26 15:07:56,169 - lava_callback - ERROR - Error processing callback for node 69c2a8c14a2c5046eb881009: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:38858 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57170 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57182 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58434 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58446 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53616 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41604 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47248 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58380 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34880 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34886 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59886 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-26 15:09:16,171 - lava_callback - INFO - Received callback for node 69c2a9124a2c5046eb88101a INFO: 172.21.0.1:59890 - "POST /node/69c2a9124a2c5046eb88101a HTTP/1.0" 202 Accepted 2026-03-26 15:09:16,177 - lava_callback - INFO - Starting async job submit for node 69c2a9124a2c5046eb88101a 2026-03-26 15:09:16,178 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 15:09:16,203 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2a9124a2c5046eb88101a HTTP/1.1" 200 None 2026-03-26 15:09:16,204 - lava_callback - DEBUG - Processing node 69c2a9124a2c5046eb88101a: baseline-arm64-baylibre 2026-03-26 15:09:16,204 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2a9124a2c5046eb88101a 2026-03-26 15:09:16,205 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 15:09:16,218 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2a9124a2c5046eb88101a HTTP/1.1" 200 None 2026-03-26 15:09:16,219 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 15:09:16,233 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248e54a2c5046eb87f89f HTTP/1.1" 200 None 2026-03-26 15:09:16,234 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 15:09:16,267 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2a9124a2c5046eb88101a HTTP/1.1" 200 None 2026-03-26 15:09:16,269 - lava_callback - ERROR - Error processing callback for node 69c2a9124a2c5046eb88101a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:59900 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38570 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57420 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42814 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58954 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52786 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52796 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51210 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51214 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52322 - "GET / HTTP/1.0" 200 OK 2026-03-26 15:10:36,202 - lava_callback - INFO - Received callback for node 69c2a9644a2c5046eb881025 INFO: 172.21.0.1:54836 - "POST /node/69c2a9644a2c5046eb881025 HTTP/1.0" 202 Accepted 2026-03-26 15:10:36,208 - lava_callback - INFO - Starting async job submit for node 69c2a9644a2c5046eb881025 2026-03-26 15:10:36,209 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 15:10:36,237 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2a9644a2c5046eb881025 HTTP/1.1" 200 None 2026-03-26 15:10:36,238 - lava_callback - DEBUG - Processing node 69c2a9644a2c5046eb881025: baseline-arm64-baylibre 2026-03-26 15:10:36,239 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2a9644a2c5046eb881025 2026-03-26 15:10:36,239 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 15:10:36,252 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2a9644a2c5046eb881025 HTTP/1.1" 200 None 2026-03-26 15:10:36,254 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 15:10:36,291 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248e54a2c5046eb87f89f HTTP/1.1" 200 None 2026-03-26 15:10:36,292 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 15:10:36,313 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2a9644a2c5046eb881025 HTTP/1.1" 200 None 2026-03-26 15:10:36,314 - lava_callback - ERROR - Error processing callback for node 69c2a9644a2c5046eb881025: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:54842 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44868 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41592 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41780 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41794 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47114 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47124 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53386 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57810 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54352 - "GET / HTTP/1.0" 200 OK 2026-03-26 15:11:56,348 - lava_callback - INFO - Received callback for node 69c2a9b54a2c5046eb881035 INFO: 172.21.0.1:53876 - "POST /node/69c2a9b54a2c5046eb881035 HTTP/1.0" 202 Accepted 2026-03-26 15:11:56,355 - lava_callback - INFO - Starting async job submit for node 69c2a9b54a2c5046eb881035 2026-03-26 15:11:56,356 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 15:11:56,370 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2a9b54a2c5046eb881035 HTTP/1.1" 200 None 2026-03-26 15:11:56,371 - lava_callback - DEBUG - Processing node 69c2a9b54a2c5046eb881035: baseline-arm64-baylibre 2026-03-26 15:11:56,372 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2a9b54a2c5046eb881035 2026-03-26 15:11:56,372 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 15:11:56,386 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2a9b54a2c5046eb881035 HTTP/1.1" 200 None 2026-03-26 15:11:56,388 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 15:11:56,402 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248e54a2c5046eb87f89f HTTP/1.1" 200 None 2026-03-26 15:11:56,404 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 15:11:56,430 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2a9b54a2c5046eb881035 HTTP/1.1" 200 None 2026-03-26 15:11:56,432 - lava_callback - ERROR - Error processing callback for node 69c2a9b54a2c5046eb881035: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:53888 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50444 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50456 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38872 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38874 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43380 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48172 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54390 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51356 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60538 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60544 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39038 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39048 - "GET / HTTP/1.0" 200 OK 2026-03-26 15:13:16,728 - lava_callback - INFO - Received callback for node 69c2aa084a2c5046eb881045 INFO: 172.21.0.1:39064 - "POST /node/69c2aa084a2c5046eb881045 HTTP/1.0" 202 Accepted 2026-03-26 15:13:16,734 - lava_callback - INFO - Starting async job submit for node 69c2aa084a2c5046eb881045 2026-03-26 15:13:16,735 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 15:13:16,749 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2aa084a2c5046eb881045 HTTP/1.1" 200 None 2026-03-26 15:13:16,750 - lava_callback - DEBUG - Processing node 69c2aa084a2c5046eb881045: baseline-arm64-baylibre 2026-03-26 15:13:16,750 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2aa084a2c5046eb881045 2026-03-26 15:13:16,751 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 15:13:16,764 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2aa084a2c5046eb881045 HTTP/1.1" 200 None 2026-03-26 15:13:16,765 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 15:13:16,791 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248e54a2c5046eb87f89f HTTP/1.1" 200 None 2026-03-26 15:13:16,793 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 15:13:16,821 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2aa084a2c5046eb881045 HTTP/1.1" 200 None 2026-03-26 15:13:16,823 - lava_callback - ERROR - Error processing callback for node 69c2aa084a2c5046eb881045: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:54380 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52928 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43708 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46372 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49750 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49760 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41638 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41652 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49628 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41684 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37864 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51412 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55796 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55812 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55730 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55744 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59344 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42478 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49670 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48526 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47208 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47218 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56964 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-26 15:16:16,035 - lava_callback - INFO - Received callback for node 69c2aaae4a2c5046eb881063 INFO: 172.21.0.1:56972 - "POST /node/69c2aaae4a2c5046eb881063 HTTP/1.0" 202 Accepted 2026-03-26 15:16:16,041 - lava_callback - INFO - Starting async job submit for node 69c2aaae4a2c5046eb881063 2026-03-26 15:16:16,042 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 15:16:16,057 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2aaae4a2c5046eb881063 HTTP/1.1" 200 None 2026-03-26 15:16:16,057 - lava_callback - DEBUG - Processing node 69c2aaae4a2c5046eb881063: baseline-arm64-clabbe 2026-03-26 15:16:16,058 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2aaae4a2c5046eb881063 2026-03-26 15:16:16,058 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 15:16:16,072 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2aaae4a2c5046eb881063 HTTP/1.1" 200 None 2026-03-26 15:16:16,073 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 15:16:16,087 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248e54a2c5046eb87f89f HTTP/1.1" 200 None 2026-03-26 15:16:16,089 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 15:16:16,110 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2aaae4a2c5046eb881063 HTTP/1.1" 200 None 2026-03-26 15:16:16,116 - lava_callback - ERROR - Error processing callback for node 69c2aaae4a2c5046eb881063: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:56984 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52788 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40808 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34052 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58618 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35718 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35724 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40004 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40008 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45552 - "GET / HTTP/1.0" 200 OK 2026-03-26 15:17:36,060 - lava_callback - INFO - Received callback for node 69c2ab014a2c5046eb881070 INFO: 172.21.0.1:33826 - "POST /node/69c2ab014a2c5046eb881070 HTTP/1.0" 202 Accepted 2026-03-26 15:17:36,066 - lava_callback - INFO - Starting async job submit for node 69c2ab014a2c5046eb881070 2026-03-26 15:17:36,067 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 15:17:36,083 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2ab014a2c5046eb881070 HTTP/1.1" 200 None 2026-03-26 15:17:36,084 - lava_callback - DEBUG - Processing node 69c2ab014a2c5046eb881070: baseline-arm64-clabbe 2026-03-26 15:17:36,084 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2ab014a2c5046eb881070 2026-03-26 15:17:36,085 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 15:17:36,098 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2ab014a2c5046eb881070 HTTP/1.1" 200 None 2026-03-26 15:17:36,100 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 15:17:36,113 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248e54a2c5046eb87f89f HTTP/1.1" 200 None 2026-03-26 15:17:36,114 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 15:17:36,137 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2ab014a2c5046eb881070 HTTP/1.1" 200 None 2026-03-26 15:17:36,138 - lava_callback - ERROR - Error processing callback for node 69c2ab014a2c5046eb881070: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:33832 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58546 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42972 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57474 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57482 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54058 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54070 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40920 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39800 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38760 - "GET / HTTP/1.0" 200 OK 2026-03-26 15:18:56,126 - lava_callback - INFO - Received callback for node 69c2ab514a2c5046eb881080 INFO: 172.21.0.1:57620 - "POST /node/69c2ab514a2c5046eb881080 HTTP/1.0" 202 Accepted 2026-03-26 15:18:56,132 - lava_callback - INFO - Starting async job submit for node 69c2ab514a2c5046eb881080 2026-03-26 15:18:56,133 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 15:18:56,147 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2ab514a2c5046eb881080 HTTP/1.1" 200 None 2026-03-26 15:18:56,148 - lava_callback - DEBUG - Processing node 69c2ab514a2c5046eb881080: baseline-arm64-clabbe 2026-03-26 15:18:56,148 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2ab514a2c5046eb881080 2026-03-26 15:18:56,148 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 15:18:56,161 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2ab514a2c5046eb881080 HTTP/1.1" 200 None 2026-03-26 15:18:56,162 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 15:18:56,176 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248e54a2c5046eb87f89f HTTP/1.1" 200 None 2026-03-26 15:18:56,177 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 15:18:56,197 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2ab514a2c5046eb881080 HTTP/1.1" 200 None 2026-03-26 15:18:56,202 - lava_callback - ERROR - Error processing callback for node 69c2ab514a2c5046eb881080: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:57622 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35416 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35418 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39052 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39068 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55948 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44090 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36980 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34746 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42262 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42266 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37058 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-26 15:20:16,446 - lava_callback - INFO - Received callback for node 69c2ab9f4a2c5046eb881094 INFO: 172.21.0.1:37074 - "POST /node/69c2ab9f4a2c5046eb881094 HTTP/1.0" 202 Accepted 2026-03-26 15:20:16,452 - lava_callback - INFO - Starting async job submit for node 69c2ab9f4a2c5046eb881094 2026-03-26 15:20:16,453 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 15:20:16,472 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2ab9f4a2c5046eb881094 HTTP/1.1" 200 None 2026-03-26 15:20:16,474 - lava_callback - DEBUG - Processing node 69c2ab9f4a2c5046eb881094: baseline-arm64-clabbe 2026-03-26 15:20:16,474 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2ab9f4a2c5046eb881094 2026-03-26 15:20:16,475 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 15:20:16,488 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2ab9f4a2c5046eb881094 HTTP/1.1" 200 None 2026-03-26 15:20:16,490 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 15:20:16,502 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248e54a2c5046eb87f89f HTTP/1.1" 200 None 2026-03-26 15:20:16,504 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 15:20:16,529 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2ab9f4a2c5046eb881094 HTTP/1.1" 200 None 2026-03-26 15:20:16,530 - lava_callback - ERROR - Error processing callback for node 69c2ab9f4a2c5046eb881094: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:37078 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42272 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60344 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43674 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58354 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54910 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54926 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48224 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48236 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43574 - "GET / HTTP/1.0" 200 OK 2026-03-26 15:21:36,238 - lava_callback - INFO - Received callback for node 69c2abf04a2c5046eb88109c INFO: 172.21.0.1:60916 - "POST /node/69c2abf04a2c5046eb88109c HTTP/1.0" 202 Accepted 2026-03-26 15:21:36,244 - lava_callback - INFO - Starting async job submit for node 69c2abf04a2c5046eb88109c 2026-03-26 15:21:36,245 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 15:21:36,259 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2abf04a2c5046eb88109c HTTP/1.1" 200 None 2026-03-26 15:21:36,260 - lava_callback - DEBUG - Processing node 69c2abf04a2c5046eb88109c: baseline-arm64-clabbe 2026-03-26 15:21:36,260 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2abf04a2c5046eb88109c 2026-03-26 15:21:36,261 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 15:21:36,274 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2abf04a2c5046eb88109c HTTP/1.1" 200 None 2026-03-26 15:21:36,276 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 15:21:36,289 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248e54a2c5046eb87f89f HTTP/1.1" 200 None 2026-03-26 15:21:36,291 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 15:21:36,311 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2abf04a2c5046eb88109c HTTP/1.1" 200 None 2026-03-26 15:21:36,312 - lava_callback - ERROR - Error processing callback for node 69c2abf04a2c5046eb88109c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:60922 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32928 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43650 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58244 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58260 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56920 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56928 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47056 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60708 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44452 - "GET / HTTP/1.0" 200 OK 2026-03-26 15:22:56,270 - lava_callback - INFO - Received callback for node 69c2ac3f4a2c5046eb8810b1 INFO: 172.21.0.1:56316 - "POST /node/69c2ac3f4a2c5046eb8810b1 HTTP/1.0" 202 Accepted 2026-03-26 15:22:56,275 - lava_callback - INFO - Starting async job submit for node 69c2ac3f4a2c5046eb8810b1 2026-03-26 15:22:56,276 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 15:22:56,291 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2ac3f4a2c5046eb8810b1 HTTP/1.1" 200 None 2026-03-26 15:22:56,292 - lava_callback - DEBUG - Processing node 69c2ac3f4a2c5046eb8810b1: baseline-arm64-clabbe 2026-03-26 15:22:56,292 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2ac3f4a2c5046eb8810b1 2026-03-26 15:22:56,293 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 15:22:56,305 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2ac3f4a2c5046eb8810b1 HTTP/1.1" 200 None 2026-03-26 15:22:56,307 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 15:22:56,322 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248e54a2c5046eb87f89f HTTP/1.1" 200 None 2026-03-26 15:22:56,323 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 15:22:56,344 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2ac3f4a2c5046eb8810b1 HTTP/1.1" 200 None 2026-03-26 15:22:56,349 - lava_callback - ERROR - Error processing callback for node 69c2ac3f4a2c5046eb8810b1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:56326 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36858 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36874 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48198 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48202 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47194 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35542 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48676 - "GET / HTTP/1.0" 200 OK 2026-03-26 15:23:56,318 - lava_callback - INFO - Received callback for node 69c2ac804a2c5046eb8810bd INFO: 172.21.0.1:44082 - "POST /node/69c2ac804a2c5046eb8810bd HTTP/1.0" 202 Accepted 2026-03-26 15:23:56,324 - lava_callback - INFO - Starting async job submit for node 69c2ac804a2c5046eb8810bd 2026-03-26 15:23:56,325 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 15:23:56,339 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2ac804a2c5046eb8810bd HTTP/1.1" 200 None 2026-03-26 15:23:56,340 - lava_callback - DEBUG - Processing node 69c2ac804a2c5046eb8810bd: baseline-arm-clabbe 2026-03-26 15:23:56,340 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2ac804a2c5046eb8810bd 2026-03-26 15:23:56,341 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 15:23:56,354 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2ac804a2c5046eb8810bd HTTP/1.1" 200 None 2026-03-26 15:23:56,355 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 15:23:56,377 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd026d05e4bbd6ba634c57 HTTP/1.1" 200 None 2026-03-26 15:23:56,378 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 15:23:56,399 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2ac804a2c5046eb8810bd HTTP/1.1" 200 None 2026-03-26 15:23:56,406 - lava_callback - ERROR - Error processing callback for node 69c2ac804a2c5046eb8810bd: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:44086 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59162 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59166 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47404 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47412 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52728 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38908 - "GET / HTTP/1.0" 200 OK 2026-03-26 15:24:37,085 - lava_callback - INFO - Received callback for node 69c2acad4a2c5046eb8810cc INFO: 172.21.0.1:38922 - "POST /node/69c2acad4a2c5046eb8810cc HTTP/1.0" 202 Accepted 2026-03-26 15:24:37,091 - lava_callback - INFO - Starting async job submit for node 69c2acad4a2c5046eb8810cc 2026-03-26 15:24:37,091 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 15:24:37,105 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2acad4a2c5046eb8810cc HTTP/1.1" 200 None 2026-03-26 15:24:37,106 - lava_callback - DEBUG - Processing node 69c2acad4a2c5046eb8810cc: baseline-arm64-baylibre 2026-03-26 15:24:37,106 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2acad4a2c5046eb8810cc 2026-03-26 15:24:37,107 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 15:24:37,119 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2acad4a2c5046eb8810cc HTTP/1.1" 200 None 2026-03-26 15:24:37,120 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 15:24:37,136 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5827544ab5cd4a7d9dd HTTP/1.1" 200 None 2026-03-26 15:24:37,137 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 15:24:37,163 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2acad4a2c5046eb8810cc HTTP/1.1" 200 None 2026-03-26 15:24:37,165 - lava_callback - ERROR - Error processing callback for node 69c2acad4a2c5046eb8810cc: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:38532 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59670 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51374 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51384 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50022 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50036 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36236 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58540 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36430 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46732 - "GET / HTTP/1.0" 200 OK 2026-03-26 15:25:57,113 - lava_callback - INFO - Received callback for node 69c2acfc4a2c5046eb8810d8 INFO: 172.21.0.1:46742 - "POST /node/69c2acfc4a2c5046eb8810d8 HTTP/1.0" 202 Accepted 2026-03-26 15:25:57,119 - lava_callback - INFO - Starting async job submit for node 69c2acfc4a2c5046eb8810d8 2026-03-26 15:25:57,120 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 15:25:57,135 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2acfc4a2c5046eb8810d8 HTTP/1.1" 200 None 2026-03-26 15:25:57,135 - lava_callback - DEBUG - Processing node 69c2acfc4a2c5046eb8810d8: baseline-arm64-baylibre 2026-03-26 15:25:57,136 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2acfc4a2c5046eb8810d8 2026-03-26 15:25:57,136 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 15:25:57,149 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2acfc4a2c5046eb8810d8 HTTP/1.1" 200 None 2026-03-26 15:25:57,150 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 15:25:57,165 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5827544ab5cd4a7d9dd HTTP/1.1" 200 None 2026-03-26 15:25:57,167 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 15:25:57,195 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2acfc4a2c5046eb8810d8 HTTP/1.1" 200 None 2026-03-26 15:25:57,201 - lava_callback - ERROR - Error processing callback for node 69c2acfc4a2c5046eb8810d8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:52106 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52114 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41272 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41278 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41522 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32826 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46522 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59212 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46074 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46084 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47376 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47380 - "GET / HTTP/1.0" 200 OK 2026-03-26 15:27:16,688 - lava_callback - INFO - Received callback for node 69c2ad504a2c5046eb8810f0 INFO: 172.21.0.1:47384 - "POST /node/69c2ad504a2c5046eb8810f0 HTTP/1.0" 202 Accepted 2026-03-26 15:27:16,693 - lava_callback - INFO - Starting async job submit for node 69c2ad504a2c5046eb8810f0 2026-03-26 15:27:16,694 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 15:27:16,712 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2ad504a2c5046eb8810f0 HTTP/1.1" 200 None 2026-03-26 15:27:16,713 - lava_callback - DEBUG - Processing node 69c2ad504a2c5046eb8810f0: baseline-arm64-clabbe 2026-03-26 15:27:16,713 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2ad504a2c5046eb8810f0 2026-03-26 15:27:16,714 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 15:27:16,727 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2ad504a2c5046eb8810f0 HTTP/1.1" 200 None 2026-03-26 15:27:16,729 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 15:27:16,742 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5827544ab5cd4a7d9dd HTTP/1.1" 200 None 2026-03-26 15:27:16,743 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 15:27:16,763 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2ad504a2c5046eb8810f0 HTTP/1.1" 200 None 2026-03-26 15:27:16,769 - lava_callback - ERROR - Error processing callback for node 69c2ad504a2c5046eb8810f0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:55860 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45114 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39904 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47214 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51972 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51976 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50598 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50614 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36520 - "GET / HTTP/1.0" 200 OK 2026-03-26 15:28:36,524 - lava_callback - INFO - Received callback for node 69c2ad9f4a2c5046eb8810fe INFO: 172.21.0.1:35426 - "POST /node/69c2ad9f4a2c5046eb8810fe HTTP/1.0" 202 Accepted 2026-03-26 15:28:36,530 - lava_callback - INFO - Starting async job submit for node 69c2ad9f4a2c5046eb8810fe 2026-03-26 15:28:36,531 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.21.0.1:35438 - "GET / HTTP/1.0" 200 OK 2026-03-26 15:28:36,544 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2ad9f4a2c5046eb8810fe HTTP/1.1" 200 None 2026-03-26 15:28:36,545 - lava_callback - DEBUG - Processing node 69c2ad9f4a2c5046eb8810fe: baseline-arm64-clabbe 2026-03-26 15:28:36,545 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2ad9f4a2c5046eb8810fe 2026-03-26 15:28:36,546 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 15:28:36,558 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2ad9f4a2c5046eb8810fe HTTP/1.1" 200 None 2026-03-26 15:28:36,560 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 15:28:36,573 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5827544ab5cd4a7d9dd HTTP/1.1" 200 None 2026-03-26 15:28:36,574 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 15:28:36,594 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2ad9f4a2c5046eb8810fe HTTP/1.1" 200 None 2026-03-26 15:28:36,595 - lava_callback - ERROR - Error processing callback for node 69c2ad9f4a2c5046eb8810fe: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:41172 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43998 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41266 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41278 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57432 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57434 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51652 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40646 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58200 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38398 - "GET / HTTP/1.0" 200 OK 2026-03-26 15:29:56,602 - lava_callback - INFO - Received callback for node 69c2adef4a2c5046eb881108 INFO: 172.21.0.1:38404 - "POST /node/69c2adef4a2c5046eb881108 HTTP/1.0" 202 Accepted 2026-03-26 15:29:56,609 - lava_callback - INFO - Starting async job submit for node 69c2adef4a2c5046eb881108 2026-03-26 15:29:56,609 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 15:29:56,623 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2adef4a2c5046eb881108 HTTP/1.1" 200 None 2026-03-26 15:29:56,624 - lava_callback - DEBUG - Processing node 69c2adef4a2c5046eb881108: baseline-arm64-clabbe 2026-03-26 15:29:56,624 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2adef4a2c5046eb881108 2026-03-26 15:29:56,624 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 15:29:56,660 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2adef4a2c5046eb881108 HTTP/1.1" 200 None 2026-03-26 15:29:56,662 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 15:29:56,675 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5827544ab5cd4a7d9dd HTTP/1.1" 200 None 2026-03-26 15:29:56,677 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 15:29:56,698 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2adef4a2c5046eb881108 HTTP/1.1" 200 None 2026-03-26 15:29:56,709 - lava_callback - ERROR - Error processing callback for node 69c2adef4a2c5046eb881108: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:50006 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50022 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39924 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39936 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40008 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33652 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56104 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49618 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56476 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56478 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44356 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44368 - "GET / HTTP/1.0" 200 OK 2026-03-26 15:31:16,687 - lava_callback - INFO - Received callback for node 69c2ae3d4a2c5046eb881117 INFO: 172.21.0.1:44382 - "POST /node/69c2ae3d4a2c5046eb881117 HTTP/1.0" 202 Accepted 2026-03-26 15:31:16,693 - lava_callback - INFO - Starting async job submit for node 69c2ae3d4a2c5046eb881117 2026-03-26 15:31:16,693 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 15:31:16,707 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2ae3d4a2c5046eb881117 HTTP/1.1" 200 None 2026-03-26 15:31:16,708 - lava_callback - DEBUG - Processing node 69c2ae3d4a2c5046eb881117: baseline-arm64-clabbe 2026-03-26 15:31:16,708 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2ae3d4a2c5046eb881117 2026-03-26 15:31:16,709 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 15:31:16,723 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2ae3d4a2c5046eb881117 HTTP/1.1" 200 None 2026-03-26 15:31:16,725 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 15:31:16,738 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5827544ab5cd4a7d9dd HTTP/1.1" 200 None 2026-03-26 15:31:16,740 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 15:31:16,760 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2ae3d4a2c5046eb881117 HTTP/1.1" 200 None 2026-03-26 15:31:16,769 - lava_callback - ERROR - Error processing callback for node 69c2ae3d4a2c5046eb881117: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:50644 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33452 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49970 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39454 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57150 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57152 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43772 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43774 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40210 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52606 - "GET / HTTP/1.0" 200 OK 2026-03-26 15:32:36,728 - lava_callback - INFO - Received callback for node 69c2ae8f4a2c5046eb881126 INFO: 172.21.0.1:52616 - "POST /node/69c2ae8f4a2c5046eb881126 HTTP/1.0" 202 Accepted 2026-03-26 15:32:36,734 - lava_callback - INFO - Starting async job submit for node 69c2ae8f4a2c5046eb881126 2026-03-26 15:32:36,735 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 15:32:36,762 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2ae8f4a2c5046eb881126 HTTP/1.1" 200 None 2026-03-26 15:32:36,763 - lava_callback - DEBUG - Processing node 69c2ae8f4a2c5046eb881126: baseline-arm64-clabbe 2026-03-26 15:32:36,763 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2ae8f4a2c5046eb881126 2026-03-26 15:32:36,764 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 15:32:36,785 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2ae8f4a2c5046eb881126 HTTP/1.1" 200 None 2026-03-26 15:32:36,786 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 15:32:36,799 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5827544ab5cd4a7d9dd HTTP/1.1" 200 None 2026-03-26 15:32:36,801 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 15:32:36,821 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2ae8f4a2c5046eb881126 HTTP/1.1" 200 None 2026-03-26 15:32:36,834 - lava_callback - ERROR - Error processing callback for node 69c2ae8f4a2c5046eb881126: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:49914 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43782 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55232 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55236 - "GET / HTTP/1.0" 200 OK 2026-03-26 15:33:08,424 - lava_callback - INFO - Received callback for node 69c4f0d6f63ce05331e2aac2 INFO: 172.21.0.1:55250 - "POST /node/69c4f0d6f63ce05331e2aac2 HTTP/1.0" 202 Accepted 2026-03-26 15:33:08,430 - lava_callback - INFO - Starting async job submit for node 69c4f0d6f63ce05331e2aac2 2026-03-26 15:33:08,447 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 15:33:08,461 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f0d6f63ce05331e2aac2 HTTP/1.1" 200 None 2026-03-26 15:33:08,462 - lava_callback - DEBUG - Processing node 69c4f0d6f63ce05331e2aac2: baseline-x86 2026-03-26 15:33:08,745 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 15:33:08,927 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 15:33:08,929 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69c4f0d6f63ce05331e2aac2/log.txt.gz 2026-03-26 15:33:09,095 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 15:33:09,323 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 15:33:09,326 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69c4f0d6f63ce05331e2aac2/lava-logs.yaml 2026-03-26 15:33:09,333 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 15:33:09,635 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 15:33:09,637 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69c4f0d6f63ce05331e2aac2/lava_callback.json.gz 2026-03-26 15:33:09,656 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 15:33:09,672 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f0d6f63ce05331e2aac2 HTTP/1.1" 200 None 2026-03-26 15:33:09,674 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 15:33:09,695 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebecf63ce05331e2a4ae HTTP/1.1" 200 None 2026-03-26 15:33:09,696 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 15:33:09,722 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4f0d6f63ce05331e2aac2 HTTP/1.1" 200 None 2026-03-26 15:33:09,731 - lava_callback - INFO - Completed processing callback for node 69c4f0d6f63ce05331e2aac2 INFO: 172.21.0.1:43176 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43180 - "GET / HTTP/1.0" 200 OK 2026-03-26 15:33:17,035 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 15:33:17,052 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:42854 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44650 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36110 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54816 - "GET / HTTP/1.0" 200 OK 2026-03-26 15:33:56,757 - lava_callback - INFO - Received callback for node 69c2aee04a2c5046eb88112d INFO: 172.21.0.1:54818 - "POST /node/69c2aee04a2c5046eb88112d HTTP/1.0" 202 Accepted 2026-03-26 15:33:56,763 - lava_callback - INFO - Starting async job submit for node 69c2aee04a2c5046eb88112d 2026-03-26 15:33:56,764 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 15:33:56,781 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2aee04a2c5046eb88112d HTTP/1.1" 200 None 2026-03-26 15:33:56,782 - lava_callback - DEBUG - Processing node 69c2aee04a2c5046eb88112d: baseline-arm64-clabbe 2026-03-26 15:33:56,782 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2aee04a2c5046eb88112d 2026-03-26 15:33:56,783 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 15:33:56,810 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2aee04a2c5046eb88112d HTTP/1.1" 200 None 2026-03-26 15:33:56,811 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 15:33:56,826 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5827544ab5cd4a7d9dd HTTP/1.1" 200 None 2026-03-26 15:33:56,827 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 15:33:56,847 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2aee04a2c5046eb88112d HTTP/1.1" 200 None 2026-03-26 15:33:56,849 - lava_callback - ERROR - Error processing callback for node 69c2aee04a2c5046eb88112d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:54844 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54860 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60654 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60662 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52080 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60904 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54696 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53832 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50018 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50022 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59380 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59392 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51076 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41864 - "GET / HTTP/1.0" 200 OK 2026-03-26 15:35:36,859 - lava_callback - INFO - Received callback for node 69c2af324a2c5046eb88113c INFO: 172.21.0.1:41878 - "POST /node/69c2af324a2c5046eb88113c HTTP/1.0" 202 Accepted 2026-03-26 15:35:36,864 - lava_callback - INFO - Starting async job submit for node 69c2af324a2c5046eb88113c 2026-03-26 15:35:36,865 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 15:35:36,886 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2af324a2c5046eb88113c HTTP/1.1" 200 None 2026-03-26 15:35:36,887 - lava_callback - DEBUG - Processing node 69c2af324a2c5046eb88113c: baseline-arm64-clabbe 2026-03-26 15:35:36,887 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2af324a2c5046eb88113c 2026-03-26 15:35:36,888 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 15:35:36,901 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2af324a2c5046eb88113c HTTP/1.1" 200 None 2026-03-26 15:35:36,902 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 15:35:36,916 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5827544ab5cd4a7d9dd HTTP/1.1" 200 None 2026-03-26 15:35:36,917 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 15:35:36,938 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2af324a2c5046eb88113c HTTP/1.1" 200 None 2026-03-26 15:35:36,945 - lava_callback - ERROR - Error processing callback for node 69c2af324a2c5046eb88113c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:42030 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40428 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53066 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53080 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52104 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52116 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54262 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38730 - "GET / HTTP/1.0" 200 OK 2026-03-26 15:36:41,360 - lava_callback - INFO - Received callback for node 69c5458ff63ce05331e2bb9f INFO: 172.21.0.1:60958 - "POST /node/69c5458ff63ce05331e2bb9f HTTP/1.0" 202 Accepted 2026-03-26 15:36:41,365 - lava_callback - INFO - Starting async job submit for node 69c5458ff63ce05331e2bb9f 2026-03-26 15:36:41,382 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 15:36:41,397 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5458ff63ce05331e2bb9f HTTP/1.1" 200 None 2026-03-26 15:36:41,398 - lava_callback - DEBUG - Processing node 69c5458ff63ce05331e2bb9f: baseline-x86 2026-03-26 15:36:41,698 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 15:36:42,021 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 15:36:42,023 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69c5458ff63ce05331e2bb9f/log.txt.gz 2026-03-26 15:36:42,187 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 15:36:42,403 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 15:36:42,405 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69c5458ff63ce05331e2bb9f/lava-logs.yaml 2026-03-26 15:36:42,413 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 15:36:42,578 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 15:36:42,579 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69c5458ff63ce05331e2bb9f/lava_callback.json.gz 2026-03-26 15:36:42,597 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 15:36:42,611 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5458ff63ce05331e2bb9f HTTP/1.1" 200 None 2026-03-26 15:36:42,612 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 15:36:42,625 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebecf63ce05331e2a4ae HTTP/1.1" 200 None 2026-03-26 15:36:42,627 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 15:36:42,656 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5458ff63ce05331e2bb9f HTTP/1.1" 200 None 2026-03-26 15:36:42,657 - lava_callback - INFO - Completed processing callback for node 69c5458ff63ce05331e2bb9f INFO: 172.21.0.1:60972 - "GET / HTTP/1.0" 200 OK 2026-03-26 15:36:47,055 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 15:36:47,071 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:58540 - "GET / HTTP/1.0" 200 OK 2026-03-26 15:36:56,883 - lava_callback - INFO - Received callback for node 69c2af844a2c5046eb88114f INFO: 172.21.0.1:58542 - "POST /node/69c2af844a2c5046eb88114f HTTP/1.0" 202 Accepted 2026-03-26 15:36:56,889 - lava_callback - INFO - Starting async job submit for node 69c2af844a2c5046eb88114f 2026-03-26 15:36:56,890 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 15:36:56,903 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2af844a2c5046eb88114f HTTP/1.1" 200 None 2026-03-26 15:36:56,904 - lava_callback - DEBUG - Processing node 69c2af844a2c5046eb88114f: baseline-arm64-clabbe 2026-03-26 15:36:56,904 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2af844a2c5046eb88114f 2026-03-26 15:36:56,905 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 15:36:56,918 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2af844a2c5046eb88114f HTTP/1.1" 200 None 2026-03-26 15:36:56,919 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 15:36:56,932 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5827544ab5cd4a7d9dd HTTP/1.1" 200 None 2026-03-26 15:36:56,934 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 15:36:56,956 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2af844a2c5046eb88114f HTTP/1.1" 200 None 2026-03-26 15:36:56,968 - lava_callback - ERROR - Error processing callback for node 69c2af844a2c5046eb88114f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:40950 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40962 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34344 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34346 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47672 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44128 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33186 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45408 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38278 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38284 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39898 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39910 - "GET / HTTP/1.0" 200 OK 2026-03-26 15:38:16,935 - lava_callback - INFO - Received callback for node 69c2afd54a2c5046eb881156 INFO: 172.21.0.1:39912 - "POST /node/69c2afd54a2c5046eb881156 HTTP/1.0" 202 Accepted 2026-03-26 15:38:16,940 - lava_callback - INFO - Starting async job submit for node 69c2afd54a2c5046eb881156 2026-03-26 15:38:16,941 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 15:38:16,956 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2afd54a2c5046eb881156 HTTP/1.1" 200 None 2026-03-26 15:38:16,957 - lava_callback - DEBUG - Processing node 69c2afd54a2c5046eb881156: baseline-arm64-clabbe 2026-03-26 15:38:16,957 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2afd54a2c5046eb881156 2026-03-26 15:38:16,957 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 15:38:16,970 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2afd54a2c5046eb881156 HTTP/1.1" 200 None 2026-03-26 15:38:16,972 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 15:38:16,985 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5827544ab5cd4a7d9dd HTTP/1.1" 200 None 2026-03-26 15:38:16,986 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 15:38:17,008 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2afd54a2c5046eb881156 HTTP/1.1" 200 None 2026-03-26 15:38:17,017 - lava_callback - ERROR - Error processing callback for node 69c2afd54a2c5046eb881156: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:60752 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37270 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60732 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58530 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56724 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56738 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51122 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51126 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58600 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37684 - "GET / HTTP/1.0" 200 OK 2026-03-26 15:39:36,990 - lava_callback - INFO - Received callback for node 69c2b02c4a2c5046eb881162 INFO: 172.21.0.1:37694 - "POST /node/69c2b02c4a2c5046eb881162 HTTP/1.0" 202 Accepted 2026-03-26 15:39:36,997 - lava_callback - INFO - Starting async job submit for node 69c2b02c4a2c5046eb881162 2026-03-26 15:39:36,998 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 15:39:37,034 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b02c4a2c5046eb881162 HTTP/1.1" 200 None 2026-03-26 15:39:37,035 - lava_callback - DEBUG - Processing node 69c2b02c4a2c5046eb881162: baseline-arm64-clabbe 2026-03-26 15:39:37,036 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2b02c4a2c5046eb881162 2026-03-26 15:39:37,036 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 15:39:37,051 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b02c4a2c5046eb881162 HTTP/1.1" 200 None 2026-03-26 15:39:37,052 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 15:39:37,066 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5827544ab5cd4a7d9dd HTTP/1.1" 200 None 2026-03-26 15:39:37,067 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 15:39:37,088 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2b02c4a2c5046eb881162 HTTP/1.1" 200 None 2026-03-26 15:39:37,089 - lava_callback - ERROR - Error processing callback for node 69c2b02c4a2c5046eb881162: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:55000 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60798 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60658 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60664 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48030 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48044 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34084 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41260 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41872 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37872 - "GET / HTTP/1.0" 200 OK 2026-03-26 15:40:57,061 - lava_callback - INFO - Received callback for node 69c2b07c4a2c5046eb881170 INFO: 172.21.0.1:37888 - "POST /node/69c2b07c4a2c5046eb881170 HTTP/1.0" 202 Accepted 2026-03-26 15:40:57,067 - lava_callback - INFO - Starting async job submit for node 69c2b07c4a2c5046eb881170 2026-03-26 15:40:57,068 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 15:40:57,082 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b07c4a2c5046eb881170 HTTP/1.1" 200 None 2026-03-26 15:40:57,083 - lava_callback - DEBUG - Processing node 69c2b07c4a2c5046eb881170: baseline-arm64-clabbe 2026-03-26 15:40:57,084 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2b07c4a2c5046eb881170 2026-03-26 15:40:57,084 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 15:40:57,098 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b07c4a2c5046eb881170 HTTP/1.1" 200 None 2026-03-26 15:40:57,100 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 15:40:57,113 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5827544ab5cd4a7d9dd HTTP/1.1" 200 None 2026-03-26 15:40:57,115 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 15:40:57,149 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2b07c4a2c5046eb881170 HTTP/1.1" 200 None 2026-03-26 15:40:57,151 - lava_callback - ERROR - Error processing callback for node 69c2b07c4a2c5046eb881170: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:47156 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47164 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33252 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33268 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41590 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55060 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52942 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44168 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52470 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52476 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43006 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43008 - "GET / HTTP/1.0" 200 OK 2026-03-26 15:42:17,088 - lava_callback - INFO - Received callback for node 69c2b0cb4a2c5046eb88117f INFO: 172.21.0.1:43022 - "POST /node/69c2b0cb4a2c5046eb88117f HTTP/1.0" 202 Accepted 2026-03-26 15:42:17,097 - lava_callback - INFO - Starting async job submit for node 69c2b0cb4a2c5046eb88117f 2026-03-26 15:42:17,098 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 15:42:17,114 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b0cb4a2c5046eb88117f HTTP/1.1" 200 None 2026-03-26 15:42:17,114 - lava_callback - DEBUG - Processing node 69c2b0cb4a2c5046eb88117f: baseline-arm64-clabbe 2026-03-26 15:42:17,115 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2b0cb4a2c5046eb88117f 2026-03-26 15:42:17,115 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 15:42:17,128 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b0cb4a2c5046eb88117f HTTP/1.1" 200 None 2026-03-26 15:42:17,130 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 15:42:17,143 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5827544ab5cd4a7d9dd HTTP/1.1" 200 None 2026-03-26 15:42:17,144 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 15:42:17,166 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2b0cb4a2c5046eb88117f HTTP/1.1" 200 None 2026-03-26 15:42:17,168 - lava_callback - ERROR - Error processing callback for node 69c2b0cb4a2c5046eb88117f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:52054 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39328 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59050 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34564 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54406 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54408 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53324 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53332 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55464 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41490 - "GET / HTTP/1.0" 200 OK 2026-03-26 15:43:38,076 - lava_callback - INFO - Received callback for node 69c2b1174a2c5046eb881186 INFO: 172.21.0.1:41494 - "POST /node/69c2b1174a2c5046eb881186 HTTP/1.0" 202 Accepted 2026-03-26 15:43:38,082 - lava_callback - INFO - Starting async job submit for node 69c2b1174a2c5046eb881186 2026-03-26 15:43:38,083 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 15:43:38,096 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b1174a2c5046eb881186 HTTP/1.1" 200 None 2026-03-26 15:43:38,097 - lava_callback - DEBUG - Processing node 69c2b1174a2c5046eb881186: baseline-arm64-baylibre 2026-03-26 15:43:38,098 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2b1174a2c5046eb881186 2026-03-26 15:43:38,098 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 15:43:38,111 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b1174a2c5046eb881186 HTTP/1.1" 200 None 2026-03-26 15:43:38,112 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 15:43:38,129 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd027905e4bbd6ba634c66 HTTP/1.1" 200 None 2026-03-26 15:43:38,131 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 15:43:38,151 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2b1174a2c5046eb881186 HTTP/1.1" 200 None 2026-03-26 15:43:38,157 - lava_callback - ERROR - Error processing callback for node 69c2b1174a2c5046eb881186: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:53468 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56466 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55968 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55980 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47064 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47068 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34604 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36176 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35088 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40662 - "GET / HTTP/1.0" 200 OK 2026-03-26 15:44:58,185 - lava_callback - INFO - Received callback for node 69c2b16f4a2c5046eb881191 INFO: 172.21.0.1:40674 - "POST /node/69c2b16f4a2c5046eb881191 HTTP/1.0" 202 Accepted 2026-03-26 15:44:58,190 - lava_callback - INFO - Starting async job submit for node 69c2b16f4a2c5046eb881191 2026-03-26 15:44:58,191 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 15:44:58,206 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b16f4a2c5046eb881191 HTTP/1.1" 200 None 2026-03-26 15:44:58,207 - lava_callback - DEBUG - Processing node 69c2b16f4a2c5046eb881191: baseline-arm64-baylibre 2026-03-26 15:44:58,207 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2b16f4a2c5046eb881191 2026-03-26 15:44:58,208 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 15:44:58,222 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b16f4a2c5046eb881191 HTTP/1.1" 200 None 2026-03-26 15:44:58,223 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 15:44:58,243 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd027905e4bbd6ba634c66 HTTP/1.1" 200 None 2026-03-26 15:44:58,245 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 15:44:58,264 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2b16f4a2c5046eb881191 HTTP/1.1" 200 None 2026-03-26 15:44:58,270 - lava_callback - ERROR - Error processing callback for node 69c2b16f4a2c5046eb881191: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:42856 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42868 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53786 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53798 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35146 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54716 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39338 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42628 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41010 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41012 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55332 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55336 - "GET / HTTP/1.0" 200 OK 2026-03-26 15:46:17,299 - lava_callback - INFO - Received callback for node 69c2b1b84a2c5046eb88119f INFO: 172.21.0.1:55352 - "POST /node/69c2b1b84a2c5046eb88119f HTTP/1.0" 202 Accepted 2026-03-26 15:46:17,304 - lava_callback - INFO - Starting async job submit for node 69c2b1b84a2c5046eb88119f 2026-03-26 15:46:17,305 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 15:46:17,320 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b1b84a2c5046eb88119f HTTP/1.1" 200 None 2026-03-26 15:46:17,321 - lava_callback - DEBUG - Processing node 69c2b1b84a2c5046eb88119f: baseline-arm-clabbe 2026-03-26 15:46:17,322 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2b1b84a2c5046eb88119f 2026-03-26 15:46:17,322 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 15:46:17,338 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b1b84a2c5046eb88119f HTTP/1.1" 200 None 2026-03-26 15:46:17,339 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 15:46:17,353 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd026d05e4bbd6ba634c57 HTTP/1.1" 200 None 2026-03-26 15:46:17,354 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 15:46:17,378 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2b1b84a2c5046eb88119f HTTP/1.1" 200 None 2026-03-26 15:46:17,384 - lava_callback - ERROR - Error processing callback for node 69c2b1b84a2c5046eb88119f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:50186 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43392 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42354 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37112 - "GET / HTTP/1.0" 200 OK 2026-03-26 15:46:57,231 - lava_callback - INFO - Received callback for node 69c2b1de4a2c5046eb8811a5 INFO: 172.21.0.1:37128 - "POST /node/69c2b1de4a2c5046eb8811a5 HTTP/1.0" 202 Accepted 2026-03-26 15:46:57,237 - lava_callback - INFO - Starting async job submit for node 69c2b1de4a2c5046eb8811a5 2026-03-26 15:46:57,237 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 15:46:57,301 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b1de4a2c5046eb8811a5 HTTP/1.1" 200 None 2026-03-26 15:46:57,301 - lava_callback - DEBUG - Processing node 69c2b1de4a2c5046eb8811a5: baseline-arm-clabbe 2026-03-26 15:46:57,302 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2b1de4a2c5046eb8811a5 2026-03-26 15:46:57,302 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 15:46:57,315 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b1de4a2c5046eb8811a5 HTTP/1.1" 200 None 2026-03-26 15:46:57,317 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 15:46:57,341 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd026d05e4bbd6ba634c57 HTTP/1.1" 200 None 2026-03-26 15:46:57,343 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 15:46:57,369 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2b1de4a2c5046eb8811a5 HTTP/1.1" 200 None 2026-03-26 15:46:57,371 - lava_callback - ERROR - Error processing callback for node 69c2b1de4a2c5046eb8811a5: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:37058 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37062 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53962 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53974 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44192 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60712 - "GET / HTTP/1.0" 200 OK 2026-03-26 15:47:37,284 - lava_callback - INFO - Received callback for node 69c2b2064a2c5046eb8811ae INFO: 172.21.0.1:60716 - "POST /node/69c2b2064a2c5046eb8811ae HTTP/1.0" 202 Accepted 2026-03-26 15:47:37,289 - lava_callback - INFO - Starting async job submit for node 69c2b2064a2c5046eb8811ae 2026-03-26 15:47:37,290 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 15:47:37,304 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b2064a2c5046eb8811ae HTTP/1.1" 200 None 2026-03-26 15:47:37,305 - lava_callback - DEBUG - Processing node 69c2b2064a2c5046eb8811ae: baseline-arm-clabbe 2026-03-26 15:47:37,306 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2b2064a2c5046eb8811ae 2026-03-26 15:47:37,306 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 15:47:37,328 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b2064a2c5046eb8811ae HTTP/1.1" 200 None 2026-03-26 15:47:37,329 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 15:47:37,342 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd026d05e4bbd6ba634c57 HTTP/1.1" 200 None 2026-03-26 15:47:37,343 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 15:47:37,364 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2b2064a2c5046eb8811ae HTTP/1.1" 200 None 2026-03-26 15:47:37,365 - lava_callback - ERROR - Error processing callback for node 69c2b2064a2c5046eb8811ae: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:44246 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54666 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35450 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35452 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35460 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34864 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34880 - "GET / HTTP/1.0" 200 OK 2026-03-26 15:48:17,386 - lava_callback - INFO - Received callback for node 69c2b23c4a2c5046eb8811bb INFO: 172.21.0.1:34882 - "POST /node/69c2b23c4a2c5046eb8811bb HTTP/1.0" 202 Accepted 2026-03-26 15:48:17,392 - lava_callback - INFO - Starting async job submit for node 69c2b23c4a2c5046eb8811bb 2026-03-26 15:48:17,393 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 15:48:17,407 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b23c4a2c5046eb8811bb HTTP/1.1" 200 None 2026-03-26 15:48:17,408 - lava_callback - DEBUG - Processing node 69c2b23c4a2c5046eb8811bb: baseline-arm64-clabbe 2026-03-26 15:48:17,408 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2b23c4a2c5046eb8811bb 2026-03-26 15:48:17,409 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 15:48:17,435 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b23c4a2c5046eb8811bb HTTP/1.1" 200 None 2026-03-26 15:48:17,437 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 15:48:17,450 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd027905e4bbd6ba634c66 HTTP/1.1" 200 None 2026-03-26 15:48:17,451 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 15:48:17,477 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2b23c4a2c5046eb8811bb HTTP/1.1" 200 None 2026-03-26 15:48:17,479 - lava_callback - ERROR - Error processing callback for node 69c2b23c4a2c5046eb8811bb: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:44154 - "GET /favicon.ico HTTP/1.0" 404 Not Found INFO: 172.21.0.1:44158 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40682 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42064 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60918 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33056 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33060 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59574 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59584 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36220 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41834 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45430 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54176 - "GET / HTTP/1.0" 200 OK 2026-03-26 15:49:57,402 - lava_callback - INFO - Received callback for node 69c2b28f4a2c5046eb8811c4 INFO: 172.21.0.1:54184 - "POST /node/69c2b28f4a2c5046eb8811c4 HTTP/1.0" 202 Accepted 2026-03-26 15:49:57,407 - lava_callback - INFO - Starting async job submit for node 69c2b28f4a2c5046eb8811c4 2026-03-26 15:49:57,408 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 15:49:57,430 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b28f4a2c5046eb8811c4 HTTP/1.1" 200 None 2026-03-26 15:49:57,431 - lava_callback - DEBUG - Processing node 69c2b28f4a2c5046eb8811c4: baseline-arm64-clabbe 2026-03-26 15:49:57,432 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2b28f4a2c5046eb8811c4 2026-03-26 15:49:57,432 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 15:49:57,451 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b28f4a2c5046eb8811c4 HTTP/1.1" 200 None 2026-03-26 15:49:57,452 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 15:49:57,478 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd027905e4bbd6ba634c66 HTTP/1.1" 200 None 2026-03-26 15:49:57,480 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 15:49:57,501 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2b28f4a2c5046eb8811c4 HTTP/1.1" 200 None 2026-03-26 15:49:57,503 - lava_callback - ERROR - Error processing callback for node 69c2b28f4a2c5046eb8811c4: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:43412 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43414 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55184 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55198 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44486 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35622 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47788 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36088 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53860 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53876 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34174 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34190 - "GET / HTTP/1.0" 200 OK 2026-03-26 15:51:17,469 - lava_callback - INFO - Received callback for node 69c2b2e14a2c5046eb8811cb INFO: 172.21.0.1:34194 - "POST /node/69c2b2e14a2c5046eb8811cb HTTP/1.0" 202 Accepted 2026-03-26 15:51:17,475 - lava_callback - INFO - Starting async job submit for node 69c2b2e14a2c5046eb8811cb 2026-03-26 15:51:17,476 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 15:51:17,490 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b2e14a2c5046eb8811cb HTTP/1.1" 200 None 2026-03-26 15:51:17,491 - lava_callback - DEBUG - Processing node 69c2b2e14a2c5046eb8811cb: baseline-arm64-clabbe 2026-03-26 15:51:17,491 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2b2e14a2c5046eb8811cb 2026-03-26 15:51:17,491 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 15:51:17,504 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b2e14a2c5046eb8811cb HTTP/1.1" 200 None 2026-03-26 15:51:17,506 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 15:51:17,519 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd027905e4bbd6ba634c66 HTTP/1.1" 200 None 2026-03-26 15:51:17,520 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 15:51:17,551 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2b2e14a2c5046eb8811cb HTTP/1.1" 200 None 2026-03-26 15:51:17,553 - lava_callback - ERROR - Error processing callback for node 69c2b2e14a2c5046eb8811cb: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:46792 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34620 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59770 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36012 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56428 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56432 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49674 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49682 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55556 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51232 - "GET / HTTP/1.0" 200 OK 2026-03-26 15:52:37,563 - lava_callback - INFO - Received callback for node 69c2b3314a2c5046eb8811d6 INFO: 172.21.0.1:51240 - "POST /node/69c2b3314a2c5046eb8811d6 HTTP/1.0" 202 Accepted 2026-03-26 15:52:37,569 - lava_callback - INFO - Starting async job submit for node 69c2b3314a2c5046eb8811d6 2026-03-26 15:52:37,569 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 15:52:37,583 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b3314a2c5046eb8811d6 HTTP/1.1" 200 None 2026-03-26 15:52:37,584 - lava_callback - DEBUG - Processing node 69c2b3314a2c5046eb8811d6: baseline-arm64-clabbe 2026-03-26 15:52:37,585 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2b3314a2c5046eb8811d6 2026-03-26 15:52:37,585 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 15:52:37,598 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b3314a2c5046eb8811d6 HTTP/1.1" 200 None 2026-03-26 15:52:37,600 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 15:52:37,614 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd027905e4bbd6ba634c66 HTTP/1.1" 200 None 2026-03-26 15:52:37,615 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 15:52:37,635 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2b3314a2c5046eb8811d6 HTTP/1.1" 200 None 2026-03-26 15:52:37,636 - lava_callback - ERROR - Error processing callback for node 69c2b3314a2c5046eb8811d6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:39134 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36924 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57204 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57220 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60834 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60844 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48714 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55636 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60976 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49092 - "GET / HTTP/1.0" 200 OK 2026-03-26 15:53:57,584 - lava_callback - INFO - Received callback for node 69c2b37f4a2c5046eb8811e0 INFO: 172.21.0.1:49096 - "POST /node/69c2b37f4a2c5046eb8811e0 HTTP/1.0" 202 Accepted 2026-03-26 15:53:57,590 - lava_callback - INFO - Starting async job submit for node 69c2b37f4a2c5046eb8811e0 2026-03-26 15:53:57,590 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 15:53:57,611 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b37f4a2c5046eb8811e0 HTTP/1.1" 200 None 2026-03-26 15:53:57,612 - lava_callback - DEBUG - Processing node 69c2b37f4a2c5046eb8811e0: baseline-arm64-clabbe 2026-03-26 15:53:57,613 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2b37f4a2c5046eb8811e0 2026-03-26 15:53:57,613 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 15:53:57,639 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b37f4a2c5046eb8811e0 HTTP/1.1" 200 None 2026-03-26 15:53:57,641 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 15:53:57,656 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd027905e4bbd6ba634c66 HTTP/1.1" 200 None 2026-03-26 15:53:57,658 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 15:53:57,741 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2b37f4a2c5046eb8811e0 HTTP/1.1" 200 None 2026-03-26 15:53:57,743 - lava_callback - ERROR - Error processing callback for node 69c2b37f4a2c5046eb8811e0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:50336 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50348 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35092 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35102 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34988 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36014 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60154 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44198 - "GET / HTTP/1.0" 200 OK 2026-03-26 15:54:57,585 - lava_callback - INFO - Received callback for node 69c2b3ce4a2c5046eb8811e7 INFO: 172.21.0.1:44202 - "POST /node/69c2b3ce4a2c5046eb8811e7 HTTP/1.0" 202 Accepted 2026-03-26 15:54:57,591 - lava_callback - INFO - Starting async job submit for node 69c2b3ce4a2c5046eb8811e7 2026-03-26 15:54:57,592 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 15:54:57,607 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b3ce4a2c5046eb8811e7 HTTP/1.1" 200 None 2026-03-26 15:54:57,608 - lava_callback - DEBUG - Processing node 69c2b3ce4a2c5046eb8811e7: baseline-arm64-clabbe 2026-03-26 15:54:57,608 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2b3ce4a2c5046eb8811e7 2026-03-26 15:54:57,609 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 15:54:57,621 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b3ce4a2c5046eb8811e7 HTTP/1.1" 200 None 2026-03-26 15:54:57,623 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 15:54:57,656 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd027905e4bbd6ba634c66 HTTP/1.1" 200 None 2026-03-26 15:54:57,658 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 15:54:57,678 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2b3ce4a2c5046eb8811e7 HTTP/1.1" 200 None 2026-03-26 15:54:57,679 - lava_callback - ERROR - Error processing callback for node 69c2b3ce4a2c5046eb8811e7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:33756 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33772 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60604 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60614 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59814 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42110 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53390 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52864 - "GET / HTTP/1.0" 200 OK 2026-03-26 15:55:58,878 - lava_callback - INFO - Received callback for node 69c2b40c4a2c5046eb8811ee INFO: 172.21.0.1:52870 - "POST /node/69c2b40c4a2c5046eb8811ee HTTP/1.0" 202 Accepted 2026-03-26 15:55:58,884 - lava_callback - INFO - Starting async job submit for node 69c2b40c4a2c5046eb8811ee 2026-03-26 15:55:58,885 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 15:55:58,899 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b40c4a2c5046eb8811ee HTTP/1.1" 200 None 2026-03-26 15:55:58,899 - lava_callback - DEBUG - Processing node 69c2b40c4a2c5046eb8811ee: baseline-arm-baylibre 2026-03-26 15:55:58,900 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2b40c4a2c5046eb8811ee 2026-03-26 15:55:58,900 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 15:55:58,914 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b40c4a2c5046eb8811ee HTTP/1.1" 200 None 2026-03-26 15:55:58,915 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 15:55:58,929 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5737544ab5cd4a7d9ce HTTP/1.1" 200 None 2026-03-26 15:55:58,930 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 15:55:58,958 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2b40c4a2c5046eb8811ee HTTP/1.1" 200 None 2026-03-26 15:55:58,960 - lava_callback - ERROR - Error processing callback for node 69c2b40c4a2c5046eb8811ee: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:58666 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58668 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51592 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51594 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57290 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44388 - "GET / HTTP/1.0" 200 OK 2026-03-26 15:56:38,747 - lava_callback - INFO - Received callback for node 69c2b4324a2c5046eb8811f9 INFO: 172.21.0.1:44402 - "POST /node/69c2b4324a2c5046eb8811f9 HTTP/1.0" 202 Accepted 2026-03-26 15:56:38,753 - lava_callback - INFO - Starting async job submit for node 69c2b4324a2c5046eb8811f9 2026-03-26 15:56:38,754 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 15:56:38,767 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b4324a2c5046eb8811f9 HTTP/1.1" 200 None 2026-03-26 15:56:38,768 - lava_callback - DEBUG - Processing node 69c2b4324a2c5046eb8811f9: baseline-arm-baylibre 2026-03-26 15:56:38,768 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2b4324a2c5046eb8811f9 2026-03-26 15:56:38,769 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 15:56:38,782 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b4324a2c5046eb8811f9 HTTP/1.1" 200 None 2026-03-26 15:56:38,783 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 15:56:38,796 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5737544ab5cd4a7d9ce HTTP/1.1" 200 None 2026-03-26 15:56:38,798 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 15:56:38,818 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2b4324a2c5046eb8811f9 HTTP/1.1" 200 None 2026-03-26 15:56:38,827 - lava_callback - ERROR - Error processing callback for node 69c2b4324a2c5046eb8811f9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:49280 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57252 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42544 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42550 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52290 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52294 - "GET / HTTP/1.0" 200 OK 2026-03-26 15:57:18,813 - lava_callback - INFO - Received callback for node 69c2b4594a2c5046eb881202 INFO: 172.21.0.1:52310 - "POST /node/69c2b4594a2c5046eb881202 HTTP/1.0" 202 Accepted 2026-03-26 15:57:18,818 - lava_callback - INFO - Starting async job submit for node 69c2b4594a2c5046eb881202 2026-03-26 15:57:18,819 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 15:57:18,835 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b4594a2c5046eb881202 HTTP/1.1" 200 None 2026-03-26 15:57:18,836 - lava_callback - DEBUG - Processing node 69c2b4594a2c5046eb881202: baseline-arm-baylibre 2026-03-26 15:57:18,836 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2b4594a2c5046eb881202 2026-03-26 15:57:18,837 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 15:57:18,850 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b4594a2c5046eb881202 HTTP/1.1" 200 None 2026-03-26 15:57:18,852 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 15:57:18,865 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5737544ab5cd4a7d9ce HTTP/1.1" 200 None 2026-03-26 15:57:18,866 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 15:57:18,889 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2b4594a2c5046eb881202 HTTP/1.1" 200 None 2026-03-26 15:57:18,890 - lava_callback - ERROR - Error processing callback for node 69c2b4594a2c5046eb881202: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:57706 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59918 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45204 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36202 - "GET / HTTP/1.0" 200 OK 2026-03-26 15:57:58,825 - lava_callback - INFO - Received callback for node 69c2b47e4a2c5046eb88120c INFO: 172.21.0.1:36210 - "POST /node/69c2b47e4a2c5046eb88120c HTTP/1.0" 202 Accepted 2026-03-26 15:57:58,830 - lava_callback - INFO - Starting async job submit for node 69c2b47e4a2c5046eb88120c 2026-03-26 15:57:58,831 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 15:57:58,849 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b47e4a2c5046eb88120c HTTP/1.1" 200 None 2026-03-26 15:57:58,850 - lava_callback - DEBUG - Processing node 69c2b47e4a2c5046eb88120c: baseline-arm-baylibre 2026-03-26 15:57:58,851 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2b47e4a2c5046eb88120c 2026-03-26 15:57:58,851 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 15:57:58,865 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b47e4a2c5046eb88120c HTTP/1.1" 200 None 2026-03-26 15:57:58,867 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 15:57:58,881 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5737544ab5cd4a7d9ce HTTP/1.1" 200 None 2026-03-26 15:57:58,882 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 15:57:58,903 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2b47e4a2c5046eb88120c HTTP/1.1" 200 None 2026-03-26 15:57:58,905 - lava_callback - ERROR - Error processing callback for node 69c2b47e4a2c5046eb88120c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:47540 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47552 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37800 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37804 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45756 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48544 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48940 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56212 - "GET / HTTP/1.0" 200 OK 2026-03-26 15:58:57,740 - lava_callback - INFO - Received callback for node 69c2b4b44a2c5046eb881217 INFO: 172.21.0.1:56224 - "POST /node/69c2b4b44a2c5046eb881217 HTTP/1.0" 202 Accepted 2026-03-26 15:58:57,746 - lava_callback - INFO - Starting async job submit for node 69c2b4b44a2c5046eb881217 2026-03-26 15:58:57,747 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 15:58:57,761 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b4b44a2c5046eb881217 HTTP/1.1" 200 None 2026-03-26 15:58:57,762 - lava_callback - DEBUG - Processing node 69c2b4b44a2c5046eb881217: baseline-arm64-clabbe 2026-03-26 15:58:57,762 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2b4b44a2c5046eb881217 2026-03-26 15:58:57,763 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 15:58:57,776 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b4b44a2c5046eb881217 HTTP/1.1" 200 None 2026-03-26 15:58:57,777 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 15:58:57,795 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd027905e4bbd6ba634c66 HTTP/1.1" 200 None 2026-03-26 15:58:57,796 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 15:58:57,822 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2b4b44a2c5046eb881217 HTTP/1.1" 200 None 2026-03-26 15:58:57,833 - lava_callback - ERROR - Error processing callback for node 69c2b4b44a2c5046eb881217: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:42870 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42886 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34246 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34252 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34040 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35108 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37172 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55786 - "GET / HTTP/1.0" 200 OK 2026-03-26 15:59:59,224 - lava_callback - INFO - Received callback for node 69c2b4f04a2c5046eb88121d INFO: 172.21.0.1:55798 - "POST /node/69c2b4f04a2c5046eb88121d HTTP/1.0" 202 Accepted 2026-03-26 15:59:59,229 - lava_callback - INFO - Starting async job submit for node 69c2b4f04a2c5046eb88121d 2026-03-26 15:59:59,230 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 15:59:59,246 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b4f04a2c5046eb88121d HTTP/1.1" 200 None 2026-03-26 15:59:59,247 - lava_callback - DEBUG - Processing node 69c2b4f04a2c5046eb88121d: baseline-arm-baylibre 2026-03-26 15:59:59,247 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2b4f04a2c5046eb88121d 2026-03-26 15:59:59,247 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 15:59:59,261 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b4f04a2c5046eb88121d HTTP/1.1" 200 None 2026-03-26 15:59:59,262 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 15:59:59,284 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5737544ab5cd4a7d9ce HTTP/1.1" 200 None 2026-03-26 15:59:59,286 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 15:59:59,315 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2b4f04a2c5046eb88121d HTTP/1.1" 200 None 2026-03-26 15:59:59,317 - lava_callback - ERROR - Error processing callback for node 69c2b4f04a2c5046eb88121d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:58302 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58310 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38996 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39008 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48642 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38472 - "GET / HTTP/1.0" 200 OK 2026-03-26 16:00:38,943 - lava_callback - INFO - Received callback for node 69c2b5164a2c5046eb881226 INFO: 172.21.0.1:38478 - "POST /node/69c2b5164a2c5046eb881226 HTTP/1.0" 202 Accepted 2026-03-26 16:00:38,949 - lava_callback - INFO - Starting async job submit for node 69c2b5164a2c5046eb881226 2026-03-26 16:00:38,950 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:00:38,964 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b5164a2c5046eb881226 HTTP/1.1" 200 None 2026-03-26 16:00:38,965 - lava_callback - DEBUG - Processing node 69c2b5164a2c5046eb881226: baseline-arm-baylibre 2026-03-26 16:00:38,965 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2b5164a2c5046eb881226 2026-03-26 16:00:38,966 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:00:38,989 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b5164a2c5046eb881226 HTTP/1.1" 200 None 2026-03-26 16:00:38,991 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:00:39,004 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5737544ab5cd4a7d9ce HTTP/1.1" 200 None 2026-03-26 16:00:39,006 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:00:39,035 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2b5164a2c5046eb881226 HTTP/1.1" 200 None 2026-03-26 16:00:39,040 - lava_callback - ERROR - Error processing callback for node 69c2b5164a2c5046eb881226: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:52046 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55068 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54580 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54596 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53310 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53314 - "GET / HTTP/1.0" 200 OK 2026-03-26 16:01:17,876 - lava_callback - INFO - Received callback for node 69c2b53e4a2c5046eb881231 INFO: 172.21.0.1:53330 - "POST /node/69c2b53e4a2c5046eb881231 HTTP/1.0" 202 Accepted 2026-03-26 16:01:17,881 - lava_callback - INFO - Starting async job submit for node 69c2b53e4a2c5046eb881231 2026-03-26 16:01:17,882 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:01:17,899 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b53e4a2c5046eb881231 HTTP/1.1" 200 None 2026-03-26 16:01:17,900 - lava_callback - DEBUG - Processing node 69c2b53e4a2c5046eb881231: baseline-arm-clabbe 2026-03-26 16:01:17,900 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2b53e4a2c5046eb881231 2026-03-26 16:01:17,901 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:01:17,914 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b53e4a2c5046eb881231 HTTP/1.1" 200 None 2026-03-26 16:01:17,915 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:01:17,928 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5737544ab5cd4a7d9ce HTTP/1.1" 200 None 2026-03-26 16:01:17,930 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:01:17,953 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2b53e4a2c5046eb881231 HTTP/1.1" 200 None 2026-03-26 16:01:17,960 - lava_callback - ERROR - Error processing callback for node 69c2b53e4a2c5046eb881231: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:38920 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46002 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48900 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35972 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33574 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33586 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42070 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42074 - "GET / HTTP/1.0" 200 OK 2026-03-26 16:02:17,958 - lava_callback - INFO - Received callback for node 69c2b5744a2c5046eb881238 INFO: 172.21.0.1:42078 - "POST /node/69c2b5744a2c5046eb881238 HTTP/1.0" 202 Accepted 2026-03-26 16:02:17,964 - lava_callback - INFO - Starting async job submit for node 69c2b5744a2c5046eb881238 2026-03-26 16:02:17,965 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:02:17,981 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b5744a2c5046eb881238 HTTP/1.1" 200 None 2026-03-26 16:02:17,982 - lava_callback - DEBUG - Processing node 69c2b5744a2c5046eb881238: baseline-arm64-clabbe 2026-03-26 16:02:17,982 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2b5744a2c5046eb881238 2026-03-26 16:02:17,982 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:02:18,014 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b5744a2c5046eb881238 HTTP/1.1" 200 None 2026-03-26 16:02:18,016 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:02:18,029 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd027905e4bbd6ba634c66 HTTP/1.1" 200 None 2026-03-26 16:02:18,030 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:02:18,050 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2b5744a2c5046eb881238 HTTP/1.1" 200 None 2026-03-26 16:02:18,052 - lava_callback - ERROR - Error processing callback for node 69c2b5744a2c5046eb881238: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:35494 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39880 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52914 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59826 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51388 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51398 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53182 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53184 - "GET / HTTP/1.0" 200 OK 2026-03-26 16:03:17,926 - lava_callback - INFO - Received callback for node 69c2b5b44a2c5046eb88123f INFO: 172.21.0.1:53198 - "POST /node/69c2b5b44a2c5046eb88123f HTTP/1.0" 202 Accepted 2026-03-26 16:03:17,932 - lava_callback - INFO - Starting async job submit for node 69c2b5b44a2c5046eb88123f 2026-03-26 16:03:17,932 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:03:17,947 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b5b44a2c5046eb88123f HTTP/1.1" 200 None 2026-03-26 16:03:17,947 - lava_callback - DEBUG - Processing node 69c2b5b44a2c5046eb88123f: baseline-arm-clabbe 2026-03-26 16:03:17,948 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2b5b44a2c5046eb88123f 2026-03-26 16:03:17,948 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:03:17,962 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b5b44a2c5046eb88123f HTTP/1.1" 200 None 2026-03-26 16:03:17,963 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:03:17,977 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5737544ab5cd4a7d9ce HTTP/1.1" 200 None 2026-03-26 16:03:17,978 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:03:17,999 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2b5b44a2c5046eb88123f HTTP/1.1" 200 None 2026-03-26 16:03:18,000 - lava_callback - ERROR - Error processing callback for node 69c2b5b44a2c5046eb88123f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:40862 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57914 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45084 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42752 - "GET / HTTP/1.0" 200 OK 2026-03-26 16:03:57,933 - lava_callback - INFO - Received callback for node 69c2b5da4a2c5046eb88124b INFO: 172.21.0.1:42766 - "POST /node/69c2b5da4a2c5046eb88124b HTTP/1.0" 202 Accepted 2026-03-26 16:03:57,939 - lava_callback - INFO - Starting async job submit for node 69c2b5da4a2c5046eb88124b 2026-03-26 16:03:57,940 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:03:57,956 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b5da4a2c5046eb88124b HTTP/1.1" 200 None 2026-03-26 16:03:57,957 - lava_callback - DEBUG - Processing node 69c2b5da4a2c5046eb88124b: baseline-arm-clabbe 2026-03-26 16:03:57,958 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2b5da4a2c5046eb88124b 2026-03-26 16:03:57,958 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:03:57,998 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b5da4a2c5046eb88124b HTTP/1.1" 200 None 2026-03-26 16:03:57,999 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:03:58,013 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5737544ab5cd4a7d9ce HTTP/1.1" 200 None 2026-03-26 16:03:58,014 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:03:58,035 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2b5da4a2c5046eb88124b HTTP/1.1" 200 None 2026-03-26 16:03:58,036 - lava_callback - ERROR - Error processing callback for node 69c2b5da4a2c5046eb88124b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:45758 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45770 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40092 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40100 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53778 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41092 - "GET / HTTP/1.0" 200 OK 2026-03-26 16:04:38,145 - lava_callback - INFO - Received callback for node 69c2b6024a2c5046eb881253 INFO: 172.21.0.1:41098 - "POST /node/69c2b6024a2c5046eb881253 HTTP/1.0" 202 Accepted 2026-03-26 16:04:38,151 - lava_callback - INFO - Starting async job submit for node 69c2b6024a2c5046eb881253 2026-03-26 16:04:38,152 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:04:38,165 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b6024a2c5046eb881253 HTTP/1.1" 200 None 2026-03-26 16:04:38,166 - lava_callback - DEBUG - Processing node 69c2b6024a2c5046eb881253: baseline-arm-clabbe 2026-03-26 16:04:38,166 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2b6024a2c5046eb881253 2026-03-26 16:04:38,167 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:04:38,180 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b6024a2c5046eb881253 HTTP/1.1" 200 None 2026-03-26 16:04:38,181 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:04:38,194 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5737544ab5cd4a7d9ce HTTP/1.1" 200 None 2026-03-26 16:04:38,196 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:04:38,215 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2b6024a2c5046eb881253 HTTP/1.1" 200 None 2026-03-26 16:04:38,217 - lava_callback - ERROR - Error processing callback for node 69c2b6024a2c5046eb881253: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:60026 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42558 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41714 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41720 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38834 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38842 - "GET / HTTP/1.0" 200 OK 2026-03-26 16:05:18,000 - lava_callback - INFO - Received callback for node 69c2b6284a2c5046eb88125c INFO: 172.21.0.1:38848 - "POST /node/69c2b6284a2c5046eb88125c HTTP/1.0" 202 Accepted 2026-03-26 16:05:18,007 - lava_callback - INFO - Starting async job submit for node 69c2b6284a2c5046eb88125c 2026-03-26 16:05:18,008 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:05:18,022 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b6284a2c5046eb88125c HTTP/1.1" 200 None 2026-03-26 16:05:18,023 - lava_callback - DEBUG - Processing node 69c2b6284a2c5046eb88125c: baseline-arm-clabbe 2026-03-26 16:05:18,023 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2b6284a2c5046eb88125c 2026-03-26 16:05:18,023 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:05:18,037 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b6284a2c5046eb88125c HTTP/1.1" 200 None 2026-03-26 16:05:18,038 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:05:18,051 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5737544ab5cd4a7d9ce HTTP/1.1" 200 None 2026-03-26 16:05:18,053 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:05:18,075 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2b6284a2c5046eb88125c HTTP/1.1" 200 None 2026-03-26 16:05:18,084 - lava_callback - ERROR - Error processing callback for node 69c2b6284a2c5046eb88125c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:58652 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44416 - "GET / HTTP/1.0" 200 OK 2026-03-26 16:05:38,021 - lava_callback - INFO - Received callback for node 69c2b64e4a2c5046eb881267 INFO: 172.21.0.1:44432 - "POST /node/69c2b64e4a2c5046eb881267 HTTP/1.0" 202 Accepted 2026-03-26 16:05:38,026 - lava_callback - INFO - Starting async job submit for node 69c2b64e4a2c5046eb881267 2026-03-26 16:05:38,027 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:05:38,042 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b64e4a2c5046eb881267 HTTP/1.1" 200 None 2026-03-26 16:05:38,042 - lava_callback - DEBUG - Processing node 69c2b64e4a2c5046eb881267: baseline-arm-clabbe 2026-03-26 16:05:38,043 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2b64e4a2c5046eb881267 2026-03-26 16:05:38,043 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:05:38,058 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b64e4a2c5046eb881267 HTTP/1.1" 200 None 2026-03-26 16:05:38,060 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:05:38,080 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5737544ab5cd4a7d9ce HTTP/1.1" 200 None 2026-03-26 16:05:38,081 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:05:38,102 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2b64e4a2c5046eb881267 HTTP/1.1" 200 None 2026-03-26 16:05:38,104 - lava_callback - ERROR - Error processing callback for node 69c2b64e4a2c5046eb881267: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:48420 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51760 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55798 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55814 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33128 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33142 - "GET / HTTP/1.0" 200 OK 2026-03-26 16:06:18,026 - lava_callback - INFO - Received callback for node 69c2b6754a2c5046eb88126e INFO: 172.21.0.1:33144 - "POST /node/69c2b6754a2c5046eb88126e HTTP/1.0" 202 Accepted 2026-03-26 16:06:18,032 - lava_callback - INFO - Starting async job submit for node 69c2b6754a2c5046eb88126e 2026-03-26 16:06:18,032 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:06:18,047 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b6754a2c5046eb88126e HTTP/1.1" 200 None 2026-03-26 16:06:18,048 - lava_callback - DEBUG - Processing node 69c2b6754a2c5046eb88126e: baseline-arm-clabbe 2026-03-26 16:06:18,049 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2b6754a2c5046eb88126e 2026-03-26 16:06:18,049 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:06:18,063 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b6754a2c5046eb88126e HTTP/1.1" 200 None 2026-03-26 16:06:18,065 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:06:18,081 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5737544ab5cd4a7d9ce HTTP/1.1" 200 None 2026-03-26 16:06:18,083 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:06:18,105 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2b6754a2c5046eb88126e HTTP/1.1" 200 None 2026-03-26 16:06:18,108 - lava_callback - ERROR - Error processing callback for node 69c2b6754a2c5046eb88126e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:50982 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56680 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33920 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56898 - "GET / HTTP/1.0" 200 OK 2026-03-26 16:06:58,102 - lava_callback - INFO - Received callback for node 69c2b69d4a2c5046eb881277 INFO: 172.21.0.1:56904 - "POST /node/69c2b69d4a2c5046eb881277 HTTP/1.0" 202 Accepted 2026-03-26 16:06:58,108 - lava_callback - INFO - Starting async job submit for node 69c2b69d4a2c5046eb881277 2026-03-26 16:06:58,109 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:06:58,137 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b69d4a2c5046eb881277 HTTP/1.1" 200 None 2026-03-26 16:06:58,138 - lava_callback - DEBUG - Processing node 69c2b69d4a2c5046eb881277: baseline-arm-clabbe 2026-03-26 16:06:58,139 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2b69d4a2c5046eb881277 2026-03-26 16:06:58,139 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:06:58,153 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b69d4a2c5046eb881277 HTTP/1.1" 200 None 2026-03-26 16:06:58,154 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:06:58,192 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5737544ab5cd4a7d9ce HTTP/1.1" 200 None 2026-03-26 16:06:58,194 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:06:58,214 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2b69d4a2c5046eb881277 HTTP/1.1" 200 None 2026-03-26 16:06:58,216 - lava_callback - ERROR - Error processing callback for node 69c2b69d4a2c5046eb881277: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:50756 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50772 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34224 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34238 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34776 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36544 - "GET / HTTP/1.0" 200 OK 2026-03-26 16:07:38,108 - lava_callback - INFO - Received callback for node 69c2b6c44a2c5046eb881282 INFO: 172.21.0.1:36552 - "POST /node/69c2b6c44a2c5046eb881282 HTTP/1.0" 202 Accepted 2026-03-26 16:07:38,135 - lava_callback - INFO - Starting async job submit for node 69c2b6c44a2c5046eb881282 2026-03-26 16:07:38,136 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:07:38,175 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b6c44a2c5046eb881282 HTTP/1.1" 200 None 2026-03-26 16:07:38,176 - lava_callback - DEBUG - Processing node 69c2b6c44a2c5046eb881282: baseline-arm-clabbe 2026-03-26 16:07:38,176 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2b6c44a2c5046eb881282 2026-03-26 16:07:38,177 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:07:38,190 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b6c44a2c5046eb881282 HTTP/1.1" 200 None 2026-03-26 16:07:38,192 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:07:38,205 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5737544ab5cd4a7d9ce HTTP/1.1" 200 None 2026-03-26 16:07:38,207 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:07:38,231 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2b6c44a2c5046eb881282 HTTP/1.1" 200 None 2026-03-26 16:07:38,233 - lava_callback - ERROR - Error processing callback for node 69c2b6c44a2c5046eb881282: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:59312 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56076 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37766 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37774 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39950 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39962 - "GET / HTTP/1.0" 200 OK 2026-03-26 16:08:18,178 - lava_callback - INFO - Received callback for node 69c2b6ec4a2c5046eb881289 INFO: 172.21.0.1:39976 - "POST /node/69c2b6ec4a2c5046eb881289 HTTP/1.0" 202 Accepted 2026-03-26 16:08:18,184 - lava_callback - INFO - Starting async job submit for node 69c2b6ec4a2c5046eb881289 2026-03-26 16:08:18,185 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:08:18,199 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b6ec4a2c5046eb881289 HTTP/1.1" 200 None 2026-03-26 16:08:18,200 - lava_callback - DEBUG - Processing node 69c2b6ec4a2c5046eb881289: baseline-arm-clabbe 2026-03-26 16:08:18,200 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2b6ec4a2c5046eb881289 2026-03-26 16:08:18,201 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:08:18,213 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b6ec4a2c5046eb881289 HTTP/1.1" 200 None 2026-03-26 16:08:18,215 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:08:18,228 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5737544ab5cd4a7d9ce HTTP/1.1" 200 None 2026-03-26 16:08:18,230 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:08:18,250 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2b6ec4a2c5046eb881289 HTTP/1.1" 200 None 2026-03-26 16:08:18,259 - lava_callback - ERROR - Error processing callback for node 69c2b6ec4a2c5046eb881289: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:38836 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59908 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53404 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37834 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46108 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46122 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52350 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52358 - "GET / HTTP/1.0" 200 OK 2026-03-26 16:09:18,223 - lava_callback - INFO - Received callback for node 69c2b7224a2c5046eb881295 INFO: 172.21.0.1:52366 - "POST /node/69c2b7224a2c5046eb881295 HTTP/1.0" 202 Accepted 2026-03-26 16:09:18,229 - lava_callback - INFO - Starting async job submit for node 69c2b7224a2c5046eb881295 2026-03-26 16:09:18,230 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:09:18,244 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b7224a2c5046eb881295 HTTP/1.1" 200 None 2026-03-26 16:09:18,245 - lava_callback - DEBUG - Processing node 69c2b7224a2c5046eb881295: baseline-arm64-clabbe 2026-03-26 16:09:18,245 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2b7224a2c5046eb881295 2026-03-26 16:09:18,246 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:09:18,258 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b7224a2c5046eb881295 HTTP/1.1" 200 None 2026-03-26 16:09:18,260 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:09:18,278 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd027905e4bbd6ba634c66 HTTP/1.1" 200 None 2026-03-26 16:09:18,280 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:09:18,307 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2b7224a2c5046eb881295 HTTP/1.1" 200 None 2026-03-26 16:09:18,313 - lava_callback - ERROR - Error processing callback for node 69c2b7224a2c5046eb881295: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:45308 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48628 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58442 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59624 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37090 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37092 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53030 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53046 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55752 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33610 - "GET / HTTP/1.0" 200 OK 2026-03-26 16:10:38,290 - lava_callback - INFO - Received callback for node 69c2b7754a2c5046eb88129a INFO: 172.21.0.1:33614 - "POST /node/69c2b7754a2c5046eb88129a HTTP/1.0" 202 Accepted 2026-03-26 16:10:38,296 - lava_callback - INFO - Starting async job submit for node 69c2b7754a2c5046eb88129a 2026-03-26 16:10:38,297 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:10:38,312 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b7754a2c5046eb88129a HTTP/1.1" 200 None 2026-03-26 16:10:38,313 - lava_callback - DEBUG - Processing node 69c2b7754a2c5046eb88129a: baseline-arm64-clabbe 2026-03-26 16:10:38,313 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2b7754a2c5046eb88129a 2026-03-26 16:10:38,314 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:10:38,327 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b7754a2c5046eb88129a HTTP/1.1" 200 None 2026-03-26 16:10:38,328 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:10:38,346 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd027905e4bbd6ba634c66 HTTP/1.1" 200 None 2026-03-26 16:10:38,348 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:10:38,369 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2b7754a2c5046eb88129a HTTP/1.1" 200 None 2026-03-26 16:10:38,370 - lava_callback - ERROR - Error processing callback for node 69c2b7754a2c5046eb88129a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:39874 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41592 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33186 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33194 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50314 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50330 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49248 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35098 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43666 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57740 - "GET / HTTP/1.0" 200 OK 2026-03-26 16:11:58,336 - lava_callback - INFO - Received callback for node 69c2b7ca4a2c5046eb8812a2 INFO: 172.21.0.1:57748 - "POST /node/69c2b7ca4a2c5046eb8812a2 HTTP/1.0" 202 Accepted 2026-03-26 16:11:58,342 - lava_callback - INFO - Starting async job submit for node 69c2b7ca4a2c5046eb8812a2 2026-03-26 16:11:58,343 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:11:58,357 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b7ca4a2c5046eb8812a2 HTTP/1.1" 200 None 2026-03-26 16:11:58,358 - lava_callback - DEBUG - Processing node 69c2b7ca4a2c5046eb8812a2: baseline-arm64-clabbe 2026-03-26 16:11:58,358 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2b7ca4a2c5046eb8812a2 2026-03-26 16:11:58,359 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:11:58,374 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b7ca4a2c5046eb8812a2 HTTP/1.1" 200 None 2026-03-26 16:11:58,375 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:11:58,389 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd027905e4bbd6ba634c66 HTTP/1.1" 200 None 2026-03-26 16:11:58,390 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:11:58,410 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2b7ca4a2c5046eb8812a2 HTTP/1.1" 200 None 2026-03-26 16:11:58,412 - lava_callback - ERROR - Error processing callback for node 69c2b7ca4a2c5046eb8812a2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:50538 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50544 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44018 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44032 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56228 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46944 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51858 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47756 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47852 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47856 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46386 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46394 - "GET / HTTP/1.0" 200 OK 2026-03-26 16:13:19,704 - lava_callback - INFO - Received callback for node 69c2b8164a2c5046eb8812ac INFO: 172.21.0.1:52284 - "POST /node/69c2b8164a2c5046eb8812ac HTTP/1.0" 202 Accepted 2026-03-26 16:13:19,711 - lava_callback - INFO - Starting async job submit for node 69c2b8164a2c5046eb8812ac 2026-03-26 16:13:19,712 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:13:19,728 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b8164a2c5046eb8812ac HTTP/1.1" 200 None 2026-03-26 16:13:19,729 - lava_callback - DEBUG - Processing node 69c2b8164a2c5046eb8812ac: baseline-arm64-baylibre 2026-03-26 16:13:19,729 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2b8164a2c5046eb8812ac 2026-03-26 16:13:19,730 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:13:19,744 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b8164a2c5046eb8812ac HTTP/1.1" 200 None 2026-03-26 16:13:19,745 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:13:19,760 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd027905e4bbd6ba634c66 HTTP/1.1" 200 None 2026-03-26 16:13:19,761 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:13:19,781 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2b8164a2c5046eb8812ac HTTP/1.1" 200 None 2026-03-26 16:13:19,782 - lava_callback - ERROR - Error processing callback for node 69c2b8164a2c5046eb8812ac: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:52292 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47268 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51568 - "GET / HTTP/1.0" 200 OK INFO: Shutting down INFO: Waiting for application shutdown. INFO: Application shutdown complete. INFO: Finished server process [1]