INFO: Started server process [1] INFO: Waiting for application startup. INFO: Application startup complete. INFO: Uvicorn running on http://0.0.0.0:8000 (Press CTRL+C to quit) INFO: 172.20.0.1:33594 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33608 - "GET /node/698f9c383013abef95d048ca%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:44722 - "GET /node/69b38767974c91e64560f8bc%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:44728 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44738 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52506 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39934 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57094 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49824 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49828 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49744 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38818 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38826 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42714 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40504 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60848 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45826 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45842 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35752 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45888 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45896 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58914 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34502 - "GET /node/6993c48e3e9ba4930770e2fe%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:34512 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60444 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53378 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53380 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55440 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55258 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55264 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43752 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35398 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60428 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54228 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54242 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37290 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51128 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51136 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52208 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45742 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37062 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41856 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41868 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48278 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53922 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53932 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43992 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36444 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59950 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59956 - "GET /node/693ae70efee8383e92b64ffd%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:59962 - "GET /node/69937f47cebe99dd8e06d0b2%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:33776 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33790 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35146 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51180 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51186 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60708 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56642 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33664 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54374 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54376 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37748 - "GET /node/6910a47d8b15debe9dd52758%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:37752 - "GET /node/698ef6def0e3ffbba67d325e%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:37760 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34596 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34608 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33942 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42006 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57174 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34790 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34800 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44116 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45338 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45352 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34114 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60180 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48658 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43138 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43154 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37390 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58742 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58756 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50096 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39612 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54826 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54798 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54810 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54292 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60944 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60960 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47888 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57330 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34890 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34896 - "GET /node/691909a29c0fefc40758617a%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:34910 - "GET /node/69893d0985773790cd34db62%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:35906 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35908 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52276 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59026 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59040 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38694 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43998 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55586 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45174 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45184 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59306 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58210 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58220 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38392 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42914 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43188 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51688 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51696 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37210 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51474 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51482 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34590 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49280 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38830 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44668 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44678 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54092 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51796 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51806 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46886 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49534 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56722 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44336 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44338 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46382 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43698 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43708 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42364 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43980 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57572 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45862 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45866 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43514 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60308 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60318 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52488 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53850 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36278 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60122 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60138 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53984 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56808 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56818 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56606 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41224 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52248 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36450 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36462 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41262 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48330 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48336 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35616 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51444 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47330 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50196 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50210 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58266 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47108 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47120 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52962 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42972 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44556 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35492 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35494 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43602 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34134 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34138 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49474 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46272 - "GET / HTTP/1.0" 200 OK 2026-03-31 00:34:59,978 - lava_callback - INFO - Received callback for node 69c873a2f41e4fb6a3785589 2026-03-31 00:34:59,985 - lava_callback - INFO - Starting async job submit for node 69c873a2f41e4fb6a3785589 INFO: 172.20.0.1:35888 - "POST /node/69c873a2f41e4fb6a3785589 HTTP/1.0" 202 Accepted 2026-03-31 00:34:59,987 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:35:00,007 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c873a2f41e4fb6a3785589 HTTP/1.1" 200 None 2026-03-31 00:35:00,008 - lava_callback - DEBUG - Processing node 69c873a2f41e4fb6a3785589: baseline-arm-baylibre 2026-03-31 00:35:00,008 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c873a2f41e4fb6a3785589 2026-03-31 00:35:00,009 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:35:00,024 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c873a2f41e4fb6a3785589 HTTP/1.1" 200 None 2026-03-31 00:35:00,026 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:35:00,044 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-03-31 00:35:00,046 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:35:00,075 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c873a2f41e4fb6a3785589 HTTP/1.1" 200 None 2026-03-31 00:35:00,081 - lava_callback - ERROR - Error processing callback for node 69c873a2f41e4fb6a3785589: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:35898 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51882 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51896 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50542 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53472 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53486 - "GET / HTTP/1.0" 200 OK 2026-03-31 00:35:39,819 - lava_callback - INFO - Received callback for node 69c873c8f41e4fb6a378559e INFO: 172.20.0.1:42916 - "POST /node/69c873c8f41e4fb6a378559e HTTP/1.0" 202 Accepted 2026-03-31 00:35:39,826 - lava_callback - INFO - Starting async job submit for node 69c873c8f41e4fb6a378559e 2026-03-31 00:35:39,827 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:35:39,844 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c873c8f41e4fb6a378559e HTTP/1.1" 200 None 2026-03-31 00:35:39,845 - lava_callback - DEBUG - Processing node 69c873c8f41e4fb6a378559e: baseline-arm-baylibre 2026-03-31 00:35:39,845 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c873c8f41e4fb6a378559e 2026-03-31 00:35:39,846 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:35:39,864 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c873c8f41e4fb6a378559e HTTP/1.1" 200 None 2026-03-31 00:35:39,866 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:35:39,880 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-03-31 00:35:39,882 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:35:39,907 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c873c8f41e4fb6a378559e HTTP/1.1" 200 None 2026-03-31 00:35:39,908 - lava_callback - ERROR - Error processing callback for node 69c873c8f41e4fb6a378559e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:42930 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47386 - "GET / HTTP/1.0" 200 OK 2026-03-31 00:35:59,850 - lava_callback - INFO - Received callback for node 69c873edf41e4fb6a37855ab INFO: 172.20.0.1:56414 - "POST /node/69c873edf41e4fb6a37855ab HTTP/1.0" 202 Accepted 2026-03-31 00:35:59,859 - lava_callback - INFO - Starting async job submit for node 69c873edf41e4fb6a37855ab 2026-03-31 00:35:59,860 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:35:59,878 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c873edf41e4fb6a37855ab HTTP/1.1" 200 None 2026-03-31 00:35:59,879 - lava_callback - DEBUG - Processing node 69c873edf41e4fb6a37855ab: baseline-arm-baylibre 2026-03-31 00:35:59,879 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c873edf41e4fb6a37855ab 2026-03-31 00:35:59,880 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:35:59,893 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c873edf41e4fb6a37855ab HTTP/1.1" 200 None 2026-03-31 00:35:59,895 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:35:59,908 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-03-31 00:35:59,909 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:35:59,934 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c873edf41e4fb6a37855ab HTTP/1.1" 200 None 2026-03-31 00:35:59,939 - lava_callback - ERROR - Error processing callback for node 69c873edf41e4fb6a37855ab: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:56428 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43538 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43548 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42534 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41674 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41678 - "GET / HTTP/1.0" 200 OK 2026-03-31 00:36:39,888 - lava_callback - INFO - Received callback for node 69c87412f41e4fb6a37855b8 INFO: 172.20.0.1:35182 - "POST /node/69c87412f41e4fb6a37855b8 HTTP/1.0" 202 Accepted 2026-03-31 00:36:39,894 - lava_callback - INFO - Starting async job submit for node 69c87412f41e4fb6a37855b8 2026-03-31 00:36:39,895 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:36:39,910 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c87412f41e4fb6a37855b8 HTTP/1.1" 200 None 2026-03-31 00:36:39,911 - lava_callback - DEBUG - Processing node 69c87412f41e4fb6a37855b8: baseline-arm-baylibre 2026-03-31 00:36:39,911 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c87412f41e4fb6a37855b8 2026-03-31 00:36:39,911 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:36:39,924 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c87412f41e4fb6a37855b8 HTTP/1.1" 200 None 2026-03-31 00:36:39,926 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:36:39,944 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-03-31 00:36:39,946 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:36:39,966 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c87412f41e4fb6a37855b8 HTTP/1.1" 200 None 2026-03-31 00:36:39,967 - lava_callback - ERROR - Error processing callback for node 69c87412f41e4fb6a37855b8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:35184 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50968 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43288 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33202 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33216 - "GET / HTTP/1.0" 200 OK 2026-03-31 00:37:25,983 - lava_callback - INFO - Received callback for node 69c8743bf41e4fb6a37855cc INFO: 172.20.0.1:49660 - "POST /node/69c8743bf41e4fb6a37855cc HTTP/1.0" 202 Accepted 2026-03-31 00:37:25,992 - lava_callback - INFO - Starting async job submit for node 69c8743bf41e4fb6a37855cc 2026-03-31 00:37:25,993 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:37:26,019 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8743bf41e4fb6a37855cc HTTP/1.1" 200 None 2026-03-31 00:37:26,020 - lava_callback - DEBUG - Processing node 69c8743bf41e4fb6a37855cc: baseline-arm-clabbe 2026-03-31 00:37:26,020 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8743bf41e4fb6a37855cc 2026-03-31 00:37:26,021 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:37:26,046 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8743bf41e4fb6a37855cc HTTP/1.1" 200 None 2026-03-31 00:37:26,048 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:37:26,092 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-03-31 00:37:26,094 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:37:26,159 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8743bf41e4fb6a37855cc HTTP/1.1" 200 None 2026-03-31 00:37:26,161 - lava_callback - ERROR - Error processing callback for node 69c8743bf41e4fb6a37855cc: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:49670 - "GET / HTTP/1.0" 200 OK 2026-03-31 00:37:28,431 - lava_callback - INFO - Received callback for node 69cb17328c06c2799bf55087 INFO: 172.20.0.1:49674 - "POST /node/69cb17328c06c2799bf55087 HTTP/1.0" 202 Accepted 2026-03-31 00:37:28,440 - lava_callback - INFO - Starting async job submit for node 69cb17328c06c2799bf55087 2026-03-31 00:37:28,477 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:37:28,513 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb17328c06c2799bf55087 HTTP/1.1" 200 None 2026-03-31 00:37:28,515 - lava_callback - DEBUG - Processing node 69cb17328c06c2799bf55087: baseline-x86 2026-03-31 00:37:28,815 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 00:37:29,108 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 00:37:29,111 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69cb17328c06c2799bf55087/log.txt.gz 2026-03-31 00:37:29,304 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 00:37:29,554 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 00:37:29,557 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69cb17328c06c2799bf55087/lava-logs.yaml 2026-03-31 00:37:29,564 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 00:37:29,971 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 00:37:29,974 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69cb17328c06c2799bf55087/lava_callback.json.gz 2026-03-31 00:37:30,003 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:37:30,020 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb17328c06c2799bf55087 HTTP/1.1" 200 None 2026-03-31 00:37:30,022 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:37:30,036 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12fb8c06c2799bf55043 HTTP/1.1" 200 None 2026-03-31 00:37:30,038 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:37:30,077 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb17328c06c2799bf55087 HTTP/1.1" 200 None 2026-03-31 00:37:30,079 - lava_callback - INFO - Completed processing callback for node 69cb17328c06c2799bf55087 2026-03-31 00:37:30,080 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:37:30,148 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:42968 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42984 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59886 - "GET / HTTP/1.0" 200 OK 2026-03-31 00:37:47,428 - lava_callback - INFO - Received callback for node 69cb17348c06c2799bf55090 INFO: 172.20.0.1:59900 - "POST /node/69cb17348c06c2799bf55090 HTTP/1.0" 202 Accepted 2026-03-31 00:37:47,434 - lava_callback - INFO - Starting async job submit for node 69cb17348c06c2799bf55090 2026-03-31 00:37:47,452 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:37:47,467 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb17348c06c2799bf55090 HTTP/1.1" 200 None 2026-03-31 00:37:47,468 - lava_callback - DEBUG - Processing node 69cb17348c06c2799bf55090: baseline-x86 2026-03-31 00:37:47,647 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 00:37:47,758 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 00:37:47,760 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69cb17348c06c2799bf55090/log.txt.gz 2026-03-31 00:37:47,864 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 00:37:48,057 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 00:37:48,059 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69cb17348c06c2799bf55090/lava-logs.yaml 2026-03-31 00:37:48,064 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 00:37:48,222 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 00:37:48,223 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69cb17348c06c2799bf55090/lava_callback.json.gz 2026-03-31 00:37:48,243 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:37:48,259 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb17348c06c2799bf55090 HTTP/1.1" 200 None 2026-03-31 00:37:48,260 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:37:48,280 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12fb8c06c2799bf55043 HTTP/1.1" 200 None 2026-03-31 00:37:48,282 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:37:48,313 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb17348c06c2799bf55090 HTTP/1.1" 200 None 2026-03-31 00:37:48,315 - lava_callback - INFO - Completed processing callback for node 69cb17348c06c2799bf55090 INFO: 172.20.0.1:48498 - "GET / HTTP/1.0" 200 OK 2026-03-31 00:37:59,986 - lava_callback - INFO - Received callback for node 69c8745cf41e4fb6a37855dc INFO: 172.20.0.1:55246 - "POST /node/69c8745cf41e4fb6a37855dc HTTP/1.0" 202 Accepted 2026-03-31 00:37:59,992 - lava_callback - INFO - Starting async job submit for node 69c8745cf41e4fb6a37855dc 2026-03-31 00:37:59,992 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:38:00,008 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8745cf41e4fb6a37855dc HTTP/1.1" 200 None 2026-03-31 00:38:00,009 - lava_callback - DEBUG - Processing node 69c8745cf41e4fb6a37855dc: baseline-arm-baylibre 2026-03-31 00:38:00,009 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8745cf41e4fb6a37855dc 2026-03-31 00:38:00,010 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:38:00,025 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8745cf41e4fb6a37855dc HTTP/1.1" 200 None 2026-03-31 00:38:00,027 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:38:00,049 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-03-31 00:38:00,051 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:38:00,071 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8745cf41e4fb6a37855dc HTTP/1.1" 200 None 2026-03-31 00:38:00,074 - lava_callback - ERROR - Error processing callback for node 69c8745cf41e4fb6a37855dc: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-31 00:38:00,151 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:38:00,181 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:55258 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35786 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35792 - "GET / HTTP/1.0" 200 OK 2026-03-31 00:38:21,292 - lava_callback - INFO - Received callback for node 69cb17348c06c2799bf5508d INFO: 172.20.0.1:57136 - "POST /node/69cb17348c06c2799bf5508d HTTP/1.0" 202 Accepted 2026-03-31 00:38:21,298 - lava_callback - INFO - Starting async job submit for node 69cb17348c06c2799bf5508d 2026-03-31 00:38:21,316 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:38:21,330 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb17348c06c2799bf5508d HTTP/1.1" 200 None 2026-03-31 00:38:21,331 - lava_callback - DEBUG - Processing node 69cb17348c06c2799bf5508d: baseline-x86 2026-03-31 00:38:21,649 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 00:38:22,111 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 00:38:22,114 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69cb17348c06c2799bf5508d/log.txt.gz 2026-03-31 00:38:22,471 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 00:38:22,980 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 00:38:22,983 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69cb17348c06c2799bf5508d/lava-logs.yaml 2026-03-31 00:38:22,993 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 00:38:23,349 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 00:38:23,352 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69cb17348c06c2799bf5508d/lava_callback.json.gz 2026-03-31 00:38:23,385 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:38:23,412 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb17348c06c2799bf5508d HTTP/1.1" 200 None 2026-03-31 00:38:23,414 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:38:23,439 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12fb8c06c2799bf55043 HTTP/1.1" 200 None 2026-03-31 00:38:23,441 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:38:23,497 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb17348c06c2799bf5508d HTTP/1.1" 200 None 2026-03-31 00:38:23,499 - lava_callback - INFO - Completed processing callback for node 69cb17348c06c2799bf5508d INFO: 172.20.0.1:57152 - "GET / HTTP/1.0" 200 OK 2026-03-31 00:38:30,184 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:38:30,222 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:32930 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32940 - "GET / HTTP/1.0" 200 OK 2026-03-31 00:38:46,181 - lava_callback - INFO - Received callback for node 69c87483f41e4fb6a37855e8 INFO: 172.20.0.1:60138 - "POST /node/69c87483f41e4fb6a37855e8 HTTP/1.0" 202 Accepted 2026-03-31 00:38:46,219 - lava_callback - INFO - Starting async job submit for node 69c87483f41e4fb6a37855e8 2026-03-31 00:38:46,220 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:38:46,253 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c87483f41e4fb6a37855e8 HTTP/1.1" 200 None 2026-03-31 00:38:46,255 - lava_callback - DEBUG - Processing node 69c87483f41e4fb6a37855e8: baseline-arm-clabbe 2026-03-31 00:38:46,255 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c87483f41e4fb6a37855e8 2026-03-31 00:38:46,256 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:38:46,285 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c87483f41e4fb6a37855e8 HTTP/1.1" 200 None 2026-03-31 00:38:46,287 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:38:46,310 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-03-31 00:38:46,313 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:38:46,364 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c87483f41e4fb6a37855e8 HTTP/1.1" 200 None 2026-03-31 00:38:46,365 - lava_callback - ERROR - Error processing callback for node 69c87483f41e4fb6a37855e8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:60148 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49036 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55052 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50368 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50378 - "GET / HTTP/1.0" 200 OK 2026-03-31 00:39:20,096 - lava_callback - INFO - Received callback for node 69c874a6f41e4fb6a37855f8 INFO: 172.20.0.1:55754 - "POST /node/69c874a6f41e4fb6a37855f8 HTTP/1.0" 202 Accepted 2026-03-31 00:39:20,104 - lava_callback - INFO - Starting async job submit for node 69c874a6f41e4fb6a37855f8 2026-03-31 00:39:20,108 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:39:20,147 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c874a6f41e4fb6a37855f8 HTTP/1.1" 200 None 2026-03-31 00:39:20,148 - lava_callback - DEBUG - Processing node 69c874a6f41e4fb6a37855f8: baseline-arm-baylibre 2026-03-31 00:39:20,148 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c874a6f41e4fb6a37855f8 2026-03-31 00:39:20,149 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:39:20,184 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c874a6f41e4fb6a37855f8 HTTP/1.1" 200 None 2026-03-31 00:39:20,186 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:39:20,240 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-03-31 00:39:20,243 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:39:20,284 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c874a6f41e4fb6a37855f8 HTTP/1.1" 200 None 2026-03-31 00:39:20,286 - lava_callback - ERROR - Error processing callback for node 69c874a6f41e4fb6a37855f8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:55768 - "GET / HTTP/1.0" 200 OK 2026-03-31 00:39:32,853 - lava_callback - INFO - Received callback for node 69cb174f8c06c2799bf550b8 INFO: 172.20.0.1:49226 - "POST /node/69cb174f8c06c2799bf550b8 HTTP/1.0" 202 Accepted 2026-03-31 00:39:32,859 - lava_callback - INFO - Starting async job submit for node 69cb174f8c06c2799bf550b8 2026-03-31 00:39:32,883 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:39:32,896 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb174f8c06c2799bf550b8 HTTP/1.1" 200 None 2026-03-31 00:39:32,897 - lava_callback - DEBUG - Processing node 69cb174f8c06c2799bf550b8: baseline-arm 2026-03-31 00:39:33,087 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 00:39:33,415 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 00:39:33,416 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69cb174f8c06c2799bf550b8/log.txt.gz 2026-03-31 00:39:33,524 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 00:39:33,730 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 00:39:33,731 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69cb174f8c06c2799bf550b8/lava-logs.yaml 2026-03-31 00:39:33,736 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 00:39:33,913 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 00:39:33,915 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69cb174f8c06c2799bf550b8/lava_callback.json.gz 2026-03-31 00:39:33,941 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:39:33,955 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb174f8c06c2799bf550b8 HTTP/1.1" 200 None 2026-03-31 00:39:33,956 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:39:33,979 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12d38c06c2799bf55022 HTTP/1.1" 200 None 2026-03-31 00:39:33,981 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:39:34,021 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb174f8c06c2799bf550b8 HTTP/1.1" 200 None 2026-03-31 00:39:34,022 - lava_callback - INFO - Completed processing callback for node 69cb174f8c06c2799bf550b8 INFO: 172.20.0.1:49238 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49252 - "GET / HTTP/1.0" 200 OK 2026-03-31 00:39:40,051 - lava_callback - INFO - Received callback for node 69c874c2f41e4fb6a3785620 INFO: 172.20.0.1:41158 - "POST /node/69c874c2f41e4fb6a3785620 HTTP/1.0" 202 Accepted 2026-03-31 00:39:40,056 - lava_callback - INFO - Starting async job submit for node 69c874c2f41e4fb6a3785620 2026-03-31 00:39:40,057 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:39:40,072 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c874c2f41e4fb6a3785620 HTTP/1.1" 200 None 2026-03-31 00:39:40,073 - lava_callback - DEBUG - Processing node 69c874c2f41e4fb6a3785620: baseline-arm-baylibre 2026-03-31 00:39:40,073 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c874c2f41e4fb6a3785620 2026-03-31 00:39:40,074 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:39:40,087 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c874c2f41e4fb6a3785620 HTTP/1.1" 200 None 2026-03-31 00:39:40,088 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:39:40,103 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f72f41e4fb6a3782f3e HTTP/1.1" 200 None 2026-03-31 00:39:40,105 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:39:40,138 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c874c2f41e4fb6a3785620 HTTP/1.1" 200 None 2026-03-31 00:39:40,140 - lava_callback - ERROR - Error processing callback for node 69c874c2f41e4fb6a3785620: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-31 00:39:40,727 - lava_callback - INFO - Received callback for node 69c874c4f41e4fb6a378562b INFO: 172.20.0.1:41174 - "POST /node/69c874c4f41e4fb6a378562b HTTP/1.0" 202 Accepted 2026-03-31 00:39:40,732 - lava_callback - INFO - Starting async job submit for node 69c874c4f41e4fb6a378562b 2026-03-31 00:39:40,733 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:39:40,747 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c874c4f41e4fb6a378562b HTTP/1.1" 200 None 2026-03-31 00:39:40,748 - lava_callback - DEBUG - Processing node 69c874c4f41e4fb6a378562b: baseline-arm-baylibre 2026-03-31 00:39:40,749 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c874c4f41e4fb6a378562b 2026-03-31 00:39:40,749 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:39:40,762 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c874c4f41e4fb6a378562b HTTP/1.1" 200 None 2026-03-31 00:39:40,764 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:39:40,777 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f72f41e4fb6a3782f3e HTTP/1.1" 200 None 2026-03-31 00:39:40,778 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:39:40,799 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c874c4f41e4fb6a378562b HTTP/1.1" 200 None 2026-03-31 00:39:40,800 - lava_callback - ERROR - Error processing callback for node 69c874c4f41e4fb6a378562b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-31 00:39:41,394 - lava_callback - INFO - Received callback for node 69c874c7f41e4fb6a378562e INFO: 172.20.0.1:41184 - "POST /node/69c874c7f41e4fb6a378562e HTTP/1.0" 202 Accepted 2026-03-31 00:39:41,399 - lava_callback - INFO - Starting async job submit for node 69c874c7f41e4fb6a378562e 2026-03-31 00:39:41,400 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:39:41,415 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c874c7f41e4fb6a378562e HTTP/1.1" 200 None 2026-03-31 00:39:41,415 - lava_callback - DEBUG - Processing node 69c874c7f41e4fb6a378562e: baseline-arm-baylibre 2026-03-31 00:39:41,416 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c874c7f41e4fb6a378562e 2026-03-31 00:39:41,416 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:39:41,433 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c874c7f41e4fb6a378562e HTTP/1.1" 200 None 2026-03-31 00:39:41,434 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:39:41,454 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f72f41e4fb6a3782f3e HTTP/1.1" 200 None 2026-03-31 00:39:41,455 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:39:41,476 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c874c7f41e4fb6a378562e HTTP/1.1" 200 None 2026-03-31 00:39:41,480 - lava_callback - ERROR - Error processing callback for node 69c874c7f41e4fb6a378562e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-31 00:39:46,019 - lava_callback - INFO - Received callback for node 69c874cbf41e4fb6a3785639 INFO: 172.20.0.1:41188 - "POST /node/69c874cbf41e4fb6a3785639 HTTP/1.0" 202 Accepted 2026-03-31 00:39:46,025 - lava_callback - INFO - Starting async job submit for node 69c874cbf41e4fb6a3785639 2026-03-31 00:39:46,025 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:39:46,040 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c874cbf41e4fb6a3785639 HTTP/1.1" 200 None 2026-03-31 00:39:46,041 - lava_callback - DEBUG - Processing node 69c874cbf41e4fb6a3785639: baseline-arm-clabbe 2026-03-31 00:39:46,041 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c874cbf41e4fb6a3785639 2026-03-31 00:39:46,041 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:39:46,055 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c874cbf41e4fb6a3785639 HTTP/1.1" 200 None 2026-03-31 00:39:46,056 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:39:46,070 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-03-31 00:39:46,072 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:39:46,093 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c874cbf41e4fb6a3785639 HTTP/1.1" 200 None 2026-03-31 00:39:46,094 - lava_callback - ERROR - Error processing callback for node 69c874cbf41e4fb6a3785639: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:41200 - "GET / HTTP/1.0" 200 OK 2026-03-31 00:39:46,697 - lava_callback - INFO - Received callback for node 69c874cef41e4fb6a3785648 INFO: 172.20.0.1:41204 - "POST /node/69c874cef41e4fb6a3785648 HTTP/1.0" 202 Accepted 2026-03-31 00:39:46,703 - lava_callback - INFO - Starting async job submit for node 69c874cef41e4fb6a3785648 2026-03-31 00:39:46,704 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:39:46,724 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c874cef41e4fb6a3785648 HTTP/1.1" 200 None 2026-03-31 00:39:46,725 - lava_callback - DEBUG - Processing node 69c874cef41e4fb6a3785648: baseline-arm-clabbe 2026-03-31 00:39:46,725 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c874cef41e4fb6a3785648 2026-03-31 00:39:46,726 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:39:46,740 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c874cef41e4fb6a3785648 HTTP/1.1" 200 None 2026-03-31 00:39:46,742 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:39:46,754 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f72f41e4fb6a3782f3e HTTP/1.1" 200 None 2026-03-31 00:39:46,756 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:39:46,776 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c874cef41e4fb6a3785648 HTTP/1.1" 200 None 2026-03-31 00:39:46,778 - lava_callback - ERROR - Error processing callback for node 69c874cef41e4fb6a3785648: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:49724 - "GET / HTTP/1.0" 200 OK 2026-03-31 00:40:00,225 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:40:00,243 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-31 00:40:06,042 - lava_callback - INFO - Received callback for node 69c874d1f41e4fb6a3785657 INFO: 172.20.0.1:54340 - "POST /node/69c874d1f41e4fb6a3785657 HTTP/1.0" 202 Accepted 2026-03-31 00:40:06,048 - lava_callback - INFO - Starting async job submit for node 69c874d1f41e4fb6a3785657 2026-03-31 00:40:06,049 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:40:06,063 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c874d1f41e4fb6a3785657 HTTP/1.1" 200 None 2026-03-31 00:40:06,064 - lava_callback - DEBUG - Processing node 69c874d1f41e4fb6a3785657: baseline-arm-clabbe 2026-03-31 00:40:06,064 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c874d1f41e4fb6a3785657 2026-03-31 00:40:06,065 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:40:06,079 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c874d1f41e4fb6a3785657 HTTP/1.1" 200 None 2026-03-31 00:40:06,080 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:40:06,093 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f72f41e4fb6a3782f3e HTTP/1.1" 200 None 2026-03-31 00:40:06,095 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:40:06,132 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c874d1f41e4fb6a3785657 HTTP/1.1" 200 None 2026-03-31 00:40:06,134 - lava_callback - ERROR - Error processing callback for node 69c874d1f41e4fb6a3785657: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:54344 - "GET / HTTP/1.0" 200 OK 2026-03-31 00:40:06,776 - lava_callback - INFO - Received callback for node 69c874d2f41e4fb6a3785681 INFO: 172.20.0.1:54346 - "POST /node/69c874d2f41e4fb6a3785681 HTTP/1.0" 202 Accepted 2026-03-31 00:40:06,785 - lava_callback - INFO - Starting async job submit for node 69c874d2f41e4fb6a3785681 2026-03-31 00:40:06,786 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:40:06,800 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c874d2f41e4fb6a3785681 HTTP/1.1" 200 None 2026-03-31 00:40:06,801 - lava_callback - DEBUG - Processing node 69c874d2f41e4fb6a3785681: baseline-arm-clabbe 2026-03-31 00:40:06,801 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c874d2f41e4fb6a3785681 2026-03-31 00:40:06,802 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:40:06,819 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c874d2f41e4fb6a3785681 HTTP/1.1" 200 None 2026-03-31 00:40:06,820 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:40:06,853 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f72f41e4fb6a3782f3e HTTP/1.1" 200 None 2026-03-31 00:40:06,855 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:40:06,878 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c874d2f41e4fb6a3785681 HTTP/1.1" 200 None 2026-03-31 00:40:06,880 - lava_callback - ERROR - Error processing callback for node 69c874d2f41e4fb6a3785681: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-31 00:40:07,475 - lava_callback - INFO - Received callback for node 69c874d3f41e4fb6a3785684 INFO: 172.20.0.1:54358 - "POST /node/69c874d3f41e4fb6a3785684 HTTP/1.0" 202 Accepted 2026-03-31 00:40:07,481 - lava_callback - INFO - Starting async job submit for node 69c874d3f41e4fb6a3785684 2026-03-31 00:40:07,482 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:40:07,496 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c874d3f41e4fb6a3785684 HTTP/1.1" 200 None 2026-03-31 00:40:07,497 - lava_callback - DEBUG - Processing node 69c874d3f41e4fb6a3785684: baseline-arm-clabbe 2026-03-31 00:40:07,497 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c874d3f41e4fb6a3785684 2026-03-31 00:40:07,498 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:40:07,520 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c874d3f41e4fb6a3785684 HTTP/1.1" 200 None 2026-03-31 00:40:07,522 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:40:07,542 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f72f41e4fb6a3782f3e HTTP/1.1" 200 None 2026-03-31 00:40:07,544 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:40:07,573 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c874d3f41e4fb6a3785684 HTTP/1.1" 200 None 2026-03-31 00:40:07,574 - lava_callback - ERROR - Error processing callback for node 69c874d3f41e4fb6a3785684: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-31 00:40:08,149 - lava_callback - INFO - Received callback for node 69c874d6f41e4fb6a378568e INFO: 172.20.0.1:54362 - "POST /node/69c874d6f41e4fb6a378568e HTTP/1.0" 202 Accepted 2026-03-31 00:40:08,154 - lava_callback - INFO - Starting async job submit for node 69c874d6f41e4fb6a378568e 2026-03-31 00:40:08,155 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:40:08,175 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c874d6f41e4fb6a378568e HTTP/1.1" 200 None 2026-03-31 00:40:08,176 - lava_callback - DEBUG - Processing node 69c874d6f41e4fb6a378568e: baseline-arm-clabbe 2026-03-31 00:40:08,176 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c874d6f41e4fb6a378568e 2026-03-31 00:40:08,177 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:40:08,198 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c874d6f41e4fb6a378568e HTTP/1.1" 200 None 2026-03-31 00:40:08,199 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:40:08,217 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f72f41e4fb6a3782f3e HTTP/1.1" 200 None 2026-03-31 00:40:08,219 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:40:08,238 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c874d6f41e4fb6a378568e HTTP/1.1" 200 None 2026-03-31 00:40:08,244 - lava_callback - ERROR - Error processing callback for node 69c874d6f41e4fb6a378568e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:51854 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51868 - "GET / HTTP/1.0" 200 OK 2026-03-31 00:40:26,038 - lava_callback - INFO - Received callback for node 69c874f0f41e4fb6a37856b7 INFO: 172.20.0.1:40440 - "POST /node/69c874f0f41e4fb6a37856b7 HTTP/1.0" 202 Accepted 2026-03-31 00:40:26,044 - lava_callback - INFO - Starting async job submit for node 69c874f0f41e4fb6a37856b7 2026-03-31 00:40:26,044 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:40:26,059 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c874f0f41e4fb6a37856b7 HTTP/1.1" 200 None 2026-03-31 00:40:26,060 - lava_callback - DEBUG - Processing node 69c874f0f41e4fb6a37856b7: baseline-arm-clabbe 2026-03-31 00:40:26,060 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c874f0f41e4fb6a37856b7 2026-03-31 00:40:26,061 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:40:26,075 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c874f0f41e4fb6a37856b7 HTTP/1.1" 200 None 2026-03-31 00:40:26,077 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:40:26,096 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-03-31 00:40:26,098 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:40:26,118 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c874f0f41e4fb6a37856b7 HTTP/1.1" 200 None 2026-03-31 00:40:26,119 - lava_callback - ERROR - Error processing callback for node 69c874f0f41e4fb6a37856b7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:40446 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33114 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33124 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37360 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56984 - "GET / HTTP/1.0" 200 OK 2026-03-31 00:41:06,102 - lava_callback - INFO - Received callback for node 69c87515f41e4fb6a3785727 INFO: 172.20.0.1:35146 - "POST /node/69c87515f41e4fb6a3785727 HTTP/1.0" 202 Accepted 2026-03-31 00:41:06,107 - lava_callback - INFO - Starting async job submit for node 69c87515f41e4fb6a3785727 2026-03-31 00:41:06,108 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:41:06,123 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c87515f41e4fb6a3785727 HTTP/1.1" 200 None 2026-03-31 00:41:06,124 - lava_callback - DEBUG - Processing node 69c87515f41e4fb6a3785727: baseline-arm-clabbe 2026-03-31 00:41:06,124 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c87515f41e4fb6a3785727 2026-03-31 00:41:06,125 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:41:06,140 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c87515f41e4fb6a3785727 HTTP/1.1" 200 None 2026-03-31 00:41:06,141 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:41:06,163 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-03-31 00:41:06,165 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:41:06,196 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c87515f41e4fb6a3785727 HTTP/1.1" 200 None 2026-03-31 00:41:06,197 - lava_callback - ERROR - Error processing callback for node 69c87515f41e4fb6a3785727: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:35150 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48068 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48074 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49526 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53200 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53202 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51300 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38082 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40540 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34616 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34620 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45418 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47946 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47962 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53156 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57016 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40094 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35898 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35910 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38692 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57330 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57344 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47984 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60636 - "GET / HTTP/1.0" 200 OK 2026-03-31 00:44:00,350 - lava_callback - INFO - Received callback for node 69c875bff41e4fb6a3785783 INFO: 172.20.0.1:53568 - "POST /node/69c875bff41e4fb6a3785783 HTTP/1.0" 202 Accepted 2026-03-31 00:44:00,355 - lava_callback - INFO - Starting async job submit for node 69c875bff41e4fb6a3785783 2026-03-31 00:44:00,356 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:44:00,372 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c875bff41e4fb6a3785783 HTTP/1.1" 200 None 2026-03-31 00:44:00,372 - lava_callback - DEBUG - Processing node 69c875bff41e4fb6a3785783: baseline-arm-baylibre 2026-03-31 00:44:00,373 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c875bff41e4fb6a3785783 2026-03-31 00:44:00,373 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:44:00,387 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c875bff41e4fb6a3785783 HTTP/1.1" 200 None 2026-03-31 00:44:00,388 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:44:00,401 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3293e18a3717aa65d01ad HTTP/1.1" 200 None 2026-03-31 00:44:00,403 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:44:00,423 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c875bff41e4fb6a3785783 HTTP/1.1" 200 None 2026-03-31 00:44:00,425 - lava_callback - ERROR - Error processing callback for node 69c875bff41e4fb6a3785783: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:53576 - "GET / HTTP/1.0" 200 OK 2026-03-31 00:44:07,985 - lava_callback - INFO - Received callback for node 69cb17338c06c2799bf5508a INFO: 172.20.0.1:53590 - "POST /node/69cb17338c06c2799bf5508a HTTP/1.0" 202 Accepted 2026-03-31 00:44:07,991 - lava_callback - INFO - Starting async job submit for node 69cb17338c06c2799bf5508a 2026-03-31 00:44:08,015 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:44:08,028 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb17338c06c2799bf5508a HTTP/1.1" 200 None 2026-03-31 00:44:08,029 - lava_callback - DEBUG - Processing node 69cb17338c06c2799bf5508a: baseline-x86 2026-03-31 00:44:08,343 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 00:44:08,498 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 00:44:08,500 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69cb17338c06c2799bf5508a/log.txt.gz 2026-03-31 00:44:08,708 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 00:44:08,942 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 00:44:08,944 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69cb17338c06c2799bf5508a/lava-logs.yaml 2026-03-31 00:44:08,957 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 00:44:09,283 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 00:44:09,285 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69cb17338c06c2799bf5508a/lava_callback.json.gz 2026-03-31 00:44:09,309 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:44:09,329 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb17338c06c2799bf5508a HTTP/1.1" 200 None 2026-03-31 00:44:09,330 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:44:09,347 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12fb8c06c2799bf55043 HTTP/1.1" 200 None 2026-03-31 00:44:09,348 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:44:09,372 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb17338c06c2799bf5508a HTTP/1.1" 200 None 2026-03-31 00:44:09,377 - lava_callback - INFO - Completed processing callback for node 69cb17338c06c2799bf5508a INFO: 172.20.0.1:38102 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38116 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39588 - "GET / HTTP/1.0" 200 OK 2026-03-31 00:44:30,246 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:44:30,264 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:36626 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36630 - "GET / HTTP/1.0" 200 OK 2026-03-31 00:44:40,342 - lava_callback - INFO - Received callback for node 69c875e6f41e4fb6a37857a1 INFO: 172.20.0.1:60082 - "POST /node/69c875e6f41e4fb6a37857a1 HTTP/1.0" 202 Accepted 2026-03-31 00:44:40,348 - lava_callback - INFO - Starting async job submit for node 69c875e6f41e4fb6a37857a1 2026-03-31 00:44:40,348 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:44:40,362 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c875e6f41e4fb6a37857a1 HTTP/1.1" 200 None 2026-03-31 00:44:40,363 - lava_callback - DEBUG - Processing node 69c875e6f41e4fb6a37857a1: baseline-arm-baylibre 2026-03-31 00:44:40,363 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c875e6f41e4fb6a37857a1 2026-03-31 00:44:40,364 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:44:40,377 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c875e6f41e4fb6a37857a1 HTTP/1.1" 200 None 2026-03-31 00:44:40,379 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:44:40,392 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3293e18a3717aa65d01ad HTTP/1.1" 200 None 2026-03-31 00:44:40,393 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:44:40,423 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c875e6f41e4fb6a37857a1 HTTP/1.1" 200 None 2026-03-31 00:44:40,424 - lava_callback - ERROR - Error processing callback for node 69c875e6f41e4fb6a37857a1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:60088 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52700 - "GET / HTTP/1.0" 200 OK 2026-03-31 00:45:06,338 - lava_callback - INFO - Received callback for node 69c8760df41e4fb6a37857cd INFO: 172.20.0.1:39330 - "POST /node/69c8760df41e4fb6a37857cd HTTP/1.0" 202 Accepted 2026-03-31 00:45:06,343 - lava_callback - INFO - Starting async job submit for node 69c8760df41e4fb6a37857cd 2026-03-31 00:45:06,344 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:45:06,358 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8760df41e4fb6a37857cd HTTP/1.1" 200 None 2026-03-31 00:45:06,359 - lava_callback - DEBUG - Processing node 69c8760df41e4fb6a37857cd: baseline-arm-clabbe 2026-03-31 00:45:06,359 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8760df41e4fb6a37857cd 2026-03-31 00:45:06,359 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:45:06,372 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8760df41e4fb6a37857cd HTTP/1.1" 200 None 2026-03-31 00:45:06,374 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:45:06,387 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3293e18a3717aa65d01ad HTTP/1.1" 200 None 2026-03-31 00:45:06,389 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:45:06,408 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8760df41e4fb6a37857cd HTTP/1.1" 200 None 2026-03-31 00:45:06,415 - lava_callback - ERROR - Error processing callback for node 69c8760df41e4fb6a37857cd: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:39334 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37640 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37646 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57446 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44934 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44938 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52072 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39862 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39800 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39830 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39838 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35342 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43680 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43696 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32770 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46260 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52318 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57640 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57642 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49834 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55736 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55742 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45862 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60296 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38824 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46732 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46740 - "GET / HTTP/1.0" 200 OK 2026-03-31 00:48:26,394 - lava_callback - INFO - Received callback for node 69c876c4f41e4fb6a3785832 INFO: 172.20.0.1:51132 - "POST /node/69c876c4f41e4fb6a3785832 HTTP/1.0" 202 Accepted 2026-03-31 00:48:26,400 - lava_callback - INFO - Starting async job submit for node 69c876c4f41e4fb6a3785832 2026-03-31 00:48:26,401 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:48:26,415 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c876c4f41e4fb6a3785832 HTTP/1.1" 200 None 2026-03-31 00:48:26,416 - lava_callback - DEBUG - Processing node 69c876c4f41e4fb6a3785832: baseline-arm-clabbe 2026-03-31 00:48:26,416 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c876c4f41e4fb6a3785832 2026-03-31 00:48:26,417 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:48:26,438 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c876c4f41e4fb6a3785832 HTTP/1.1" 200 None 2026-03-31 00:48:26,439 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:48:26,453 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3293e18a3717aa65d01ad HTTP/1.1" 200 None 2026-03-31 00:48:26,454 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:48:26,476 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c876c4f41e4fb6a3785832 HTTP/1.1" 200 None 2026-03-31 00:48:26,478 - lava_callback - ERROR - Error processing callback for node 69c876c4f41e4fb6a3785832: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:51136 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53656 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53672 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57136 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59858 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38720 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58188 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58204 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50760 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33784 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51364 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51378 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48498 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38732 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47520 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47528 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34730 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36176 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46716 - "GET / HTTP/1.0" 200 OK 2026-03-31 00:50:46,476 - lava_callback - INFO - Received callback for node 69c87756f41e4fb6a3785885 INFO: 172.20.0.1:46730 - "POST /node/69c87756f41e4fb6a3785885 HTTP/1.0" 202 Accepted 2026-03-31 00:50:46,481 - lava_callback - INFO - Starting async job submit for node 69c87756f41e4fb6a3785885 2026-03-31 00:50:46,482 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:50:46,503 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c87756f41e4fb6a3785885 HTTP/1.1" 200 None 2026-03-31 00:50:46,504 - lava_callback - DEBUG - Processing node 69c87756f41e4fb6a3785885: baseline-arm-clabbe 2026-03-31 00:50:46,504 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c87756f41e4fb6a3785885 2026-03-31 00:50:46,505 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:50:46,518 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c87756f41e4fb6a3785885 HTTP/1.1" 200 None 2026-03-31 00:50:46,520 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.20.0.1:46736 - "GET / HTTP/1.0" 200 OK 2026-03-31 00:50:46,533 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3293e18a3717aa65d01ad HTTP/1.1" 200 None 2026-03-31 00:50:46,534 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:50:46,555 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c87756f41e4fb6a3785885 HTTP/1.1" 200 None 2026-03-31 00:50:46,556 - lava_callback - ERROR - Error processing callback for node 69c87756f41e4fb6a3785885: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:40572 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46882 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47906 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47914 - "GET / HTTP/1.0" 200 OK 2026-03-31 00:51:26,460 - lava_callback - INFO - Received callback for node 69c8777cf41e4fb6a3785894 INFO: 172.20.0.1:51224 - "POST /node/69c8777cf41e4fb6a3785894 HTTP/1.0" 202 Accepted 2026-03-31 00:51:26,466 - lava_callback - INFO - Starting async job submit for node 69c8777cf41e4fb6a3785894 2026-03-31 00:51:26,467 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:51:26,495 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8777cf41e4fb6a3785894 HTTP/1.1" 200 None 2026-03-31 00:51:26,496 - lava_callback - DEBUG - Processing node 69c8777cf41e4fb6a3785894: baseline-arm-clabbe 2026-03-31 00:51:26,496 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8777cf41e4fb6a3785894 2026-03-31 00:51:26,497 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:51:26,510 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8777cf41e4fb6a3785894 HTTP/1.1" 200 None 2026-03-31 00:51:26,511 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.20.0.1:51232 - "GET / HTTP/1.0" 200 OK 2026-03-31 00:51:26,524 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3293e18a3717aa65d01ad HTTP/1.1" 200 None 2026-03-31 00:51:26,525 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:51:26,545 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8777cf41e4fb6a3785894 HTTP/1.1" 200 None 2026-03-31 00:51:26,547 - lava_callback - ERROR - Error processing callback for node 69c8777cf41e4fb6a3785894: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:46086 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40860 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40870 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34308 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57752 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39134 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39138 - "GET / HTTP/1.0" 200 OK 2026-03-31 00:52:26,515 - lava_callback - INFO - Received callback for node 69c877c7f41e4fb6a37858b8 INFO: 172.20.0.1:48386 - "POST /node/69c877c7f41e4fb6a37858b8 HTTP/1.0" 202 Accepted 2026-03-31 00:52:26,523 - lava_callback - INFO - Starting async job submit for node 69c877c7f41e4fb6a37858b8 INFO: 172.20.0.1:48394 - "GET / HTTP/1.0" 200 OK 2026-03-31 00:52:26,525 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:52:26,541 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c877c7f41e4fb6a37858b8 HTTP/1.1" 200 None 2026-03-31 00:52:26,542 - lava_callback - DEBUG - Processing node 69c877c7f41e4fb6a37858b8: baseline-arm-clabbe 2026-03-31 00:52:26,542 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c877c7f41e4fb6a37858b8 2026-03-31 00:52:26,543 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:52:26,571 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c877c7f41e4fb6a37858b8 HTTP/1.1" 200 None 2026-03-31 00:52:26,573 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:52:26,589 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3293e18a3717aa65d01ad HTTP/1.1" 200 None 2026-03-31 00:52:26,591 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:52:26,616 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c877c7f41e4fb6a37858b8 HTTP/1.1" 200 None 2026-03-31 00:52:26,618 - lava_callback - ERROR - Error processing callback for node 69c877c7f41e4fb6a37858b8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:42438 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49148 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49164 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60926 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41774 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53342 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53354 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60290 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60942 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53372 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53386 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51342 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47698 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53888 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53902 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37730 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55736 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40430 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40436 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45786 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34436 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52536 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52552 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60796 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38378 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46318 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46332 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52506 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42688 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33460 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33468 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55118 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57388 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50054 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50068 - "GET / HTTP/1.0" 200 OK 2026-03-31 00:56:47,353 - lava_callback - INFO - Received callback for node 69cb1bb88c06c2799bf5521c INFO: 172.20.0.1:50076 - "POST /node/69cb1bb88c06c2799bf5521c HTTP/1.0" 202 Accepted 2026-03-31 00:56:47,359 - lava_callback - INFO - Starting async job submit for node 69cb1bb88c06c2799bf5521c 2026-03-31 00:56:47,362 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:56:47,376 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1bb88c06c2799bf5521c HTTP/1.1" 200 None 2026-03-31 00:56:47,377 - lava_callback - DEBUG - Processing node 69cb1bb88c06c2799bf5521c: baseline-arm 2026-03-31 00:56:47,382 - lava_callback - WARNING - Failed to upload log 2026-03-31 00:56:47,385 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 00:56:47,501 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 00:56:47,503 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69cb1bb88c06c2799bf5521c/lava-logs.yaml 2026-03-31 00:56:47,504 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 00:56:47,620 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 00:56:47,622 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69cb1bb88c06c2799bf5521c/lava_callback.json.gz 2026-03-31 00:56:47,625 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:56:47,644 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1bb88c06c2799bf5521c HTTP/1.1" 200 None 2026-03-31 00:56:47,646 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:56:47,659 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12cb8c06c2799bf55018 HTTP/1.1" 200 None 2026-03-31 00:56:47,661 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:56:47,681 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb1bb88c06c2799bf5521c HTTP/1.1" 200 None 2026-03-31 00:56:47,690 - lava_callback - INFO - Completed processing callback for node 69cb1bb88c06c2799bf5521c 2026-03-31 00:56:52,140 - lava_callback - INFO - Received callback for node 69cb1bb88c06c2799bf5521f INFO: 172.20.0.1:60128 - "POST /node/69cb1bb88c06c2799bf5521f HTTP/1.0" 202 Accepted 2026-03-31 00:56:52,190 - lava_callback - INFO - Starting async job submit for node 69cb1bb88c06c2799bf5521f 2026-03-31 00:56:52,217 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:56:52,242 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1bb88c06c2799bf5521f HTTP/1.1" 200 None 2026-03-31 00:56:52,243 - lava_callback - DEBUG - Processing node 69cb1bb88c06c2799bf5521f: baseline-arm 2026-03-31 00:56:52,413 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 00:56:52,539 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 00:56:52,541 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69cb1bb88c06c2799bf5521f/log.txt.gz 2026-03-31 00:56:52,764 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 00:56:53,139 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 00:56:53,141 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69cb1bb88c06c2799bf5521f/lava-logs.yaml 2026-03-31 00:56:53,145 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 00:56:53,272 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 00:56:53,274 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69cb1bb88c06c2799bf5521f/lava_callback.json.gz 2026-03-31 00:56:53,304 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:56:53,329 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1bb88c06c2799bf5521f HTTP/1.1" 200 None 2026-03-31 00:56:53,332 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:56:53,355 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12cb8c06c2799bf55018 HTTP/1.1" 200 None 2026-03-31 00:56:53,357 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:56:53,392 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb1bb88c06c2799bf5521f HTTP/1.1" 200 None 2026-03-31 00:56:53,394 - lava_callback - INFO - Completed processing callback for node 69cb1bb88c06c2799bf5521f INFO: 172.20.0.1:60134 - "GET / HTTP/1.0" 200 OK 2026-03-31 00:57:00,268 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:57:00,357 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:36650 - "GET / HTTP/1.0" 200 OK 2026-03-31 00:57:07,763 - lava_callback - INFO - Received callback for node 69cb17358c06c2799bf55093 INFO: 172.20.0.1:36652 - "POST /node/69cb17358c06c2799bf55093 HTTP/1.0" 202 Accepted 2026-03-31 00:57:07,771 - lava_callback - INFO - Starting async job submit for node 69cb17358c06c2799bf55093 2026-03-31 00:57:07,804 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:57:07,830 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb17358c06c2799bf55093 HTTP/1.1" 200 None 2026-03-31 00:57:07,831 - lava_callback - DEBUG - Processing node 69cb17358c06c2799bf55093: baseline-x86 2026-03-31 00:57:08,368 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 00:57:08,625 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 00:57:08,627 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69cb17358c06c2799bf55093/log.txt.gz 2026-03-31 00:57:08,885 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 00:57:09,073 - lava_callback - INFO - Received callback for node 69cb1bbb8c06c2799bf55228 2026-03-31 00:57:09,125 - lava_callback - INFO - Starting async job submit for node 69cb1bbb8c06c2799bf55228 INFO: 172.20.0.1:36666 - "POST /node/69cb1bbb8c06c2799bf55228 HTTP/1.0" 202 Accepted 2026-03-31 00:57:09,161 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:57:09,189 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1bbb8c06c2799bf55228 HTTP/1.1" 200 None 2026-03-31 00:57:09,190 - lava_callback - DEBUG - Processing node 69cb1bbb8c06c2799bf55228: baseline-arm 2026-03-31 00:57:09,418 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 00:57:09,425 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69cb17358c06c2799bf55093/lava-logs.yaml 2026-03-31 00:57:09,450 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 00:57:09,452 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 00:57:09,868 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 00:57:09,870 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69cb17358c06c2799bf55093/lava_callback.json.gz 2026-03-31 00:57:09,896 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:57:09,908 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 00:57:09,910 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69cb1bbb8c06c2799bf55228/log.txt.gz 2026-03-31 00:57:09,938 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb17358c06c2799bf55093 HTTP/1.1" 200 None 2026-03-31 00:57:09,946 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:57:10,001 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12fb8c06c2799bf55043 HTTP/1.1" 200 None 2026-03-31 00:57:10,003 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:57:10,082 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 00:57:10,088 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb17358c06c2799bf55093 HTTP/1.1" 200 None 2026-03-31 00:57:10,089 - lava_callback - INFO - Completed processing callback for node 69cb17358c06c2799bf55093 2026-03-31 00:57:10,701 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 00:57:10,703 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69cb1bbb8c06c2799bf55228/lava-logs.yaml 2026-03-31 00:57:10,709 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 00:57:11,055 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 00:57:11,057 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69cb1bbb8c06c2799bf55228/lava_callback.json.gz 2026-03-31 00:57:11,098 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:57:11,137 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1bbb8c06c2799bf55228 HTTP/1.1" 200 None 2026-03-31 00:57:11,139 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:57:11,165 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12cb8c06c2799bf55018 HTTP/1.1" 200 None 2026-03-31 00:57:11,167 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:57:11,228 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb1bbb8c06c2799bf55228 HTTP/1.1" 200 None 2026-03-31 00:57:11,229 - lava_callback - INFO - Completed processing callback for node 69cb1bbb8c06c2799bf55228 2026-03-31 00:57:13,591 - lava_callback - INFO - Received callback for node 69cb17518c06c2799bf550bb INFO: 172.20.0.1:34818 - "POST /node/69cb17518c06c2799bf550bb HTTP/1.0" 202 Accepted 2026-03-31 00:57:13,602 - lava_callback - INFO - Starting async job submit for node 69cb17518c06c2799bf550bb 2026-03-31 00:57:13,750 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:57:13,778 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb17518c06c2799bf550bb HTTP/1.1" 200 None 2026-03-31 00:57:13,779 - lava_callback - DEBUG - Processing node 69cb17518c06c2799bf550bb: ltp-smoke 2026-03-31 00:57:14,667 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 00:57:15,041 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 00:57:15,043 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoke-69cb17518c06c2799bf550bb/log.txt.gz INFO: 172.20.0.1:34826 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-31 00:57:15,569 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 00:57:16,163 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 00:57:16,166 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-smoke-69cb17518c06c2799bf550bb/lava-logs.yaml 2026-03-31 00:57:16,182 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:34832 - "GET / HTTP/1.0" 200 OK 2026-03-31 00:57:16,686 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 00:57:16,688 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoke-69cb17518c06c2799bf550bb/lava_callback.json.gz 2026-03-31 00:57:16,907 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:57:16,933 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb17518c06c2799bf550bb HTTP/1.1" 200 None 2026-03-31 00:57:16,936 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:57:16,964 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12d38c06c2799bf55022 HTTP/1.1" 200 None 2026-03-31 00:57:16,967 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:57:17,026 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb17518c06c2799bf550bb HTTP/1.1" 200 None 2026-03-31 00:57:17,244 - lava_callback - INFO - Completed processing callback for node 69cb17518c06c2799bf550bb INFO: 172.20.0.1:44934 - "GET / HTTP/1.0" 200 OK 2026-03-31 00:57:30,360 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:57:30,387 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:58844 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40816 - "GET / HTTP/1.0" 200 OK 2026-03-31 00:57:43,136 - lava_callback - INFO - Received callback for node 69cb1bb78c06c2799bf55215 INFO: 172.20.0.1:40820 - "POST /node/69cb1bb78c06c2799bf55215 HTTP/1.0" 202 Accepted 2026-03-31 00:57:43,146 - lava_callback - INFO - Starting async job submit for node 69cb1bb78c06c2799bf55215 2026-03-31 00:57:43,189 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:57:43,218 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1bb78c06c2799bf55215 HTTP/1.1" 200 None 2026-03-31 00:57:43,219 - lava_callback - DEBUG - Processing node 69cb1bb78c06c2799bf55215: baseline-arm 2026-03-31 00:57:43,532 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 00:57:43,888 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 00:57:43,890 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69cb1bb78c06c2799bf55215/log.txt.gz 2026-03-31 00:57:44,118 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 00:57:44,790 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 00:57:44,793 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69cb1bb78c06c2799bf55215/lava-logs.yaml 2026-03-31 00:57:44,800 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 00:57:45,205 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 00:57:45,207 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69cb1bb78c06c2799bf55215/lava_callback.json.gz 2026-03-31 00:57:45,245 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:57:45,271 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1bb78c06c2799bf55215 HTTP/1.1" 200 None 2026-03-31 00:57:45,274 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:57:45,299 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12cb8c06c2799bf55018 HTTP/1.1" 200 None 2026-03-31 00:57:45,301 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:57:45,348 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb1bb78c06c2799bf55215 HTTP/1.1" 200 None 2026-03-31 00:57:45,357 - lava_callback - INFO - Completed processing callback for node 69cb1bb78c06c2799bf55215 INFO: 172.20.0.1:40828 - "GET / HTTP/1.0" 200 OK 2026-03-31 00:57:53,592 - lava_callback - INFO - Received callback for node 69cb1bba8c06c2799bf55225 INFO: 172.20.0.1:37534 - "POST /node/69cb1bba8c06c2799bf55225 HTTP/1.0" 202 Accepted 2026-03-31 00:57:53,602 - lava_callback - INFO - Starting async job submit for node 69cb1bba8c06c2799bf55225 2026-03-31 00:57:53,647 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:57:53,695 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1bba8c06c2799bf55225 HTTP/1.1" 200 None 2026-03-31 00:57:53,697 - lava_callback - DEBUG - Processing node 69cb1bba8c06c2799bf55225: baseline-arm 2026-03-31 00:57:54,212 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 00:57:54,627 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 00:57:54,630 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69cb1bba8c06c2799bf55225/log.txt.gz 2026-03-31 00:57:54,873 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 00:57:55,407 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 00:57:55,409 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69cb1bba8c06c2799bf55225/lava-logs.yaml 2026-03-31 00:57:55,418 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 00:57:55,920 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 00:57:55,922 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69cb1bba8c06c2799bf55225/lava_callback.json.gz 2026-03-31 00:57:55,964 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:57:55,995 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1bba8c06c2799bf55225 HTTP/1.1" 200 None 2026-03-31 00:57:55,997 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:57:56,022 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12cb8c06c2799bf55018 HTTP/1.1" 200 None 2026-03-31 00:57:56,025 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:57:56,080 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb1bba8c06c2799bf55225 HTTP/1.1" 200 None 2026-03-31 00:57:56,082 - lava_callback - INFO - Completed processing callback for node 69cb1bba8c06c2799bf55225 2026-03-31 00:57:56,223 - lava_callback - INFO - Received callback for node 69cb1bca8c06c2799bf5524c INFO: 172.20.0.1:37536 - "POST /node/69cb1bca8c06c2799bf5524c HTTP/1.0" 202 Accepted 2026-03-31 00:57:56,232 - lava_callback - INFO - Starting async job submit for node 69cb1bca8c06c2799bf5524c 2026-03-31 00:57:56,285 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:57:56,321 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1bca8c06c2799bf5524c HTTP/1.1" 200 None 2026-03-31 00:57:56,323 - lava_callback - DEBUG - Processing node 69cb1bca8c06c2799bf5524c: baseline-arm-broonie INFO: 172.20.0.1:37546 - "GET / HTTP/1.0" 200 OK 2026-03-31 00:57:56,700 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 00:57:57,114 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 00:57:57,117 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69cb1bca8c06c2799bf5524c/log.txt.gz 2026-03-31 00:57:57,404 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 00:57:57,662 - lava_callback - INFO - Received callback for node 69cb1bcb8c06c2799bf5524f INFO: 172.20.0.1:37558 - "POST /node/69cb1bcb8c06c2799bf5524f HTTP/1.0" 202 Accepted 2026-03-31 00:57:57,673 - lava_callback - INFO - Starting async job submit for node 69cb1bcb8c06c2799bf5524f 2026-03-31 00:57:57,723 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:57:57,761 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1bcb8c06c2799bf5524f HTTP/1.1" 200 None 2026-03-31 00:57:57,762 - lava_callback - DEBUG - Processing node 69cb1bcb8c06c2799bf5524f: baseline-arm-broonie 2026-03-31 00:57:58,049 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 00:57:58,077 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69cb1bca8c06c2799bf5524c/lava-logs.yaml 2026-03-31 00:57:58,093 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 00:57:58,283 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 00:57:58,518 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 00:57:58,523 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69cb1bca8c06c2799bf5524c/lava_callback.json.gz 2026-03-31 00:57:58,545 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 00:57:58,547 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69cb1bcb8c06c2799bf5524f/log.txt.gz 2026-03-31 00:57:58,641 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:57:58,689 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1bca8c06c2799bf5524c HTTP/1.1" 200 None 2026-03-31 00:57:58,696 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:57:58,760 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12cb8c06c2799bf55018 HTTP/1.1" 200 None 2026-03-31 00:57:58,763 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:57:58,864 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb1bca8c06c2799bf5524c HTTP/1.1" 200 None 2026-03-31 00:57:58,880 - lava_callback - INFO - Completed processing callback for node 69cb1bca8c06c2799bf5524c 2026-03-31 00:57:58,963 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 00:57:59,667 - lava_callback - INFO - Received callback for node 69cb1bdf8c06c2799bf552bc INFO: 172.20.0.1:37570 - "POST /node/69cb1bdf8c06c2799bf552bc HTTP/1.0" 202 Accepted 2026-03-31 00:57:59,717 - lava_callback - INFO - Starting async job submit for node 69cb1bdf8c06c2799bf552bc 2026-03-31 00:57:59,717 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 00:57:59,748 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69cb1bcb8c06c2799bf5524f/lava-logs.yaml 2026-03-31 00:57:59,774 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 00:57:59,794 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:57:59,819 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1bdf8c06c2799bf552bc HTTP/1.1" 200 None 2026-03-31 00:57:59,821 - lava_callback - DEBUG - Processing node 69cb1bdf8c06c2799bf552bc: baseline-arm-pengutronix 2026-03-31 00:58:00,073 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 00:58:00,297 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 00:58:00,299 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69cb1bcb8c06c2799bf5524f/lava_callback.json.gz 2026-03-31 00:58:00,354 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:58:00,389 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:58:00,393 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1bcb8c06c2799bf5524f HTTP/1.1" 200 None 2026-03-31 00:58:00,395 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:58:00,422 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12cb8c06c2799bf55018 HTTP/1.1" 200 None 2026-03-31 00:58:00,425 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:58:00,430 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-31 00:58:00,481 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb1bcb8c06c2799bf5524f HTTP/1.1" 200 None 2026-03-31 00:58:00,483 - lava_callback - INFO - Completed processing callback for node 69cb1bcb8c06c2799bf5524f 2026-03-31 00:58:00,556 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 00:58:00,558 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cb1bdf8c06c2799bf552bc/log.txt.gz 2026-03-31 00:58:00,714 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 00:58:00,744 - lava_callback - INFO - Received callback for node 69cb1bdb8c06c2799bf552af INFO: 172.20.0.1:56046 - "POST /node/69cb1bdb8c06c2799bf552af HTTP/1.0" 202 Accepted 2026-03-31 00:58:00,757 - lava_callback - INFO - Starting async job submit for node 69cb1bdb8c06c2799bf552af 2026-03-31 00:58:00,817 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:58:00,847 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1bdb8c06c2799bf552af HTTP/1.1" 200 None 2026-03-31 00:58:00,849 - lava_callback - DEBUG - Processing node 69cb1bdb8c06c2799bf552af: baseline-arm-pengutronix 2026-03-31 00:58:01,193 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 00:58:01,443 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 00:58:01,445 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cb1bdf8c06c2799bf552bc/lava-logs.yaml 2026-03-31 00:58:01,450 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 00:58:01,666 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 00:58:01,668 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cb1bdb8c06c2799bf552af/log.txt.gz 2026-03-31 00:58:01,821 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 00:58:01,821 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 00:58:01,824 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cb1bdf8c06c2799bf552bc/lava_callback.json.gz 2026-03-31 00:58:01,862 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:58:01,889 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1bdf8c06c2799bf552bc HTTP/1.1" 200 None 2026-03-31 00:58:01,892 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:58:01,917 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12cb8c06c2799bf55018 HTTP/1.1" 200 None 2026-03-31 00:58:01,920 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:58:01,961 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb1bdf8c06c2799bf552bc HTTP/1.1" 200 None 2026-03-31 00:58:01,963 - lava_callback - INFO - Completed processing callback for node 69cb1bdf8c06c2799bf552bc 2026-03-31 00:58:02,413 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 00:58:02,415 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cb1bdb8c06c2799bf552af/lava-logs.yaml 2026-03-31 00:58:02,420 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 00:58:02,835 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 00:58:02,837 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cb1bdb8c06c2799bf552af/lava_callback.json.gz 2026-03-31 00:58:02,883 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:58:02,946 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1bdb8c06c2799bf552af HTTP/1.1" 200 None 2026-03-31 00:58:02,949 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:58:02,988 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12cb8c06c2799bf55018 HTTP/1.1" 200 None 2026-03-31 00:58:02,990 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:58:03,064 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb1bdb8c06c2799bf552af HTTP/1.1" 200 None 2026-03-31 00:58:03,066 - lava_callback - INFO - Completed processing callback for node 69cb1bdb8c06c2799bf552af INFO: 172.20.0.1:56052 - "GET / HTTP/1.0" 200 OK 2026-03-31 00:58:12,048 - lava_callback - INFO - Received callback for node 69cb1bcc8c06c2799bf55252 INFO: 172.20.0.1:55754 - "POST /node/69cb1bcc8c06c2799bf55252 HTTP/1.0" 202 Accepted 2026-03-31 00:58:12,058 - lava_callback - INFO - Starting async job submit for node 69cb1bcc8c06c2799bf55252 2026-03-31 00:58:12,103 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:58:12,141 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1bcc8c06c2799bf55252 HTTP/1.1" 200 None 2026-03-31 00:58:12,142 - lava_callback - DEBUG - Processing node 69cb1bcc8c06c2799bf55252: baseline-arm-broonie 2026-03-31 00:58:12,556 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 00:58:12,803 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 00:58:12,805 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69cb1bcc8c06c2799bf55252/log.txt.gz 2026-03-31 00:58:13,056 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 00:58:13,515 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 00:58:13,518 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69cb1bcc8c06c2799bf55252/lava-logs.yaml 2026-03-31 00:58:13,528 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 00:58:13,933 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 00:58:13,936 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69cb1bcc8c06c2799bf55252/lava_callback.json.gz 2026-03-31 00:58:14,014 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:58:14,039 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1bcc8c06c2799bf55252 HTTP/1.1" 200 None 2026-03-31 00:58:14,041 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:58:14,065 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12cb8c06c2799bf55018 HTTP/1.1" 200 None 2026-03-31 00:58:14,067 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:58:14,107 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb1bcc8c06c2799bf55252 HTTP/1.1" 200 None 2026-03-31 00:58:14,109 - lava_callback - INFO - Completed processing callback for node 69cb1bcc8c06c2799bf55252 INFO: 172.20.0.1:55758 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55762 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56684 - "GET / HTTP/1.0" 200 OK 2026-03-31 00:58:30,432 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:58:30,449 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 2026-03-31 00:58:30,920 - lava_callback - INFO - Received callback for node 69cb1bd08c06c2799bf55263 INFO: 172.20.0.1:35746 - "POST /node/69cb1bd08c06c2799bf55263 HTTP/1.0" 202 Accepted 2026-03-31 00:58:30,926 - lava_callback - INFO - Starting async job submit for node 69cb1bd08c06c2799bf55263 2026-03-31 00:58:30,947 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:58:30,963 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1bd08c06c2799bf55263 HTTP/1.1" 200 None 2026-03-31 00:58:30,963 - lava_callback - DEBUG - Processing node 69cb1bd08c06c2799bf55263: baseline-arm-clabbe 2026-03-31 00:58:31,184 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 00:58:31,312 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 00:58:31,313 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69cb1bd08c06c2799bf55263/log.txt.gz 2026-03-31 00:58:31,447 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 00:58:31,671 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 00:58:31,673 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69cb1bd08c06c2799bf55263/lava-logs.yaml 2026-03-31 00:58:31,678 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 00:58:31,817 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 00:58:31,818 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69cb1bd08c06c2799bf55263/lava_callback.json.gz 2026-03-31 00:58:31,841 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:58:31,862 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1bd08c06c2799bf55263 HTTP/1.1" 200 None 2026-03-31 00:58:31,863 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:58:31,877 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12cb8c06c2799bf55018 HTTP/1.1" 200 None 2026-03-31 00:58:31,879 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:58:31,905 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb1bd08c06c2799bf55263 HTTP/1.1" 200 None 2026-03-31 00:58:31,906 - lava_callback - INFO - Completed processing callback for node 69cb1bd08c06c2799bf55263 INFO: 172.20.0.1:35752 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59014 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59022 - "GET / HTTP/1.0" 200 OK 2026-03-31 00:58:54,164 - lava_callback - INFO - Received callback for node 69cb1c3e8c06c2799bf55375 INFO: 172.20.0.1:58216 - "POST /node/69cb1c3e8c06c2799bf55375 HTTP/1.0" 202 Accepted 2026-03-31 00:58:54,171 - lava_callback - INFO - Starting async job submit for node 69cb1c3e8c06c2799bf55375 2026-03-31 00:58:54,174 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:58:54,189 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1c3e8c06c2799bf55375 HTTP/1.1" 200 None 2026-03-31 00:58:54,190 - lava_callback - DEBUG - Processing node 69cb1c3e8c06c2799bf55375: baseline-arm 2026-03-31 00:58:54,195 - lava_callback - WARNING - Failed to upload log 2026-03-31 00:58:54,198 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 00:58:54,326 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 00:58:54,328 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69cb1c3e8c06c2799bf55375/lava-logs.yaml 2026-03-31 00:58:54,330 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 00:58:54,454 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 00:58:54,456 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69cb1c3e8c06c2799bf55375/lava_callback.json.gz 2026-03-31 00:58:54,459 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:58:54,478 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1c3e8c06c2799bf55375 HTTP/1.1" 200 None 2026-03-31 00:58:54,480 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:58:54,497 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12cb8c06c2799bf55018 HTTP/1.1" 200 None 2026-03-31 00:58:54,499 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:58:54,526 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb1c3e8c06c2799bf55375 HTTP/1.1" 200 None 2026-03-31 00:58:54,532 - lava_callback - INFO - Completed processing callback for node 69cb1c3e8c06c2799bf55375 INFO: 172.20.0.1:58232 - "GET / HTTP/1.0" 200 OK 2026-03-31 00:59:00,450 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:59:00,466 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-31 00:59:01,329 - lava_callback - INFO - Received callback for node 69cb1bdc8c06c2799bf552b6 INFO: 172.20.0.1:38004 - "POST /node/69cb1bdc8c06c2799bf552b6 HTTP/1.0" 202 Accepted 2026-03-31 00:59:01,335 - lava_callback - INFO - Starting async job submit for node 69cb1bdc8c06c2799bf552b6 2026-03-31 00:59:01,360 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:59:01,377 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1bdc8c06c2799bf552b6 HTTP/1.1" 200 None 2026-03-31 00:59:01,378 - lava_callback - DEBUG - Processing node 69cb1bdc8c06c2799bf552b6: baseline-arm-pengutronix 2026-03-31 00:59:01,539 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 00:59:01,679 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 00:59:01,681 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cb1bdc8c06c2799bf552b6/log.txt.gz 2026-03-31 00:59:01,785 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 00:59:01,990 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 00:59:01,992 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cb1bdc8c06c2799bf552b6/lava-logs.yaml 2026-03-31 00:59:01,996 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 00:59:02,129 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 00:59:02,131 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cb1bdc8c06c2799bf552b6/lava_callback.json.gz 2026-03-31 00:59:02,178 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:59:02,196 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1bdc8c06c2799bf552b6 HTTP/1.1" 200 None 2026-03-31 00:59:02,198 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:59:02,212 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12cb8c06c2799bf55018 HTTP/1.1" 200 None 2026-03-31 00:59:02,214 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:59:02,239 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb1bdc8c06c2799bf552b6 HTTP/1.1" 200 None 2026-03-31 00:59:02,294 - lava_callback - INFO - Completed processing callback for node 69cb1bdc8c06c2799bf552b6 INFO: 172.20.0.1:38016 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41874 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41878 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48034 - "GET / HTTP/1.0" 200 OK 2026-03-31 00:59:30,468 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:59:30,485 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:37688 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60008 - "GET / HTTP/1.0" 200 OK 2026-03-31 00:59:41,272 - lava_callback - INFO - Received callback for node 69c87974f41e4fb6a37859a3 INFO: 172.20.0.1:60018 - "POST /node/69c87974f41e4fb6a37859a3 HTTP/1.0" 202 Accepted 2026-03-31 00:59:41,278 - lava_callback - INFO - Starting async job submit for node 69c87974f41e4fb6a37859a3 2026-03-31 00:59:41,279 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:59:41,294 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c87974f41e4fb6a37859a3 HTTP/1.1" 200 None 2026-03-31 00:59:41,295 - lava_callback - DEBUG - Processing node 69c87974f41e4fb6a37859a3: baseline-arm-baylibre 2026-03-31 00:59:41,295 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c87974f41e4fb6a37859a3 2026-03-31 00:59:41,295 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:59:41,314 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c87974f41e4fb6a37859a3 HTTP/1.1" 200 None 2026-03-31 00:59:41,315 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:59:41,329 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086a5ce0c9bc43bccb62b HTTP/1.1" 200 None 2026-03-31 00:59:41,330 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:59:41,350 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c87974f41e4fb6a37859a3 HTTP/1.1" 200 None 2026-03-31 00:59:41,357 - lava_callback - ERROR - Error processing callback for node 69c87974f41e4fb6a37859a3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-31 00:59:41,915 - lava_callback - INFO - Received callback for node 69cb1bfc8c06c2799bf552fa INFO: 172.20.0.1:60030 - "POST /node/69cb1bfc8c06c2799bf552fa HTTP/1.0" 202 Accepted 2026-03-31 00:59:41,921 - lava_callback - INFO - Starting async job submit for node 69cb1bfc8c06c2799bf552fa 2026-03-31 00:59:41,946 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:59:41,960 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1bfc8c06c2799bf552fa HTTP/1.1" 200 None 2026-03-31 00:59:41,961 - lava_callback - DEBUG - Processing node 69cb1bfc8c06c2799bf552fa: ltp-dio 2026-03-31 00:59:42,355 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 00:59:42,508 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 00:59:42,510 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-dio-69cb1bfc8c06c2799bf552fa/log.txt.gz 2026-03-31 00:59:42,770 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 00:59:43,061 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 00:59:43,063 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-dio-69cb1bfc8c06c2799bf552fa/lava-logs.yaml 2026-03-31 00:59:43,072 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 00:59:43,252 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 00:59:43,253 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-dio-69cb1bfc8c06c2799bf552fa/lava_callback.json.gz 2026-03-31 00:59:43,279 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:59:43,292 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1bfc8c06c2799bf552fa HTTP/1.1" 200 None 2026-03-31 00:59:43,294 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:59:43,308 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12cb8c06c2799bf55018 HTTP/1.1" 200 None 2026-03-31 00:59:43,309 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 00:59:43,346 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb1bfc8c06c2799bf552fa HTTP/1.1" 200 None 2026-03-31 00:59:43,348 - lava_callback - INFO - Completed processing callback for node 69cb1bfc8c06c2799bf552fa INFO: 172.20.0.1:60046 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37982 - "GET / HTTP/1.0" 200 OK 2026-03-31 01:00:00,487 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:00:00,503 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:58860 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55660 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55672 - "GET / HTTP/1.0" 200 OK 2026-03-31 01:00:18,347 - lava_callback - INFO - Received callback for node 69cb1bd18c06c2799bf55267 INFO: 172.20.0.1:55676 - "POST /node/69cb1bd18c06c2799bf55267 HTTP/1.0" 202 Accepted 2026-03-31 01:00:18,353 - lava_callback - INFO - Starting async job submit for node 69cb1bd18c06c2799bf55267 2026-03-31 01:00:18,381 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:00:18,396 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1bd18c06c2799bf55267 HTTP/1.1" 200 None 2026-03-31 01:00:18,397 - lava_callback - DEBUG - Processing node 69cb1bd18c06c2799bf55267: baseline-arm-clabbe 2026-03-31 01:00:18,641 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 01:00:18,768 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 01:00:18,769 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69cb1bd18c06c2799bf55267/log.txt.gz 2026-03-31 01:00:18,907 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 01:00:19,102 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 01:00:19,104 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69cb1bd18c06c2799bf55267/lava-logs.yaml 2026-03-31 01:00:19,109 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 01:00:19,237 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 01:00:19,238 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69cb1bd18c06c2799bf55267/lava_callback.json.gz 2026-03-31 01:00:19,261 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:00:19,276 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1bd18c06c2799bf55267 HTTP/1.1" 200 None 2026-03-31 01:00:19,277 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:00:19,291 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12cb8c06c2799bf55018 HTTP/1.1" 200 None 2026-03-31 01:00:19,292 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:00:19,321 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb1bd18c06c2799bf55267 HTTP/1.1" 200 None 2026-03-31 01:00:19,327 - lava_callback - INFO - Completed processing callback for node 69cb1bd18c06c2799bf55267 INFO: 172.20.0.1:52530 - "GET / HTTP/1.0" 200 OK 2026-03-31 01:00:30,505 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:00:30,524 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:49958 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43496 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43506 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60670 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51660 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52482 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-31 01:01:15,373 - lava_callback - INFO - Received callback for node 69cb1c0e8c06c2799bf55317 INFO: 172.20.0.1:52488 - "POST /node/69cb1c0e8c06c2799bf55317 HTTP/1.0" 202 Accepted 2026-03-31 01:01:15,381 - lava_callback - INFO - Starting async job submit for node 69cb1c0e8c06c2799bf55317 2026-03-31 01:01:15,414 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:01:15,428 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1c0e8c06c2799bf55317 HTTP/1.1" 200 None 2026-03-31 01:01:15,429 - lava_callback - DEBUG - Processing node 69cb1c0e8c06c2799bf55317: ltp-syscalls 2026-03-31 01:01:16,150 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 01:01:16,282 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 01:01:16,285 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-syscalls-69cb1c0e8c06c2799bf55317/log.txt.gz INFO: 172.20.0.1:52500 - "GET / HTTP/1.0" 200 OK 2026-03-31 01:01:16,760 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 01:01:17,014 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 01:01:17,016 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-syscalls-69cb1c0e8c06c2799bf55317/lava-logs.yaml 2026-03-31 01:01:17,036 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 01:01:17,186 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 01:01:17,188 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-syscalls-69cb1c0e8c06c2799bf55317/lava_callback.json.gz 2026-03-31 01:01:17,234 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:01:17,256 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1c0e8c06c2799bf55317 HTTP/1.1" 200 None 2026-03-31 01:01:17,258 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:01:17,275 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12cb8c06c2799bf55018 HTTP/1.1" 200 None 2026-03-31 01:01:17,278 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:01:17,314 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb1c0e8c06c2799bf55317 HTTP/1.1" 200 None 2026-03-31 01:01:17,316 - lava_callback - INFO - Completed processing callback for node 69cb1c0e8c06c2799bf55317 2026-03-31 01:01:21,293 - lava_callback - INFO - Received callback for node 69c879ccf41e4fb6a37859c8 INFO: 172.20.0.1:51742 - "POST /node/69c879ccf41e4fb6a37859c8 HTTP/1.0" 202 Accepted 2026-03-31 01:01:21,302 - lava_callback - INFO - Starting async job submit for node 69c879ccf41e4fb6a37859c8 2026-03-31 01:01:21,303 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:01:21,326 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c879ccf41e4fb6a37859c8 HTTP/1.1" 200 None 2026-03-31 01:01:21,327 - lava_callback - DEBUG - Processing node 69c879ccf41e4fb6a37859c8: baseline-arm64-baylibre 2026-03-31 01:01:21,328 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c879ccf41e4fb6a37859c8 2026-03-31 01:01:21,329 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:01:21,343 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c879ccf41e4fb6a37859c8 HTTP/1.1" 200 None 2026-03-31 01:01:21,345 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:01:21,364 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-03-31 01:01:21,366 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:01:21,392 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c879ccf41e4fb6a37859c8 HTTP/1.1" 200 None 2026-03-31 01:01:21,394 - lava_callback - ERROR - Error processing callback for node 69c879ccf41e4fb6a37859c8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:51756 - "GET / HTTP/1.0" 200 OK 2026-03-31 01:01:30,526 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:01:30,543 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:52210 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39466 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39476 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60390 - "GET / HTTP/1.0" 200 OK 2026-03-31 01:01:57,151 - lava_callback - INFO - Received callback for node 69cb1bd58c06c2799bf55277 INFO: 172.20.0.1:60392 - "POST /node/69cb1bd58c06c2799bf55277 HTTP/1.0" 202 Accepted 2026-03-31 01:01:57,157 - lava_callback - INFO - Starting async job submit for node 69cb1bd58c06c2799bf55277 2026-03-31 01:01:57,181 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:01:57,195 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1bd58c06c2799bf55277 HTTP/1.1" 200 None 2026-03-31 01:01:57,196 - lava_callback - DEBUG - Processing node 69cb1bd58c06c2799bf55277: baseline-arm-clabbe 2026-03-31 01:01:57,459 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 01:01:57,588 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 01:01:57,590 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69cb1bd58c06c2799bf55277/log.txt.gz 2026-03-31 01:01:57,723 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 01:01:57,965 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 01:01:57,967 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69cb1bd58c06c2799bf55277/lava-logs.yaml 2026-03-31 01:01:57,974 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 01:01:58,203 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 01:01:58,206 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69cb1bd58c06c2799bf55277/lava_callback.json.gz 2026-03-31 01:01:58,246 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:01:58,260 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1bd58c06c2799bf55277 HTTP/1.1" 200 None 2026-03-31 01:01:58,262 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:01:58,275 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12cb8c06c2799bf55018 HTTP/1.1" 200 None 2026-03-31 01:01:58,277 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:01:58,300 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb1bd58c06c2799bf55277 HTTP/1.1" 200 None 2026-03-31 01:01:58,305 - lava_callback - INFO - Completed processing callback for node 69cb1bd58c06c2799bf55277 2026-03-31 01:02:00,545 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:02:00,563 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:49702 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38288 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38298 - "GET / HTTP/1.0" 200 OK 2026-03-31 01:02:21,297 - lava_callback - INFO - Received callback for node 69c87a17f41e4fb6a37859d6 INFO: 172.20.0.1:39076 - "POST /node/69c87a17f41e4fb6a37859d6 HTTP/1.0" 202 Accepted 2026-03-31 01:02:21,303 - lava_callback - INFO - Starting async job submit for node 69c87a17f41e4fb6a37859d6 2026-03-31 01:02:21,304 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:02:21,319 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c87a17f41e4fb6a37859d6 HTTP/1.1" 200 None 2026-03-31 01:02:21,320 - lava_callback - DEBUG - Processing node 69c87a17f41e4fb6a37859d6: baseline-arm64-baylibre 2026-03-31 01:02:21,320 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c87a17f41e4fb6a37859d6 2026-03-31 01:02:21,321 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:02:21,334 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c87a17f41e4fb6a37859d6 HTTP/1.1" 200 None 2026-03-31 01:02:21,336 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:02:21,350 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-03-31 01:02:21,351 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:02:21,372 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c87a17f41e4fb6a37859d6 HTTP/1.1" 200 None 2026-03-31 01:02:21,373 - lava_callback - ERROR - Error processing callback for node 69c87a17f41e4fb6a37859d6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:39078 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53814 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45606 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45618 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53600 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50918 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49940 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49956 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46166 - "GET / HTTP/1.0" 200 OK 2026-03-31 01:03:36,137 - lava_callback - INFO - Received callback for node 69cb1bd68c06c2799bf552a1 INFO: 172.20.0.1:58678 - "POST /node/69cb1bd68c06c2799bf552a1 HTTP/1.0" 202 Accepted 2026-03-31 01:03:36,147 - lava_callback - INFO - Starting async job submit for node 69cb1bd68c06c2799bf552a1 2026-03-31 01:03:36,174 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:03:36,193 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1bd68c06c2799bf552a1 HTTP/1.1" 200 None 2026-03-31 01:03:36,194 - lava_callback - DEBUG - Processing node 69cb1bd68c06c2799bf552a1: baseline-arm-clabbe 2026-03-31 01:03:36,428 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:58690 - "GET / HTTP/1.0" 200 OK 2026-03-31 01:03:36,565 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 01:03:36,567 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69cb1bd68c06c2799bf552a1/log.txt.gz 2026-03-31 01:03:36,695 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 01:03:36,864 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 01:03:36,866 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69cb1bd68c06c2799bf552a1/lava-logs.yaml 2026-03-31 01:03:36,873 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 01:03:37,004 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 01:03:37,006 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69cb1bd68c06c2799bf552a1/lava_callback.json.gz 2026-03-31 01:03:37,027 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:03:37,041 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1bd68c06c2799bf552a1 HTTP/1.1" 200 None 2026-03-31 01:03:37,042 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:03:37,055 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12cb8c06c2799bf55018 HTTP/1.1" 200 None 2026-03-31 01:03:37,057 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:03:37,090 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb1bd68c06c2799bf552a1 HTTP/1.1" 200 None 2026-03-31 01:03:37,091 - lava_callback - INFO - Completed processing callback for node 69cb1bd68c06c2799bf552a1 2026-03-31 01:03:38,603 - lava_callback - INFO - Received callback for node 69cb1d558c06c2799bf553fa INFO: 172.20.0.1:58696 - "POST /node/69cb1d558c06c2799bf553fa HTTP/1.0" 202 Accepted 2026-03-31 01:03:38,613 - lava_callback - INFO - Starting async job submit for node 69cb1d558c06c2799bf553fa 2026-03-31 01:03:38,618 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:03:38,632 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1d558c06c2799bf553fa HTTP/1.1" 200 None 2026-03-31 01:03:38,633 - lava_callback - DEBUG - Processing node 69cb1d558c06c2799bf553fa: baseline-arm 2026-03-31 01:03:38,638 - lava_callback - WARNING - Failed to upload log 2026-03-31 01:03:38,642 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 01:03:38,779 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 01:03:38,781 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69cb1d558c06c2799bf553fa/lava-logs.yaml 2026-03-31 01:03:38,783 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 01:03:38,901 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 01:03:38,903 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69cb1d558c06c2799bf553fa/lava_callback.json.gz 2026-03-31 01:03:38,906 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:03:38,921 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1d558c06c2799bf553fa HTTP/1.1" 200 None 2026-03-31 01:03:38,923 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:03:38,944 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12cb8c06c2799bf55018 HTTP/1.1" 200 None 2026-03-31 01:03:38,946 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:03:38,974 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb1d558c06c2799bf553fa HTTP/1.1" 200 None 2026-03-31 01:03:38,986 - lava_callback - INFO - Completed processing callback for node 69cb1d558c06c2799bf553fa INFO: 172.20.0.1:53898 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53910 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58724 - "GET / HTTP/1.0" 200 OK 2026-03-31 01:04:00,565 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:04:00,583 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:39508 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48550 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48552 - "GET / HTTP/1.0" 200 OK 2026-03-31 01:04:21,449 - lava_callback - INFO - Received callback for node 69c87a88f41e4fb6a37859f2 INFO: 172.20.0.1:44742 - "POST /node/69c87a88f41e4fb6a37859f2 HTTP/1.0" 202 Accepted 2026-03-31 01:04:21,455 - lava_callback - INFO - Starting async job submit for node 69c87a88f41e4fb6a37859f2 2026-03-31 01:04:21,456 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:04:21,471 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c87a88f41e4fb6a37859f2 HTTP/1.1" 200 None 2026-03-31 01:04:21,472 - lava_callback - DEBUG - Processing node 69c87a88f41e4fb6a37859f2: baseline-arm64-baylibre 2026-03-31 01:04:21,472 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c87a88f41e4fb6a37859f2 2026-03-31 01:04:21,473 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:04:21,496 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c87a88f41e4fb6a37859f2 HTTP/1.1" 200 None 2026-03-31 01:04:21,498 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:04:21,517 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-03-31 01:04:21,519 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:04:21,547 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c87a88f41e4fb6a37859f2 HTTP/1.1" 200 None 2026-03-31 01:04:21,549 - lava_callback - ERROR - Error processing callback for node 69c87a88f41e4fb6a37859f2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:44748 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34194 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38166 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38178 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39532 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59990 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44304 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44318 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37688 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49228 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54112 - "GET / HTTP/1.0" 200 OK 2026-03-31 01:05:41,514 - lava_callback - INFO - Received callback for node 69c87ad6f41e4fb6a3785a06 INFO: 172.20.0.1:54118 - "POST /node/69c87ad6f41e4fb6a3785a06 HTTP/1.0" 202 Accepted 2026-03-31 01:05:41,521 - lava_callback - INFO - Starting async job submit for node 69c87ad6f41e4fb6a3785a06 2026-03-31 01:05:41,521 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:05:41,537 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c87ad6f41e4fb6a3785a06 HTTP/1.1" 200 None 2026-03-31 01:05:41,537 - lava_callback - DEBUG - Processing node 69c87ad6f41e4fb6a3785a06: baseline-arm64-baylibre 2026-03-31 01:05:41,538 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c87ad6f41e4fb6a3785a06 2026-03-31 01:05:41,538 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:05:41,553 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c87ad6f41e4fb6a3785a06 HTTP/1.1" 200 None 2026-03-31 01:05:41,554 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:05:41,578 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-03-31 01:05:41,581 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:05:41,603 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c87ad6f41e4fb6a3785a06 HTTP/1.1" 200 None 2026-03-31 01:05:41,611 - lava_callback - ERROR - Error processing callback for node 69c87ad6f41e4fb6a3785a06: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:54122 - "GET / HTTP/1.0" 200 OK 2026-03-31 01:05:53,150 - lava_callback - INFO - Received callback for node 69cb1d8a8c06c2799bf57991 INFO: 172.20.0.1:39146 - "POST /node/69cb1d8a8c06c2799bf57991 HTTP/1.0" 202 Accepted 2026-03-31 01:05:53,157 - lava_callback - INFO - Starting async job submit for node 69cb1d8a8c06c2799bf57991 2026-03-31 01:05:53,184 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:05:53,202 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1d8a8c06c2799bf57991 HTTP/1.1" 200 None 2026-03-31 01:05:53,203 - lava_callback - DEBUG - Processing node 69cb1d8a8c06c2799bf57991: baseline-arm-pengutronix 2026-03-31 01:05:53,374 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 01:05:53,553 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 01:05:53,555 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cb1d8a8c06c2799bf57991/log.txt.gz 2026-03-31 01:05:53,678 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 01:05:53,896 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 01:05:53,898 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cb1d8a8c06c2799bf57991/lava-logs.yaml 2026-03-31 01:05:53,903 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 01:05:54,082 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 01:05:54,084 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cb1d8a8c06c2799bf57991/lava_callback.json.gz 2026-03-31 01:05:54,174 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:05:54,191 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1d8a8c06c2799bf57991 HTTP/1.1" 200 None 2026-03-31 01:05:54,193 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:05:54,207 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12cb8c06c2799bf55018 HTTP/1.1" 200 None 2026-03-31 01:05:54,209 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:05:54,236 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb1d8a8c06c2799bf57991 HTTP/1.1" 200 None 2026-03-31 01:05:54,294 - lava_callback - INFO - Completed processing callback for node 69cb1d8a8c06c2799bf57991 INFO: 172.20.0.1:39160 - "GET / HTTP/1.0" 200 OK 2026-03-31 01:06:00,585 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:06:00,602 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:43238 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36142 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36144 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60232 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43642 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58364 - "GET / HTTP/1.0" 200 OK 2026-03-31 01:06:41,494 - lava_callback - INFO - Received callback for node 69c87b15f41e4fb6a3785a13 INFO: 172.20.0.1:58380 - "POST /node/69c87b15f41e4fb6a3785a13 HTTP/1.0" 202 Accepted 2026-03-31 01:06:41,500 - lava_callback - INFO - Starting async job submit for node 69c87b15f41e4fb6a3785a13 2026-03-31 01:06:41,501 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:06:41,524 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c87b15f41e4fb6a3785a13 HTTP/1.1" 200 None 2026-03-31 01:06:41,525 - lava_callback - DEBUG - Processing node 69c87b15f41e4fb6a3785a13: baseline-arm-baylibre 2026-03-31 01:06:41,525 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c87b15f41e4fb6a3785a13 2026-03-31 01:06:41,526 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:06:41,541 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c87b15f41e4fb6a3785a13 HTTP/1.1" 200 None 2026-03-31 01:06:41,543 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:06:41,557 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086a5ce0c9bc43bccb62b HTTP/1.1" 200 None 2026-03-31 01:06:41,559 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:06:41,583 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c87b15f41e4fb6a3785a13 HTTP/1.1" 200 None 2026-03-31 01:06:41,588 - lava_callback - ERROR - Error processing callback for node 69c87b15f41e4fb6a3785a13: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:58394 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52766 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37266 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51498 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51500 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44268 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49744 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50388 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50394 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42956 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41316 - "GET / HTTP/1.0" 200 OK 2026-03-31 01:08:07,489 - lava_callback - INFO - Received callback for node 69c87b6ff41e4fb6a3785a2f INFO: 172.20.0.1:41322 - "POST /node/69c87b6ff41e4fb6a3785a2f HTTP/1.0" 202 Accepted 2026-03-31 01:08:07,495 - lava_callback - INFO - Starting async job submit for node 69c87b6ff41e4fb6a3785a2f 2026-03-31 01:08:07,496 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:08:07,511 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c87b6ff41e4fb6a3785a2f HTTP/1.1" 200 None 2026-03-31 01:08:07,511 - lava_callback - DEBUG - Processing node 69c87b6ff41e4fb6a3785a2f: baseline-arm64-clabbe 2026-03-31 01:08:07,512 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c87b6ff41e4fb6a3785a2f 2026-03-31 01:08:07,512 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:08:07,526 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c87b6ff41e4fb6a3785a2f HTTP/1.1" 200 None 2026-03-31 01:08:07,527 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:08:07,541 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-03-31 01:08:07,542 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:08:07,562 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c87b6ff41e4fb6a3785a2f HTTP/1.1" 200 None 2026-03-31 01:08:07,567 - lava_callback - ERROR - Error processing callback for node 69c87b6ff41e4fb6a3785a2f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:54658 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54668 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44196 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36072 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33668 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33676 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48684 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39430 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38712 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38718 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39050 - "GET / HTTP/1.0" 200 OK 2026-03-31 01:09:27,346 - lava_callback - INFO - Received callback for node 69c87bbef41e4fb6a3785a46 INFO: 172.20.0.1:39052 - "POST /node/69c87bbef41e4fb6a3785a46 HTTP/1.0" 202 Accepted 2026-03-31 01:09:27,357 - lava_callback - INFO - Starting async job submit for node 69c87bbef41e4fb6a3785a46 2026-03-31 01:09:27,358 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:09:27,407 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c87bbef41e4fb6a3785a46 HTTP/1.1" 200 None 2026-03-31 01:09:27,409 - lava_callback - DEBUG - Processing node 69c87bbef41e4fb6a3785a46: baseline-arm64-clabbe 2026-03-31 01:09:27,409 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c87bbef41e4fb6a3785a46 2026-03-31 01:09:27,410 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:09:27,438 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c87bbef41e4fb6a3785a46 HTTP/1.1" 200 None 2026-03-31 01:09:27,440 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:09:27,469 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-03-31 01:09:27,472 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:09:27,502 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c87bbef41e4fb6a3785a46 HTTP/1.1" 200 None 2026-03-31 01:09:27,504 - lava_callback - ERROR - Error processing callback for node 69c87bbef41e4fb6a3785a46: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:39618 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33330 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33344 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48878 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49848 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49852 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49868 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34796 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38472 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54880 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54892 - "GET / HTTP/1.0" 200 OK 2026-03-31 01:10:47,399 - lava_callback - INFO - Received callback for node 69c87c0bf41e4fb6a3785a4d INFO: 172.20.0.1:54898 - "POST /node/69c87c0bf41e4fb6a3785a4d HTTP/1.0" 202 Accepted 2026-03-31 01:10:47,406 - lava_callback - INFO - Starting async job submit for node 69c87c0bf41e4fb6a3785a4d 2026-03-31 01:10:47,407 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:10:47,422 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c87c0bf41e4fb6a3785a4d HTTP/1.1" 200 None 2026-03-31 01:10:47,423 - lava_callback - DEBUG - Processing node 69c87c0bf41e4fb6a3785a4d: baseline-arm64-clabbe 2026-03-31 01:10:47,423 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c87c0bf41e4fb6a3785a4d 2026-03-31 01:10:47,424 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:10:47,437 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c87c0bf41e4fb6a3785a4d HTTP/1.1" 200 None 2026-03-31 01:10:47,438 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:10:47,451 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-03-31 01:10:47,452 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:10:47,477 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c87c0bf41e4fb6a3785a4d HTTP/1.1" 200 None 2026-03-31 01:10:47,483 - lava_callback - ERROR - Error processing callback for node 69c87c0bf41e4fb6a3785a4d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-31 01:10:54,804 - lava_callback - INFO - Received callback for node 69cb1f0d8c06c2799bf57a01 INFO: 172.20.0.1:60508 - "POST /node/69cb1f0d8c06c2799bf57a01 HTTP/1.0" 202 Accepted 2026-03-31 01:10:54,814 - lava_callback - INFO - Starting async job submit for node 69cb1f0d8c06c2799bf57a01 2026-03-31 01:10:54,819 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:10:54,844 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1f0d8c06c2799bf57a01 HTTP/1.1" 200 None 2026-03-31 01:10:54,845 - lava_callback - DEBUG - Processing node 69cb1f0d8c06c2799bf57a01: baseline-arm 2026-03-31 01:10:54,853 - lava_callback - WARNING - Failed to upload log 2026-03-31 01:10:54,857 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 01:10:54,986 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 01:10:54,988 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69cb1f0d8c06c2799bf57a01/lava-logs.yaml 2026-03-31 01:10:54,989 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 01:10:55,115 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 01:10:55,117 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69cb1f0d8c06c2799bf57a01/lava_callback.json.gz 2026-03-31 01:10:55,120 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:10:55,134 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1f0d8c06c2799bf57a01 HTTP/1.1" 200 None 2026-03-31 01:10:55,135 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:10:55,154 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12cb8c06c2799bf55018 HTTP/1.1" 200 None 2026-03-31 01:10:55,156 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:10:55,185 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb1f0d8c06c2799bf57a01 HTTP/1.1" 200 None 2026-03-31 01:10:55,190 - lava_callback - INFO - Completed processing callback for node 69cb1f0d8c06c2799bf57a01 INFO: 172.20.0.1:60524 - "GET / HTTP/1.0" 200 OK 2026-03-31 01:11:00,605 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:11:00,628 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:52672 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49158 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49162 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46144 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56172 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35050 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35056 - "GET / HTTP/1.0" 200 OK 2026-03-31 01:11:52,645 - lava_callback - INFO - Received callback for node 69cb1f458c06c2799bf57a81 INFO: 172.20.0.1:40062 - "POST /node/69cb1f458c06c2799bf57a81 HTTP/1.0" 202 Accepted 2026-03-31 01:11:52,651 - lava_callback - INFO - Starting async job submit for node 69cb1f458c06c2799bf57a81 2026-03-31 01:11:52,653 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:11:52,668 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1f458c06c2799bf57a81 HTTP/1.1" 200 None 2026-03-31 01:11:52,669 - lava_callback - DEBUG - Processing node 69cb1f458c06c2799bf57a81: baseline-nfs-arm64 2026-03-31 01:11:52,671 - lava_callback - WARNING - Failed to upload log 2026-03-31 01:11:52,672 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 01:11:52,786 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 01:11:52,788 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cb1f458c06c2799bf57a81/lava-logs.yaml 2026-03-31 01:11:52,789 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 01:11:52,895 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 01:11:52,897 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cb1f458c06c2799bf57a81/lava_callback.json.gz 2026-03-31 01:11:52,899 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:11:52,913 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1f458c06c2799bf57a81 HTTP/1.1" 200 None 2026-03-31 01:11:52,915 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:11:52,929 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12e08c06c2799bf5502b HTTP/1.1" 200 None 2026-03-31 01:11:52,931 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:11:52,986 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb1f458c06c2799bf57a81 HTTP/1.1" 200 None 2026-03-31 01:11:52,991 - lava_callback - INFO - Completed processing callback for node 69cb1f458c06c2799bf57a81 INFO: 172.20.0.1:40066 - "GET / HTTP/1.0" 200 OK 2026-03-31 01:11:56,681 - lava_callback - INFO - Received callback for node 69cb1f348c06c2799bf57a3f INFO: 172.20.0.1:40068 - "POST /node/69cb1f348c06c2799bf57a3f HTTP/1.0" 202 Accepted 2026-03-31 01:11:56,690 - lava_callback - INFO - Starting async job submit for node 69cb1f348c06c2799bf57a3f 2026-03-31 01:11:56,723 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:11:56,745 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1f348c06c2799bf57a3f HTTP/1.1" 200 None 2026-03-31 01:11:56,747 - lava_callback - DEBUG - Processing node 69cb1f348c06c2799bf57a3f: baseline-arm64 2026-03-31 01:11:56,849 - lava_callback - INFO - Received callback for node 69cb1f278c06c2799bf57a1e INFO: 172.20.0.1:40072 - "POST /node/69cb1f278c06c2799bf57a1e HTTP/1.0" 202 Accepted 2026-03-31 01:11:56,877 - lava_callback - INFO - Starting async job submit for node 69cb1f278c06c2799bf57a1e 2026-03-31 01:11:56,972 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:11:57,052 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1f278c06c2799bf57a1e HTTP/1.1" 200 None 2026-03-31 01:11:57,066 - lava_callback - DEBUG - Processing node 69cb1f278c06c2799bf57a1e: baseline-arm64 2026-03-31 01:11:57,398 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 01:11:57,400 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 01:11:57,977 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 01:11:57,979 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb1f348c06c2799bf57a3f/log.txt.gz 2026-03-31 01:11:58,016 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 01:11:58,018 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb1f278c06c2799bf57a1e/log.txt.gz 2026-03-31 01:11:58,292 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 01:11:58,293 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 01:11:58,854 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 01:11:58,856 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb1f348c06c2799bf57a3f/lava-logs.yaml 2026-03-31 01:11:58,866 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 01:11:58,894 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 01:11:58,897 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb1f278c06c2799bf57a1e/lava-logs.yaml 2026-03-31 01:11:58,902 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 01:11:59,042 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 01:11:59,044 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb1f348c06c2799bf57a3f/lava_callback.json.gz 2026-03-31 01:11:59,068 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 01:11:59,070 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:11:59,070 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb1f278c06c2799bf57a1e/lava_callback.json.gz 2026-03-31 01:11:59,092 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:11:59,107 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1f278c06c2799bf57a1e HTTP/1.1" 200 None 2026-03-31 01:11:59,109 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1f348c06c2799bf57a3f HTTP/1.1" 200 None 2026-03-31 01:11:59,110 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:11:59,112 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:11:59,136 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12e08c06c2799bf5502b HTTP/1.1" 200 None 2026-03-31 01:11:59,138 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12e08c06c2799bf5502b HTTP/1.1" 200 None 2026-03-31 01:11:59,139 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:11:59,140 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:11:59,171 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb1f348c06c2799bf57a3f HTTP/1.1" 200 None 2026-03-31 01:11:59,173 - lava_callback - INFO - Completed processing callback for node 69cb1f348c06c2799bf57a3f 2026-03-31 01:11:59,210 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb1f278c06c2799bf57a1e HTTP/1.1" 200 None 2026-03-31 01:11:59,212 - lava_callback - INFO - Completed processing callback for node 69cb1f278c06c2799bf57a1e 2026-03-31 01:12:00,631 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:12:00,662 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-31 01:12:05,778 - lava_callback - INFO - Received callback for node 69cb1f2b8c06c2799bf57a2a INFO: 172.20.0.1:33876 - "POST /node/69cb1f2b8c06c2799bf57a2a HTTP/1.0" 202 Accepted 2026-03-31 01:12:05,799 - lava_callback - INFO - Starting async job submit for node 69cb1f2b8c06c2799bf57a2a 2026-03-31 01:12:05,852 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:12:05,897 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1f2b8c06c2799bf57a2a HTTP/1.1" 200 None 2026-03-31 01:12:05,898 - lava_callback - DEBUG - Processing node 69cb1f2b8c06c2799bf57a2a: baseline-arm64 2026-03-31 01:12:06,133 - lava_callback - WARNING - Failed to upload log 2026-03-31 01:12:06,271 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:33892 - "GET / HTTP/1.0" 200 OK 2026-03-31 01:12:06,555 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 01:12:06,557 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb1f2b8c06c2799bf57a2a/lava-logs.yaml 2026-03-31 01:12:06,561 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 01:12:07,030 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 01:12:07,033 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb1f2b8c06c2799bf57a2a/lava_callback.json.gz 2026-03-31 01:12:07,089 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:12:07,117 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1f2b8c06c2799bf57a2a HTTP/1.1" 200 None 2026-03-31 01:12:07,120 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:12:07,144 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12e08c06c2799bf5502b HTTP/1.1" 200 None 2026-03-31 01:12:07,146 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:12:07,176 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb1f2b8c06c2799bf57a2a HTTP/1.1" 200 None 2026-03-31 01:12:07,264 - lava_callback - INFO - Completed processing callback for node 69cb1f2b8c06c2799bf57a2a 2026-03-31 01:12:07,472 - lava_callback - INFO - Received callback for node 69c87c57f41e4fb6a3785a61 INFO: 172.20.0.1:33896 - "POST /node/69c87c57f41e4fb6a3785a61 HTTP/1.0" 202 Accepted 2026-03-31 01:12:07,481 - lava_callback - INFO - Starting async job submit for node 69c87c57f41e4fb6a3785a61 2026-03-31 01:12:07,482 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:12:07,505 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c87c57f41e4fb6a3785a61 HTTP/1.1" 200 None 2026-03-31 01:12:07,507 - lava_callback - DEBUG - Processing node 69c87c57f41e4fb6a3785a61: baseline-arm64-clabbe 2026-03-31 01:12:07,507 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c87c57f41e4fb6a3785a61 2026-03-31 01:12:07,508 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:12:07,528 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c87c57f41e4fb6a3785a61 HTTP/1.1" 200 None 2026-03-31 01:12:07,530 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:12:07,565 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-03-31 01:12:07,567 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:12:07,609 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c87c57f41e4fb6a3785a61 HTTP/1.1" 200 None 2026-03-31 01:12:07,611 - lava_callback - ERROR - Error processing callback for node 69c87c57f41e4fb6a3785a61: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:33456 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33470 - "GET / HTTP/1.0" 200 OK 2026-03-31 01:12:21,984 - lava_callback - INFO - Received callback for node 69cb1f358c06c2799bf57a42 INFO: 172.20.0.1:38840 - "POST /node/69cb1f358c06c2799bf57a42 HTTP/1.0" 202 Accepted 2026-03-31 01:12:21,991 - lava_callback - INFO - Starting async job submit for node 69cb1f358c06c2799bf57a42 2026-03-31 01:12:22,023 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:12:22,046 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1f358c06c2799bf57a42 HTTP/1.1" 200 None 2026-03-31 01:12:22,047 - lava_callback - DEBUG - Processing node 69cb1f358c06c2799bf57a42: baseline-arm64 2026-03-31 01:12:22,073 - lava_callback - INFO - Received callback for node 69cb1f298c06c2799bf57a24 2026-03-31 01:12:22,107 - lava_callback - INFO - Starting async job submit for node 69cb1f298c06c2799bf57a24 INFO: 172.20.0.1:38850 - "POST /node/69cb1f298c06c2799bf57a24 HTTP/1.0" 202 Accepted 2026-03-31 01:12:22,210 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:12:22,299 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1f298c06c2799bf57a24 HTTP/1.1" 200 None 2026-03-31 01:12:22,301 - lava_callback - DEBUG - Processing node 69cb1f298c06c2799bf57a24: baseline-arm64 2026-03-31 01:12:22,828 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 01:12:23,006 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 01:12:23,313 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 01:12:23,316 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb1f358c06c2799bf57a42/log.txt.gz 2026-03-31 01:12:23,367 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 01:12:23,387 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb1f298c06c2799bf57a24/log.txt.gz 2026-03-31 01:12:23,645 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 01:12:23,839 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 01:12:24,179 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 01:12:24,181 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb1f358c06c2799bf57a42/lava-logs.yaml 2026-03-31 01:12:24,188 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 01:12:24,208 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 01:12:24,209 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb1f298c06c2799bf57a24/lava-logs.yaml 2026-03-31 01:12:24,216 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 01:12:24,353 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 01:12:24,355 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb1f358c06c2799bf57a42/lava_callback.json.gz 2026-03-31 01:12:24,373 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 01:12:24,380 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:12:24,380 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb1f298c06c2799bf57a24/lava_callback.json.gz 2026-03-31 01:12:24,404 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:12:24,419 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1f358c06c2799bf57a42 HTTP/1.1" 200 None 2026-03-31 01:12:24,422 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:12:24,422 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1f298c06c2799bf57a24 HTTP/1.1" 200 None 2026-03-31 01:12:24,424 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:12:24,441 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12e08c06c2799bf5502b HTTP/1.1" 200 None 2026-03-31 01:12:24,444 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:12:24,445 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12e08c06c2799bf5502b HTTP/1.1" 200 None 2026-03-31 01:12:24,447 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:12:24,484 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb1f358c06c2799bf57a42 HTTP/1.1" 200 None 2026-03-31 01:12:24,486 - lava_callback - INFO - Completed processing callback for node 69cb1f358c06c2799bf57a42 2026-03-31 01:12:24,517 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb1f298c06c2799bf57a24 HTTP/1.1" 200 None 2026-03-31 01:12:24,519 - lava_callback - INFO - Completed processing callback for node 69cb1f298c06c2799bf57a24 2026-03-31 01:12:25,261 - lava_callback - INFO - Received callback for node 69cb1f398c06c2799bf57a4b INFO: 172.20.0.1:38852 - "POST /node/69cb1f398c06c2799bf57a4b HTTP/1.0" 202 Accepted 2026-03-31 01:12:25,271 - lava_callback - INFO - Starting async job submit for node 69cb1f398c06c2799bf57a4b 2026-03-31 01:12:25,297 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:12:25,312 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1f398c06c2799bf57a4b HTTP/1.1" 200 None 2026-03-31 01:12:25,313 - lava_callback - DEBUG - Processing node 69cb1f398c06c2799bf57a4b: baseline-arm64 2026-03-31 01:12:25,789 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 01:12:26,079 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 01:12:26,081 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb1f398c06c2799bf57a4b/log.txt.gz 2026-03-31 01:12:26,256 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:38862 - "GET / HTTP/1.0" 200 OK 2026-03-31 01:12:26,797 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 01:12:26,799 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb1f398c06c2799bf57a4b/lava-logs.yaml 2026-03-31 01:12:26,807 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 01:12:26,935 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 01:12:26,936 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb1f398c06c2799bf57a4b/lava_callback.json.gz 2026-03-31 01:12:26,959 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:12:26,973 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1f398c06c2799bf57a4b HTTP/1.1" 200 None 2026-03-31 01:12:26,974 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:12:26,988 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12e08c06c2799bf5502b HTTP/1.1" 200 None 2026-03-31 01:12:26,990 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:12:27,016 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb1f398c06c2799bf57a4b HTTP/1.1" 200 None 2026-03-31 01:12:27,018 - lava_callback - INFO - Completed processing callback for node 69cb1f398c06c2799bf57a4b 2026-03-31 01:12:30,664 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:12:30,681 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 2026-03-31 01:12:31,107 - lava_callback - INFO - Received callback for node 69cb1f3a8c06c2799bf57a4e INFO: 172.20.0.1:54502 - "POST /node/69cb1f3a8c06c2799bf57a4e HTTP/1.0" 202 Accepted 2026-03-31 01:12:31,117 - lava_callback - INFO - Starting async job submit for node 69cb1f3a8c06c2799bf57a4e 2026-03-31 01:12:31,162 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:12:31,201 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1f3a8c06c2799bf57a4e HTTP/1.1" 200 None 2026-03-31 01:12:31,202 - lava_callback - DEBUG - Processing node 69cb1f3a8c06c2799bf57a4e: baseline-arm64 2026-03-31 01:12:31,593 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 01:12:32,011 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 01:12:32,013 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb1f3a8c06c2799bf57a4e/log.txt.gz 2026-03-31 01:12:32,202 - lava_callback - INFO - Received callback for node 69cb1f2a8c06c2799bf57a27 INFO: 172.20.0.1:54508 - "POST /node/69cb1f2a8c06c2799bf57a27 HTTP/1.0" 202 Accepted 2026-03-31 01:12:32,220 - lava_callback - INFO - Starting async job submit for node 69cb1f2a8c06c2799bf57a27 2026-03-31 01:12:32,244 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 01:12:32,245 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:12:32,259 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1f2a8c06c2799bf57a27 HTTP/1.1" 200 None 2026-03-31 01:12:32,260 - lava_callback - DEBUG - Processing node 69cb1f2a8c06c2799bf57a27: baseline-arm64 2026-03-31 01:12:32,643 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 01:12:32,744 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 01:12:32,747 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb1f3a8c06c2799bf57a4e/lava-logs.yaml 2026-03-31 01:12:32,757 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 01:12:32,869 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 01:12:32,871 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb1f2a8c06c2799bf57a27/log.txt.gz 2026-03-31 01:12:32,965 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 01:12:32,999 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb1f3a8c06c2799bf57a4e/lava_callback.json.gz 2026-03-31 01:12:33,039 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:12:33,064 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 01:12:33,073 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1f3a8c06c2799bf57a4e HTTP/1.1" 200 None 2026-03-31 01:12:33,074 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:12:33,089 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12e08c06c2799bf5502b HTTP/1.1" 200 None 2026-03-31 01:12:33,090 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:12:33,128 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb1f3a8c06c2799bf57a4e HTTP/1.1" 200 None 2026-03-31 01:12:33,129 - lava_callback - INFO - Completed processing callback for node 69cb1f3a8c06c2799bf57a4e 2026-03-31 01:12:33,665 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 01:12:33,668 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb1f2a8c06c2799bf57a27/lava-logs.yaml 2026-03-31 01:12:33,678 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 01:12:33,999 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 01:12:34,001 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb1f2a8c06c2799bf57a27/lava_callback.json.gz 2026-03-31 01:12:34,033 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:12:34,068 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1f2a8c06c2799bf57a27 HTTP/1.1" 200 None 2026-03-31 01:12:34,070 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:12:34,105 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12e08c06c2799bf5502b HTTP/1.1" 200 None 2026-03-31 01:12:34,108 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:12:34,152 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb1f2a8c06c2799bf57a27 HTTP/1.1" 200 None 2026-03-31 01:12:34,153 - lava_callback - INFO - Completed processing callback for node 69cb1f2a8c06c2799bf57a27 2026-03-31 01:12:34,891 - lava_callback - INFO - Received callback for node 69cb1f318c06c2799bf57a39 INFO: 172.20.0.1:54514 - "POST /node/69cb1f318c06c2799bf57a39 HTTP/1.0" 202 Accepted 2026-03-31 01:12:34,902 - lava_callback - INFO - Starting async job submit for node 69cb1f318c06c2799bf57a39 2026-03-31 01:12:34,939 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:12:34,968 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1f318c06c2799bf57a39 HTTP/1.1" 200 None 2026-03-31 01:12:34,970 - lava_callback - DEBUG - Processing node 69cb1f318c06c2799bf57a39: baseline-arm64 2026-03-31 01:12:35,581 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 01:12:36,083 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 01:12:36,085 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb1f318c06c2799bf57a39/log.txt.gz 2026-03-31 01:12:36,284 - lava_callback - INFO - Received callback for node 69cb1f308c06c2799bf57a36 INFO: 172.20.0.1:54516 - "POST /node/69cb1f308c06c2799bf57a36 HTTP/1.0" 202 Accepted 2026-03-31 01:12:36,307 - lava_callback - INFO - Starting async job submit for node 69cb1f308c06c2799bf57a36 2026-03-31 01:12:36,380 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:12:36,458 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1f308c06c2799bf57a36 HTTP/1.1" 200 None 2026-03-31 01:12:36,476 - lava_callback - DEBUG - Processing node 69cb1f308c06c2799bf57a36: baseline-arm64 2026-03-31 01:12:36,493 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:54522 - "GET / HTTP/1.0" 200 OK 2026-03-31 01:12:37,182 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 01:12:37,198 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb1f318c06c2799bf57a39/lava-logs.yaml 2026-03-31 01:12:37,201 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 01:12:37,207 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 01:12:37,411 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 01:12:37,413 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb1f308c06c2799bf57a36/log.txt.gz 2026-03-31 01:12:37,617 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 01:12:37,619 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb1f318c06c2799bf57a39/lava_callback.json.gz 2026-03-31 01:12:37,691 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:12:37,740 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1f318c06c2799bf57a39 HTTP/1.1" 200 None 2026-03-31 01:12:37,743 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:12:37,808 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12e08c06c2799bf5502b HTTP/1.1" 200 None 2026-03-31 01:12:37,817 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:12:37,885 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 01:12:37,895 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb1f318c06c2799bf57a39 HTTP/1.1" 200 None 2026-03-31 01:12:37,896 - lava_callback - INFO - Completed processing callback for node 69cb1f318c06c2799bf57a39 2026-03-31 01:12:38,186 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 01:12:38,188 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb1f308c06c2799bf57a36/lava-logs.yaml 2026-03-31 01:12:38,197 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 01:12:38,322 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 01:12:38,324 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb1f308c06c2799bf57a36/lava_callback.json.gz 2026-03-31 01:12:38,349 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:12:38,368 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1f308c06c2799bf57a36 HTTP/1.1" 200 None 2026-03-31 01:12:38,370 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:12:38,388 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12e08c06c2799bf5502b HTTP/1.1" 200 None 2026-03-31 01:12:38,390 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:12:38,451 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb1f308c06c2799bf57a36 HTTP/1.1" 200 None 2026-03-31 01:12:38,452 - lava_callback - INFO - Completed processing callback for node 69cb1f308c06c2799bf57a36 INFO: 172.20.0.1:56520 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56528 - "GET / HTTP/1.0" 200 OK 2026-03-31 01:12:46,911 - lava_callback - INFO - Received callback for node 69cb1f2c8c06c2799bf57a2d INFO: 172.20.0.1:56532 - "POST /node/69cb1f2c8c06c2799bf57a2d HTTP/1.0" 202 Accepted 2026-03-31 01:12:46,924 - lava_callback - INFO - Starting async job submit for node 69cb1f2c8c06c2799bf57a2d 2026-03-31 01:12:46,977 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:12:47,001 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1f2c8c06c2799bf57a2d HTTP/1.1" 200 None 2026-03-31 01:12:47,002 - lava_callback - DEBUG - Processing node 69cb1f2c8c06c2799bf57a2d: baseline-arm64 2026-03-31 01:12:47,256 - lava_callback - WARNING - Failed to upload log 2026-03-31 01:12:47,466 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 01:12:47,916 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 01:12:47,918 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb1f2c8c06c2799bf57a2d/lava-logs.yaml 2026-03-31 01:12:47,923 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 01:12:48,077 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 01:12:48,079 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb1f2c8c06c2799bf57a2d/lava_callback.json.gz 2026-03-31 01:12:48,136 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:12:48,164 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1f2c8c06c2799bf57a2d HTTP/1.1" 200 None 2026-03-31 01:12:48,166 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:12:48,188 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12e08c06c2799bf5502b HTTP/1.1" 200 None 2026-03-31 01:12:48,190 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:12:48,246 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb1f2c8c06c2799bf57a2d HTTP/1.1" 200 None 2026-03-31 01:12:48,350 - lava_callback - INFO - Completed processing callback for node 69cb1f2c8c06c2799bf57a2d 2026-03-31 01:12:49,134 - lava_callback - INFO - Received callback for node 69cb1f2f8c06c2799bf57a33 INFO: 172.20.0.1:56536 - "POST /node/69cb1f2f8c06c2799bf57a33 HTTP/1.0" 202 Accepted 2026-03-31 01:12:49,149 - lava_callback - INFO - Starting async job submit for node 69cb1f2f8c06c2799bf57a33 2026-03-31 01:12:49,199 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:12:49,224 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1f2f8c06c2799bf57a33 HTTP/1.1" 200 None 2026-03-31 01:12:49,226 - lava_callback - DEBUG - Processing node 69cb1f2f8c06c2799bf57a33: baseline-arm64 2026-03-31 01:12:49,530 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 01:12:49,664 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 01:12:49,666 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb1f2f8c06c2799bf57a33/log.txt.gz 2026-03-31 01:12:49,729 - lava_callback - INFO - Received callback for node 69cb1f388c06c2799bf57a48 INFO: 172.20.0.1:44178 - "POST /node/69cb1f388c06c2799bf57a48 HTTP/1.0" 202 Accepted 2026-03-31 01:12:49,753 - lava_callback - INFO - Starting async job submit for node 69cb1f388c06c2799bf57a48 2026-03-31 01:12:49,838 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:12:49,918 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1f388c06c2799bf57a48 HTTP/1.1" 200 None 2026-03-31 01:12:49,919 - lava_callback - DEBUG - Processing node 69cb1f388c06c2799bf57a48: baseline-arm64 2026-03-31 01:12:50,130 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 01:12:50,551 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 01:12:50,553 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb1f2f8c06c2799bf57a33/lava-logs.yaml 2026-03-31 01:12:50,568 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 01:12:50,690 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 01:12:51,101 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 01:12:51,103 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb1f388c06c2799bf57a48/log.txt.gz 2026-03-31 01:12:51,152 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 01:12:51,154 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb1f2f8c06c2799bf57a33/lava_callback.json.gz 2026-03-31 01:12:51,245 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:12:51,317 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1f2f8c06c2799bf57a33 HTTP/1.1" 200 None 2026-03-31 01:12:51,343 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:12:51,414 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12e08c06c2799bf5502b HTTP/1.1" 200 None 2026-03-31 01:12:51,418 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:12:51,419 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 01:12:51,450 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb1f2f8c06c2799bf57a33 HTTP/1.1" 200 None 2026-03-31 01:12:51,452 - lava_callback - INFO - Completed processing callback for node 69cb1f2f8c06c2799bf57a33 2026-03-31 01:12:52,061 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 01:12:52,064 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb1f388c06c2799bf57a48/lava-logs.yaml 2026-03-31 01:12:52,072 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 01:12:52,228 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 01:12:52,230 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb1f388c06c2799bf57a48/lava_callback.json.gz 2026-03-31 01:12:52,253 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:12:52,268 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1f388c06c2799bf57a48 HTTP/1.1" 200 None 2026-03-31 01:12:52,269 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:12:52,286 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12e08c06c2799bf5502b HTTP/1.1" 200 None 2026-03-31 01:12:52,288 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:12:52,323 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb1f388c06c2799bf57a48 HTTP/1.1" 200 None 2026-03-31 01:12:52,325 - lava_callback - INFO - Completed processing callback for node 69cb1f388c06c2799bf57a48 INFO: 172.20.0.1:44192 - "GET / HTTP/1.0" 200 OK 2026-03-31 01:12:58,104 - lava_callback - INFO - Received callback for node 69cb1f4a8c06c2799bf57a8e INFO: 172.20.0.1:44200 - "POST /node/69cb1f4a8c06c2799bf57a8e HTTP/1.0" 202 Accepted 2026-03-31 01:12:58,114 - lava_callback - INFO - Starting async job submit for node 69cb1f4a8c06c2799bf57a8e 2026-03-31 01:12:58,157 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:12:58,211 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1f4a8c06c2799bf57a8e HTTP/1.1" 200 None 2026-03-31 01:12:58,213 - lava_callback - DEBUG - Processing node 69cb1f4a8c06c2799bf57a8e: baseline-nfs-arm64 2026-03-31 01:12:58,737 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 01:12:59,180 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 01:12:59,182 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cb1f4a8c06c2799bf57a8e/log.txt.gz 2026-03-31 01:12:59,494 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 01:12:59,680 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 01:12:59,682 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cb1f4a8c06c2799bf57a8e/lava-logs.yaml 2026-03-31 01:12:59,692 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 01:13:00,105 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 01:13:00,107 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cb1f4a8c06c2799bf57a8e/lava_callback.json.gz 2026-03-31 01:13:00,153 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:13:00,177 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1f4a8c06c2799bf57a8e HTTP/1.1" 200 None 2026-03-31 01:13:00,179 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:13:00,210 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12e08c06c2799bf5502b HTTP/1.1" 200 None 2026-03-31 01:13:00,212 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:13:00,236 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb1f4a8c06c2799bf57a8e HTTP/1.1" 200 None 2026-03-31 01:13:00,238 - lava_callback - INFO - Completed processing callback for node 69cb1f4a8c06c2799bf57a8e 2026-03-31 01:13:00,684 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:13:00,713 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 INFO: 172.20.0.1:54274 - "GET / HTTP/1.0" 200 OK 2026-03-31 01:13:06,682 - lava_callback - INFO - Received callback for node 69cb1f288c06c2799bf57a21 INFO: 172.20.0.1:54288 - "POST /node/69cb1f288c06c2799bf57a21 HTTP/1.0" 202 Accepted 2026-03-31 01:13:06,692 - lava_callback - INFO - Starting async job submit for node 69cb1f288c06c2799bf57a21 2026-03-31 01:13:06,751 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:13:06,788 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1f288c06c2799bf57a21 HTTP/1.1" 200 None 2026-03-31 01:13:06,790 - lava_callback - DEBUG - Processing node 69cb1f288c06c2799bf57a21: baseline-arm64 2026-03-31 01:13:07,223 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 01:13:08,128 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 01:13:08,130 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb1f288c06c2799bf57a21/log.txt.gz 2026-03-31 01:13:08,380 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 01:13:08,575 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 01:13:08,577 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb1f288c06c2799bf57a21/lava-logs.yaml 2026-03-31 01:13:08,586 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 01:13:09,080 - lava_callback - INFO - Received callback for node 69cb1f498c06c2799bf57a8b INFO: 172.20.0.1:54292 - "POST /node/69cb1f498c06c2799bf57a8b HTTP/1.0" 202 Accepted 2026-03-31 01:13:09,090 - lava_callback - INFO - Starting async job submit for node 69cb1f498c06c2799bf57a8b 2026-03-31 01:13:09,142 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:13:09,178 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1f498c06c2799bf57a8b HTTP/1.1" 200 None 2026-03-31 01:13:09,179 - lava_callback - DEBUG - Processing node 69cb1f498c06c2799bf57a8b: baseline-nfs-arm64 2026-03-31 01:13:09,429 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 01:13:09,442 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb1f288c06c2799bf57a21/lava_callback.json.gz 2026-03-31 01:13:09,527 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:13:09,613 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1f288c06c2799bf57a21 HTTP/1.1" 200 None 2026-03-31 01:13:09,641 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:13:09,774 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12e08c06c2799bf5502b HTTP/1.1" 200 None 2026-03-31 01:13:09,787 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:13:09,819 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 01:13:09,859 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb1f288c06c2799bf57a21 HTTP/1.1" 200 None 2026-03-31 01:13:09,861 - lava_callback - INFO - Completed processing callback for node 69cb1f288c06c2799bf57a21 2026-03-31 01:13:10,787 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 01:13:10,789 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cb1f498c06c2799bf57a8b/log.txt.gz 2026-03-31 01:13:11,100 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 01:13:12,071 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 01:13:12,073 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cb1f498c06c2799bf57a8b/lava-logs.yaml 2026-03-31 01:13:12,083 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 01:13:12,265 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 01:13:12,267 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cb1f498c06c2799bf57a8b/lava_callback.json.gz 2026-03-31 01:13:12,294 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:13:12,309 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1f498c06c2799bf57a8b HTTP/1.1" 200 None 2026-03-31 01:13:12,310 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:13:12,335 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12e08c06c2799bf5502b HTTP/1.1" 200 None 2026-03-31 01:13:12,337 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:13:12,365 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb1f498c06c2799bf57a8b HTTP/1.1" 200 None 2026-03-31 01:13:12,375 - lava_callback - INFO - Completed processing callback for node 69cb1f498c06c2799bf57a8b 2026-03-31 01:13:12,757 - lava_callback - INFO - Received callback for node 69cb1f428c06c2799bf57a79 INFO: 172.20.0.1:60584 - "POST /node/69cb1f428c06c2799bf57a79 HTTP/1.0" 202 Accepted 2026-03-31 01:13:12,768 - lava_callback - INFO - Starting async job submit for node 69cb1f428c06c2799bf57a79 2026-03-31 01:13:12,818 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:13:12,840 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1f428c06c2799bf57a79 HTTP/1.1" 200 None 2026-03-31 01:13:12,842 - lava_callback - DEBUG - Processing node 69cb1f428c06c2799bf57a79: baseline-nfs-arm64 2026-03-31 01:13:13,675 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 01:13:14,515 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 01:13:14,517 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cb1f428c06c2799bf57a79/log.txt.gz 2026-03-31 01:13:14,941 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:60588 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-31 01:13:15,761 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 01:13:15,764 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cb1f428c06c2799bf57a79/lava-logs.yaml 2026-03-31 01:13:15,778 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 01:13:16,035 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 01:13:16,037 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cb1f428c06c2799bf57a79/lava_callback.json.gz 2026-03-31 01:13:16,062 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:13:16,076 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1f428c06c2799bf57a79 HTTP/1.1" 200 None 2026-03-31 01:13:16,077 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:13:16,091 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12e08c06c2799bf5502b HTTP/1.1" 200 None 2026-03-31 01:13:16,093 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:13:16,153 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb1f428c06c2799bf57a79 HTTP/1.1" 200 None 2026-03-31 01:13:16,155 - lava_callback - INFO - Completed processing callback for node 69cb1f428c06c2799bf57a79 INFO: 172.20.0.1:60590 - "GET / HTTP/1.0" 200 OK 2026-03-31 01:13:16,572 - lava_callback - INFO - Received callback for node 69cb1f718c06c2799bf57b05 INFO: 172.20.0.1:60602 - "POST /node/69cb1f718c06c2799bf57b05 HTTP/1.0" 202 Accepted 2026-03-31 01:13:16,581 - lava_callback - INFO - Starting async job submit for node 69cb1f718c06c2799bf57b05 2026-03-31 01:13:16,621 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:13:16,652 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1f718c06c2799bf57b05 HTTP/1.1" 200 None 2026-03-31 01:13:16,653 - lava_callback - DEBUG - Processing node 69cb1f718c06c2799bf57b05: baseline-arm64-pengutronix 2026-03-31 01:13:17,022 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 01:13:17,301 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 01:13:17,303 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69cb1f718c06c2799bf57b05/log.txt.gz 2026-03-31 01:13:17,426 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 01:13:18,277 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 01:13:18,280 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69cb1f718c06c2799bf57b05/lava-logs.yaml 2026-03-31 01:13:18,286 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 01:13:18,519 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 01:13:18,521 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69cb1f718c06c2799bf57b05/lava_callback.json.gz 2026-03-31 01:13:18,542 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:13:18,557 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1f718c06c2799bf57b05 HTTP/1.1" 200 None 2026-03-31 01:13:18,559 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:13:18,573 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12e08c06c2799bf5502b HTTP/1.1" 200 None 2026-03-31 01:13:18,574 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:13:18,599 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb1f718c06c2799bf57b05 HTTP/1.1" 200 None 2026-03-31 01:13:18,600 - lava_callback - INFO - Completed processing callback for node 69cb1f718c06c2799bf57b05 2026-03-31 01:13:20,223 - lava_callback - INFO - Received callback for node 69cb1f448c06c2799bf57a7c INFO: 172.20.0.1:50002 - "POST /node/69cb1f448c06c2799bf57a7c HTTP/1.0" 202 Accepted 2026-03-31 01:13:20,235 - lava_callback - INFO - Starting async job submit for node 69cb1f448c06c2799bf57a7c 2026-03-31 01:13:20,285 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:13:20,316 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1f448c06c2799bf57a7c HTTP/1.1" 200 None 2026-03-31 01:13:20,317 - lava_callback - DEBUG - Processing node 69cb1f448c06c2799bf57a7c: baseline-nfs-arm64 2026-03-31 01:13:21,392 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 01:13:22,028 - lava_callback - INFO - Received callback for node 69cb1f738c06c2799bf57b08 INFO: 172.20.0.1:50012 - "POST /node/69cb1f738c06c2799bf57b08 HTTP/1.0" 202 Accepted 2026-03-31 01:13:22,039 - lava_callback - INFO - Starting async job submit for node 69cb1f738c06c2799bf57b08 2026-03-31 01:13:22,096 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:13:22,137 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1f738c06c2799bf57b08 HTTP/1.1" 200 None 2026-03-31 01:13:22,139 - lava_callback - DEBUG - Processing node 69cb1f738c06c2799bf57b08: baseline-arm64-pengutronix 2026-03-31 01:13:22,192 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 01:13:22,195 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cb1f448c06c2799bf57a7c/log.txt.gz 2026-03-31 01:13:22,802 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 01:13:23,114 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 01:13:23,690 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 01:13:23,693 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69cb1f738c06c2799bf57b08/log.txt.gz 2026-03-31 01:13:23,948 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 01:13:23,993 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 01:13:23,996 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cb1f448c06c2799bf57a7c/lava-logs.yaml 2026-03-31 01:13:24,011 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 01:13:24,912 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 01:13:24,915 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69cb1f738c06c2799bf57b08/lava-logs.yaml 2026-03-31 01:13:24,923 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 01:13:25,080 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 01:13:25,083 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cb1f448c06c2799bf57a7c/lava_callback.json.gz 2026-03-31 01:13:25,131 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:13:25,168 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1f448c06c2799bf57a7c HTTP/1.1" 200 None 2026-03-31 01:13:25,170 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:13:25,196 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12e08c06c2799bf5502b HTTP/1.1" 200 None 2026-03-31 01:13:25,198 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:13:25,230 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 01:13:25,232 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69cb1f738c06c2799bf57b08/lava_callback.json.gz 2026-03-31 01:13:25,261 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb1f448c06c2799bf57a7c HTTP/1.1" 200 None 2026-03-31 01:13:25,263 - lava_callback - INFO - Completed processing callback for node 69cb1f448c06c2799bf57a7c 2026-03-31 01:13:25,289 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:13:25,314 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1f738c06c2799bf57b08 HTTP/1.1" 200 None 2026-03-31 01:13:25,316 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:13:25,333 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12e08c06c2799bf5502b HTTP/1.1" 200 None 2026-03-31 01:13:25,335 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:13:25,369 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb1f738c06c2799bf57b08 HTTP/1.1" 200 None 2026-03-31 01:13:25,376 - lava_callback - INFO - Completed processing callback for node 69cb1f738c06c2799bf57b08 INFO: 172.20.0.1:50016 - "GET / HTTP/1.0" 200 OK 2026-03-31 01:13:27,745 - lava_callback - INFO - Received callback for node 69c87ca3f41e4fb6a3785a67 INFO: 172.20.0.1:50024 - "POST /node/69c87ca3f41e4fb6a3785a67 HTTP/1.0" 202 Accepted 2026-03-31 01:13:27,755 - lava_callback - INFO - Starting async job submit for node 69c87ca3f41e4fb6a3785a67 2026-03-31 01:13:27,756 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:13:27,783 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c87ca3f41e4fb6a3785a67 HTTP/1.1" 200 None 2026-03-31 01:13:27,785 - lava_callback - DEBUG - Processing node 69c87ca3f41e4fb6a3785a67: baseline-arm64-clabbe 2026-03-31 01:13:27,785 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c87ca3f41e4fb6a3785a67 2026-03-31 01:13:27,786 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:13:27,818 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c87ca3f41e4fb6a3785a67 HTTP/1.1" 200 None 2026-03-31 01:13:27,821 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:13:27,859 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-03-31 01:13:27,862 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:13:27,901 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c87ca3f41e4fb6a3785a67 HTTP/1.1" 200 None 2026-03-31 01:13:27,902 - lava_callback - ERROR - Error processing callback for node 69c87ca3f41e4fb6a3785a67: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-31 01:13:29,704 - lava_callback - INFO - Received callback for node 69cb1f3d8c06c2799bf57a6d INFO: 172.20.0.1:54426 - "POST /node/69cb1f3d8c06c2799bf57a6d HTTP/1.0" 202 Accepted 2026-03-31 01:13:29,714 - lava_callback - INFO - Starting async job submit for node 69cb1f3d8c06c2799bf57a6d 2026-03-31 01:13:29,756 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:13:29,787 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1f3d8c06c2799bf57a6d HTTP/1.1" 200 None 2026-03-31 01:13:29,788 - lava_callback - DEBUG - Processing node 69cb1f3d8c06c2799bf57a6d: baseline-nfs-arm64 2026-03-31 01:13:30,260 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 01:13:30,716 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:13:30,747 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 2026-03-31 01:13:30,912 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 01:13:30,914 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cb1f3d8c06c2799bf57a6d/log.txt.gz 2026-03-31 01:13:31,246 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 01:13:32,004 - lava_callback - INFO - Received callback for node 69cb1f468c06c2799bf57a85 INFO: 172.20.0.1:54432 - "POST /node/69cb1f468c06c2799bf57a85 HTTP/1.0" 202 Accepted 2026-03-31 01:13:32,018 - lava_callback - INFO - Starting async job submit for node 69cb1f468c06c2799bf57a85 2026-03-31 01:13:32,069 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:13:32,095 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1f468c06c2799bf57a85 HTTP/1.1" 200 None 2026-03-31 01:13:32,097 - lava_callback - DEBUG - Processing node 69cb1f468c06c2799bf57a85: baseline-nfs-arm64 2026-03-31 01:13:32,111 - lava_callback - INFO - Received callback for node 69cb1f748c06c2799bf57b0b 2026-03-31 01:13:32,125 - lava_callback - INFO - Starting async job submit for node 69cb1f748c06c2799bf57b0b INFO: 172.20.0.1:54438 - "POST /node/69cb1f748c06c2799bf57b0b HTTP/1.0" 202 Accepted 2026-03-31 01:13:32,206 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:13:32,246 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 01:13:32,279 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cb1f3d8c06c2799bf57a6d/lava-logs.yaml 2026-03-31 01:13:32,345 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 01:13:32,345 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1f748c06c2799bf57b0b HTTP/1.1" 200 None 2026-03-31 01:13:32,347 - lava_callback - DEBUG - Processing node 69cb1f748c06c2799bf57b0b: baseline-arm64-pengutronix 2026-03-31 01:13:33,083 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 01:13:33,086 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 01:13:33,334 - lava_callback - INFO - Received callback for node 69cb1f408c06c2799bf57a73 2026-03-31 01:13:33,350 - lava_callback - INFO - Starting async job submit for node 69cb1f408c06c2799bf57a73 INFO: 172.20.0.1:54440 - "POST /node/69cb1f408c06c2799bf57a73 HTTP/1.0" 202 Accepted 2026-03-31 01:13:33,406 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:13:33,443 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1f408c06c2799bf57a73 HTTP/1.1" 200 None 2026-03-31 01:13:33,445 - lava_callback - DEBUG - Processing node 69cb1f408c06c2799bf57a73: baseline-nfs-arm64 2026-03-31 01:13:33,617 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 01:13:33,619 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cb1f3d8c06c2799bf57a6d/lava_callback.json.gz 2026-03-31 01:13:33,760 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:13:33,817 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 01:13:33,819 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cb1f468c06c2799bf57a85/log.txt.gz 2026-03-31 01:13:33,895 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 01:13:33,895 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1f3d8c06c2799bf57a6d HTTP/1.1" 200 None 2026-03-31 01:13:33,970 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:13:33,977 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69cb1f748c06c2799bf57b0b/log.txt.gz 2026-03-31 01:13:34,188 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12e08c06c2799bf5502b HTTP/1.1" 200 None 2026-03-31 01:13:34,194 - lava_callback - INFO - Received callback for node 69cb1f658c06c2799bf57ae6 2026-03-31 01:13:34,217 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:13:34,261 - lava_callback - INFO - Starting async job submit for node 69cb1f658c06c2799bf57ae6 INFO: 172.20.0.1:54446 - "POST /node/69cb1f658c06c2799bf57ae6 HTTP/1.0" 202 Accepted 2026-03-31 01:13:34,427 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb1f3d8c06c2799bf57a6d HTTP/1.1" 200 None 2026-03-31 01:13:34,433 - lava_callback - INFO - Completed processing callback for node 69cb1f3d8c06c2799bf57a6d 2026-03-31 01:13:34,506 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:13:34,647 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1f658c06c2799bf57ae6 HTTP/1.1" 200 None 2026-03-31 01:13:34,649 - lava_callback - DEBUG - Processing node 69cb1f658c06c2799bf57ae6: baseline-arm64-broonie 2026-03-31 01:13:35,059 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 01:13:35,126 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 01:13:35,128 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 01:13:35,445 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 01:13:35,465 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cb1f408c06c2799bf57a73/log.txt.gz 2026-03-31 01:13:35,477 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 01:13:35,486 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69cb1f748c06c2799bf57b0b/lava-logs.yaml 2026-03-31 01:13:35,493 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 01:13:35,494 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 01:13:35,496 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cb1f468c06c2799bf57a85/lava-logs.yaml 2026-03-31 01:13:35,498 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 01:13:35,516 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 01:13:35,855 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 01:13:36,065 - lava_callback - INFO - Received callback for node 69cb1f578c06c2799bf57aa7 INFO: 172.20.0.1:54462 - "POST /node/69cb1f578c06c2799bf57aa7 HTTP/1.0" 202 Accepted 2026-03-31 01:13:36,075 - lava_callback - INFO - Starting async job submit for node 69cb1f578c06c2799bf57aa7 2026-03-31 01:13:36,121 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:13:36,158 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1f578c06c2799bf57aa7 HTTP/1.1" 200 None 2026-03-31 01:13:36,160 - lava_callback - DEBUG - Processing node 69cb1f578c06c2799bf57aa7: baseline-arm64-broonie 2026-03-31 01:13:36,502 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 01:13:36,504 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cb1f658c06c2799bf57ae6/log.txt.gz INFO: 172.20.0.1:54470 - "GET / HTTP/1.0" 200 OK 2026-03-31 01:13:36,628 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 01:13:36,645 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 01:13:36,657 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cb1f468c06c2799bf57a85/lava_callback.json.gz 2026-03-31 01:13:36,669 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69cb1f748c06c2799bf57b0b/lava_callback.json.gz 2026-03-31 01:13:36,780 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:13:36,790 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 01:13:36,811 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:13:36,818 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cb1f408c06c2799bf57a73/lava-logs.yaml 2026-03-31 01:13:36,826 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 01:13:36,862 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1f748c06c2799bf57b0b HTTP/1.1" 200 None 2026-03-31 01:13:36,879 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:13:36,881 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 01:13:36,889 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1f468c06c2799bf57a85 HTTP/1.1" 200 None 2026-03-31 01:13:36,894 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:13:36,959 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12e08c06c2799bf5502b HTTP/1.1" 200 None 2026-03-31 01:13:36,962 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:13:36,971 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12e08c06c2799bf5502b HTTP/1.1" 200 None 2026-03-31 01:13:36,974 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:13:37,016 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 01:13:37,054 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb1f468c06c2799bf57a85 HTTP/1.1" 200 None 2026-03-31 01:13:37,056 - lava_callback - INFO - Completed processing callback for node 69cb1f468c06c2799bf57a85 2026-03-31 01:13:37,084 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb1f748c06c2799bf57b0b HTTP/1.1" 200 None 2026-03-31 01:13:37,085 - lava_callback - INFO - Completed processing callback for node 69cb1f748c06c2799bf57b0b 2026-03-31 01:13:38,036 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 01:13:38,038 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cb1f578c06c2799bf57aa7/log.txt.gz 2026-03-31 01:13:38,139 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 01:13:38,145 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 01:13:38,184 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cb1f408c06c2799bf57a73/lava_callback.json.gz 2026-03-31 01:13:38,194 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cb1f658c06c2799bf57ae6/lava-logs.yaml 2026-03-31 01:13:38,238 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:13:38,260 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 01:13:38,261 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 01:13:38,291 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1f408c06c2799bf57a73 HTTP/1.1" 200 None 2026-03-31 01:13:38,293 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:13:38,314 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12e08c06c2799bf5502b HTTP/1.1" 200 None 2026-03-31 01:13:38,316 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:13:38,357 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb1f408c06c2799bf57a73 HTTP/1.1" 200 None 2026-03-31 01:13:38,359 - lava_callback - INFO - Completed processing callback for node 69cb1f408c06c2799bf57a73 2026-03-31 01:13:39,209 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 01:13:39,211 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cb1f658c06c2799bf57ae6/lava_callback.json.gz 2026-03-31 01:13:39,259 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:13:39,286 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1f658c06c2799bf57ae6 HTTP/1.1" 200 None 2026-03-31 01:13:39,289 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:13:39,292 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 01:13:39,294 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cb1f578c06c2799bf57aa7/lava-logs.yaml 2026-03-31 01:13:39,305 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 01:13:39,320 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12e08c06c2799bf5502b HTTP/1.1" 200 None 2026-03-31 01:13:39,323 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:13:39,357 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb1f658c06c2799bf57ae6 HTTP/1.1" 200 None 2026-03-31 01:13:39,359 - lava_callback - INFO - Completed processing callback for node 69cb1f658c06c2799bf57ae6 2026-03-31 01:13:39,603 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 01:13:39,606 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cb1f578c06c2799bf57aa7/lava_callback.json.gz 2026-03-31 01:13:39,652 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:13:39,696 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1f578c06c2799bf57aa7 HTTP/1.1" 200 None 2026-03-31 01:13:39,699 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:13:39,723 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12e08c06c2799bf5502b HTTP/1.1" 200 None 2026-03-31 01:13:39,725 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:13:39,777 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb1f578c06c2799bf57aa7 HTTP/1.1" 200 None 2026-03-31 01:13:39,779 - lava_callback - INFO - Completed processing callback for node 69cb1f578c06c2799bf57aa7 INFO: 172.20.0.1:33288 - "GET / HTTP/1.0" 200 OK 2026-03-31 01:13:44,451 - lava_callback - INFO - Received callback for node 69cb1f338c06c2799bf57a3c INFO: 172.20.0.1:33296 - "POST /node/69cb1f338c06c2799bf57a3c HTTP/1.0" 202 Accepted 2026-03-31 01:13:44,461 - lava_callback - INFO - Starting async job submit for node 69cb1f338c06c2799bf57a3c 2026-03-31 01:13:44,514 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:13:44,541 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1f338c06c2799bf57a3c HTTP/1.1" 200 None 2026-03-31 01:13:44,543 - lava_callback - DEBUG - Processing node 69cb1f338c06c2799bf57a3c: baseline-arm64 2026-03-31 01:13:45,211 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 01:13:45,947 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 01:13:45,949 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb1f338c06c2799bf57a3c/log.txt.gz 2026-03-31 01:13:46,430 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:33300 - "GET / HTTP/1.0" 200 OK 2026-03-31 01:13:47,231 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 01:13:47,234 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb1f338c06c2799bf57a3c/lava-logs.yaml 2026-03-31 01:13:47,244 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 01:13:47,810 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 01:13:47,813 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb1f338c06c2799bf57a3c/lava_callback.json.gz 2026-03-31 01:13:47,896 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:13:47,919 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1f338c06c2799bf57a3c HTTP/1.1" 200 None 2026-03-31 01:13:47,921 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:13:47,951 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12e08c06c2799bf5502b HTTP/1.1" 200 None 2026-03-31 01:13:47,953 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:13:47,988 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb1f338c06c2799bf57a3c HTTP/1.1" 200 None 2026-03-31 01:13:48,095 - lava_callback - INFO - Completed processing callback for node 69cb1f338c06c2799bf57a3c 2026-03-31 01:13:52,696 - lava_callback - INFO - Received callback for node 69cb1f5d8c06c2799bf57acb INFO: 172.20.0.1:51394 - "POST /node/69cb1f5d8c06c2799bf57acb HTTP/1.0" 202 Accepted 2026-03-31 01:13:52,708 - lava_callback - INFO - Starting async job submit for node 69cb1f5d8c06c2799bf57acb 2026-03-31 01:13:52,744 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:13:52,758 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1f5d8c06c2799bf57acb HTTP/1.1" 200 None 2026-03-31 01:13:52,760 - lava_callback - DEBUG - Processing node 69cb1f5d8c06c2799bf57acb: baseline-arm64-broonie 2026-03-31 01:13:53,245 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 01:13:53,733 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 01:13:53,736 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cb1f5d8c06c2799bf57acb/log.txt.gz 2026-03-31 01:13:53,950 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 01:13:54,193 - lava_callback - INFO - Received callback for node 69cb1f638c06c2799bf57ae3 INFO: 172.20.0.1:51404 - "POST /node/69cb1f638c06c2799bf57ae3 HTTP/1.0" 202 Accepted 2026-03-31 01:13:54,203 - lava_callback - INFO - Starting async job submit for node 69cb1f638c06c2799bf57ae3 2026-03-31 01:13:54,250 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:13:54,271 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1f638c06c2799bf57ae3 HTTP/1.1" 200 None 2026-03-31 01:13:54,272 - lava_callback - DEBUG - Processing node 69cb1f638c06c2799bf57ae3: baseline-arm64-broonie 2026-03-31 01:13:54,624 - lava_callback - INFO - Received callback for node 69cb1f678c06c2799bf57aea INFO: 172.20.0.1:51410 - "POST /node/69cb1f678c06c2799bf57aea HTTP/1.0" 202 Accepted 2026-03-31 01:13:54,645 - lava_callback - INFO - Starting async job submit for node 69cb1f678c06c2799bf57aea 2026-03-31 01:13:54,733 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:13:54,783 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1f678c06c2799bf57aea HTTP/1.1" 200 None 2026-03-31 01:13:54,785 - lava_callback - DEBUG - Processing node 69cb1f678c06c2799bf57aea: baseline-arm64-broonie 2026-03-31 01:13:54,801 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 01:13:54,983 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 01:13:54,986 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cb1f5d8c06c2799bf57acb/lava-logs.yaml 2026-03-31 01:13:55,009 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 01:13:55,186 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 01:13:55,188 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cb1f638c06c2799bf57ae3/log.txt.gz 2026-03-31 01:13:55,436 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 01:13:55,598 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 01:13:55,970 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 01:13:55,972 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cb1f5d8c06c2799bf57acb/lava_callback.json.gz 2026-03-31 01:13:56,018 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:13:56,054 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1f5d8c06c2799bf57acb HTTP/1.1" 200 None 2026-03-31 01:13:56,056 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:13:56,104 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12e08c06c2799bf5502b HTTP/1.1" 200 None 2026-03-31 01:13:56,107 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:13:56,153 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb1f5d8c06c2799bf57acb HTTP/1.1" 200 None 2026-03-31 01:13:56,155 - lava_callback - INFO - Completed processing callback for node 69cb1f5d8c06c2799bf57acb 2026-03-31 01:13:56,303 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 01:13:56,305 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cb1f678c06c2799bf57aea/log.txt.gz INFO: 172.20.0.1:51412 - "GET / HTTP/1.0" 200 OK 2026-03-31 01:13:56,560 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 01:13:56,592 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cb1f638c06c2799bf57ae3/lava-logs.yaml 2026-03-31 01:13:56,633 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 01:13:56,633 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 01:13:57,504 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 01:13:57,506 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cb1f638c06c2799bf57ae3/lava_callback.json.gz 2026-03-31 01:13:57,548 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:13:57,563 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 01:13:57,566 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cb1f678c06c2799bf57aea/lava-logs.yaml 2026-03-31 01:13:57,569 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1f638c06c2799bf57ae3 HTTP/1.1" 200 None 2026-03-31 01:13:57,572 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:13:57,575 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 01:13:57,587 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12e08c06c2799bf5502b HTTP/1.1" 200 None 2026-03-31 01:13:57,589 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:13:57,619 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb1f638c06c2799bf57ae3 HTTP/1.1" 200 None 2026-03-31 01:13:57,621 - lava_callback - INFO - Completed processing callback for node 69cb1f638c06c2799bf57ae3 2026-03-31 01:13:58,400 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 01:13:58,402 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cb1f678c06c2799bf57aea/lava_callback.json.gz 2026-03-31 01:13:58,457 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:13:58,484 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1f678c06c2799bf57aea HTTP/1.1" 200 None 2026-03-31 01:13:58,486 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:13:58,524 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12e08c06c2799bf5502b HTTP/1.1" 200 None 2026-03-31 01:13:58,527 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:13:58,581 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb1f678c06c2799bf57aea HTTP/1.1" 200 None 2026-03-31 01:13:58,583 - lava_callback - INFO - Completed processing callback for node 69cb1f678c06c2799bf57aea 2026-03-31 01:13:59,362 - lava_callback - INFO - Received callback for node 69cb1bb98c06c2799bf55222 INFO: 172.20.0.1:51422 - "POST /node/69cb1bb98c06c2799bf55222 HTTP/1.0" 202 Accepted 2026-03-31 01:13:59,375 - lava_callback - INFO - Starting async job submit for node 69cb1bb98c06c2799bf55222 2026-03-31 01:13:59,459 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:13:59,482 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1bb98c06c2799bf55222 HTTP/1.1" 200 None 2026-03-31 01:13:59,483 - lava_callback - DEBUG - Processing node 69cb1bb98c06c2799bf55222: baseline-arm 2026-03-31 01:13:59,788 - lava_callback - INFO - Received callback for node 69cb1f6c8c06c2799bf57afc INFO: 172.20.0.1:46198 - "POST /node/69cb1f6c8c06c2799bf57afc HTTP/1.0" 202 Accepted 2026-03-31 01:13:59,819 - lava_callback - INFO - Starting async job submit for node 69cb1f6c8c06c2799bf57afc 2026-03-31 01:13:59,894 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:13:59,983 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1f6c8c06c2799bf57afc HTTP/1.1" 200 None 2026-03-31 01:13:59,995 - lava_callback - DEBUG - Processing node 69cb1f6c8c06c2799bf57afc: baseline-arm64-clabbe 2026-03-31 01:14:00,754 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:14:00,791 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 01:14:00,825 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 2026-03-31 01:14:00,948 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 01:14:01,045 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 01:14:01,046 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-clabbe-69cb1f6c8c06c2799bf57afc/log.txt.gz 2026-03-31 01:14:01,117 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 01:14:01,156 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69cb1bb98c06c2799bf55222/log.txt.gz 2026-03-31 01:14:01,268 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 01:14:01,780 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 01:14:02,343 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 01:14:02,345 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-clabbe-69cb1f6c8c06c2799bf57afc/lava-logs.yaml 2026-03-31 01:14:02,352 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 01:14:02,389 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 01:14:02,392 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69cb1bb98c06c2799bf55222/lava-logs.yaml 2026-03-31 01:14:02,410 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 01:14:02,932 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 01:14:02,934 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-clabbe-69cb1f6c8c06c2799bf57afc/lava_callback.json.gz 2026-03-31 01:14:02,986 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:14:03,012 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1f6c8c06c2799bf57afc HTTP/1.1" 200 None 2026-03-31 01:14:03,014 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:14:03,035 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12e08c06c2799bf5502b HTTP/1.1" 200 None 2026-03-31 01:14:03,037 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:14:03,081 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb1f6c8c06c2799bf57afc HTTP/1.1" 200 None 2026-03-31 01:14:03,083 - lava_callback - INFO - Completed processing callback for node 69cb1f6c8c06c2799bf57afc 2026-03-31 01:14:03,508 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 01:14:03,510 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69cb1bb98c06c2799bf55222/lava_callback.json.gz DEBUG: Setup failed for 69cb1bb98c06c2799bf55222. Transitting job node result: incomplete -> fail 2026-03-31 01:14:03,666 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:14:03,680 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1bb98c06c2799bf55222 HTTP/1.1" 200 None 2026-03-31 01:14:03,682 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:14:03,695 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12cb8c06c2799bf55018 HTTP/1.1" 200 None 2026-03-31 01:14:03,697 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:14:03,727 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb1bb98c06c2799bf55222 HTTP/1.1" 200 None 2026-03-31 01:14:03,857 - lava_callback - INFO - Completed processing callback for node 69cb1bb98c06c2799bf55222 INFO: 172.20.0.1:46210 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42442 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42452 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35802 - "GET / HTTP/1.0" 200 OK 2026-03-31 01:14:27,594 - lava_callback - INFO - Received callback for node 69c87ceef41e4fb6a3785a7b INFO: 172.20.0.1:35816 - "POST /node/69c87ceef41e4fb6a3785a7b HTTP/1.0" 202 Accepted 2026-03-31 01:14:27,603 - lava_callback - INFO - Starting async job submit for node 69c87ceef41e4fb6a3785a7b 2026-03-31 01:14:27,604 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:14:27,648 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c87ceef41e4fb6a3785a7b HTTP/1.1" 200 None 2026-03-31 01:14:27,650 - lava_callback - DEBUG - Processing node 69c87ceef41e4fb6a3785a7b: baseline-arm64-clabbe 2026-03-31 01:14:27,650 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c87ceef41e4fb6a3785a7b 2026-03-31 01:14:27,651 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:14:27,693 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c87ceef41e4fb6a3785a7b HTTP/1.1" 200 None 2026-03-31 01:14:27,697 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:14:27,755 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-03-31 01:14:27,757 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:14:27,790 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c87ceef41e4fb6a3785a7b HTTP/1.1" 200 None 2026-03-31 01:14:27,791 - lava_callback - ERROR - Error processing callback for node 69c87ceef41e4fb6a3785a7b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-31 01:14:28,244 - lava_callback - INFO - Received callback for node 69cb1f768c06c2799bf57b0e INFO: 172.20.0.1:35820 - "POST /node/69cb1f768c06c2799bf57b0e HTTP/1.0" 202 Accepted 2026-03-31 01:14:28,253 - lava_callback - INFO - Starting async job submit for node 69cb1f768c06c2799bf57b0e 2026-03-31 01:14:28,298 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:14:28,331 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1f768c06c2799bf57b0e HTTP/1.1" 200 None 2026-03-31 01:14:28,332 - lava_callback - DEBUG - Processing node 69cb1f768c06c2799bf57b0e: baseline-arm64-foundriesio 2026-03-31 01:14:28,717 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 01:14:28,999 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 01:14:29,001 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69cb1f768c06c2799bf57b0e/log.txt.gz 2026-03-31 01:14:29,206 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 01:14:30,166 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 01:14:30,168 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69cb1f768c06c2799bf57b0e/lava-logs.yaml 2026-03-31 01:14:30,176 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 01:14:30,326 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 01:14:30,328 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69cb1f768c06c2799bf57b0e/lava_callback.json.gz 2026-03-31 01:14:30,350 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:14:30,365 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1f768c06c2799bf57b0e HTTP/1.1" 200 None 2026-03-31 01:14:30,366 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:14:30,379 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12e08c06c2799bf5502b HTTP/1.1" 200 None 2026-03-31 01:14:30,381 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:14:30,412 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb1f768c06c2799bf57b0e HTTP/1.1" 200 None 2026-03-31 01:14:30,414 - lava_callback - INFO - Completed processing callback for node 69cb1f768c06c2799bf57b0e 2026-03-31 01:14:30,828 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:14:30,856 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:34116 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43844 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43860 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52272 - "GET / HTTP/1.0" 200 OK 2026-03-31 01:14:57,359 - lava_callback - INFO - Received callback for node 69cb1f2e8c06c2799bf57a30 INFO: 172.20.0.1:52284 - "POST /node/69cb1f2e8c06c2799bf57a30 HTTP/1.0" 202 Accepted 2026-03-31 01:14:57,368 - lava_callback - INFO - Starting async job submit for node 69cb1f2e8c06c2799bf57a30 2026-03-31 01:14:57,387 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:14:57,401 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1f2e8c06c2799bf57a30 HTTP/1.1" 200 None 2026-03-31 01:14:57,403 - lava_callback - DEBUG - Processing node 69cb1f2e8c06c2799bf57a30: baseline-arm64 2026-03-31 01:14:57,935 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 01:14:58,077 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 01:14:58,079 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb1f2e8c06c2799bf57a30/log.txt.gz 2026-03-31 01:14:58,366 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 01:14:58,815 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 01:14:58,818 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb1f2e8c06c2799bf57a30/lava-logs.yaml 2026-03-31 01:14:58,827 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 01:14:58,989 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 01:14:58,991 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb1f2e8c06c2799bf57a30/lava_callback.json.gz 2026-03-31 01:14:59,011 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:14:59,027 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1f2e8c06c2799bf57a30 HTTP/1.1" 200 None 2026-03-31 01:14:59,029 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:14:59,050 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12e08c06c2799bf5502b HTTP/1.1" 200 None 2026-03-31 01:14:59,052 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:14:59,086 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb1f2e8c06c2799bf57a30 HTTP/1.1" 200 None 2026-03-31 01:14:59,105 - lava_callback - INFO - Completed processing callback for node 69cb1f2e8c06c2799bf57a30 2026-03-31 01:15:00,859 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:15:00,893 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:54812 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48576 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48580 - "GET / HTTP/1.0" 200 OK 2026-03-31 01:15:22,088 - lava_callback - INFO - Received callback for node 69c87d26f41e4fb6a3785a88 INFO: 172.20.0.1:38374 - "POST /node/69c87d26f41e4fb6a3785a88 HTTP/1.0" 202 Accepted 2026-03-31 01:15:22,097 - lava_callback - INFO - Starting async job submit for node 69c87d26f41e4fb6a3785a88 2026-03-31 01:15:22,098 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:15:22,121 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c87d26f41e4fb6a3785a88 HTTP/1.1" 200 None 2026-03-31 01:15:22,123 - lava_callback - DEBUG - Processing node 69c87d26f41e4fb6a3785a88: baseline-arm-baylibre 2026-03-31 01:15:22,123 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c87d26f41e4fb6a3785a88 2026-03-31 01:15:22,124 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:15:22,156 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c87d26f41e4fb6a3785a88 HTTP/1.1" 200 None 2026-03-31 01:15:22,158 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:15:22,181 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086a5ce0c9bc43bccb62b HTTP/1.1" 200 None 2026-03-31 01:15:22,183 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:15:22,216 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c87d26f41e4fb6a3785a88 HTTP/1.1" 200 None 2026-03-31 01:15:22,217 - lava_callback - ERROR - Error processing callback for node 69c87d26f41e4fb6a3785a88: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:38386 - "GET / HTTP/1.0" 200 OK 2026-03-31 01:15:34,811 - lava_callback - INFO - Received callback for node 69cb1f5f8c06c2799bf57ad6 INFO: 172.20.0.1:36030 - "POST /node/69cb1f5f8c06c2799bf57ad6 HTTP/1.0" 202 Accepted 2026-03-31 01:15:34,821 - lava_callback - INFO - Starting async job submit for node 69cb1f5f8c06c2799bf57ad6 2026-03-31 01:15:34,860 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:15:34,892 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1f5f8c06c2799bf57ad6 HTTP/1.1" 200 None 2026-03-31 01:15:34,893 - lava_callback - DEBUG - Processing node 69cb1f5f8c06c2799bf57ad6: baseline-arm64-broonie 2026-03-31 01:15:35,312 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 01:15:35,624 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 01:15:35,626 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cb1f5f8c06c2799bf57ad6/log.txt.gz 2026-03-31 01:15:35,872 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 01:15:36,267 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 01:15:36,269 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cb1f5f8c06c2799bf57ad6/lava-logs.yaml 2026-03-31 01:15:36,279 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 01:15:36,471 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 01:15:36,473 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cb1f5f8c06c2799bf57ad6/lava_callback.json.gz 2026-03-31 01:15:36,497 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.20.0.1:36046 - "GET / HTTP/1.0" 200 OK 2026-03-31 01:15:36,514 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1f5f8c06c2799bf57ad6 HTTP/1.1" 200 None 2026-03-31 01:15:36,516 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:15:36,530 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12e08c06c2799bf5502b HTTP/1.1" 200 None 2026-03-31 01:15:36,533 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:15:36,566 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb1f5f8c06c2799bf57ad6 HTTP/1.1" 200 None 2026-03-31 01:15:36,567 - lava_callback - INFO - Completed processing callback for node 69cb1f5f8c06c2799bf57ad6 INFO: 172.20.0.1:47640 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47654 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41664 - "GET / HTTP/1.0" 200 OK 2026-03-31 01:16:00,896 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:16:00,912 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-31 01:16:04,808 - lava_callback - INFO - Received callback for node 69cb1f3f8c06c2799bf57a70 INFO: 172.20.0.1:39536 - "POST /node/69cb1f3f8c06c2799bf57a70 HTTP/1.0" 202 Accepted 2026-03-31 01:16:04,814 - lava_callback - INFO - Starting async job submit for node 69cb1f3f8c06c2799bf57a70 2026-03-31 01:16:04,833 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:16:04,847 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1f3f8c06c2799bf57a70 HTTP/1.1" 200 None 2026-03-31 01:16:04,848 - lava_callback - DEBUG - Processing node 69cb1f3f8c06c2799bf57a70: baseline-nfs-arm64 2026-03-31 01:16:05,248 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 01:16:05,552 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 01:16:05,554 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cb1f3f8c06c2799bf57a70/log.txt.gz 2026-03-31 01:16:05,802 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 01:16:06,001 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 01:16:06,003 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cb1f3f8c06c2799bf57a70/lava-logs.yaml 2026-03-31 01:16:06,013 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 01:16:06,162 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 01:16:06,164 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cb1f3f8c06c2799bf57a70/lava_callback.json.gz 2026-03-31 01:16:06,182 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:16:06,198 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1f3f8c06c2799bf57a70 HTTP/1.1" 200 None 2026-03-31 01:16:06,200 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:16:06,214 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12e08c06c2799bf5502b HTTP/1.1" 200 None 2026-03-31 01:16:06,215 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:16:06,241 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb1f3f8c06c2799bf57a70 HTTP/1.1" 200 None 2026-03-31 01:16:06,243 - lava_callback - INFO - Completed processing callback for node 69cb1f3f8c06c2799bf57a70 INFO: 172.20.0.1:39540 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53834 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53846 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49522 - "GET / HTTP/1.0" 200 OK 2026-03-31 01:16:30,915 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:16:30,930 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-31 01:16:34,440 - lava_callback - INFO - Received callback for node 69cb203a8c06c2799bf57c2c INFO: 172.20.0.1:43170 - "POST /node/69cb203a8c06c2799bf57c2c HTTP/1.0" 202 Accepted 2026-03-31 01:16:34,446 - lava_callback - INFO - Starting async job submit for node 69cb203a8c06c2799bf57c2c 2026-03-31 01:16:34,463 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:16:34,478 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb203a8c06c2799bf57c2c HTTP/1.1" 200 None 2026-03-31 01:16:34,479 - lava_callback - DEBUG - Processing node 69cb203a8c06c2799bf57c2c: baseline-arm-pengutronix 2026-03-31 01:16:34,632 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 01:16:34,778 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 01:16:34,780 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cb203a8c06c2799bf57c2c/log.txt.gz 2026-03-31 01:16:34,869 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 01:16:35,110 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 01:16:35,112 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cb203a8c06c2799bf57c2c/lava-logs.yaml 2026-03-31 01:16:35,116 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 01:16:35,243 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 01:16:35,245 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cb203a8c06c2799bf57c2c/lava_callback.json.gz 2026-03-31 01:16:35,264 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:16:35,279 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb203a8c06c2799bf57c2c HTTP/1.1" 200 None 2026-03-31 01:16:35,281 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:16:35,294 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12cb8c06c2799bf55018 HTTP/1.1" 200 None 2026-03-31 01:16:35,295 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:16:35,329 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb203a8c06c2799bf57c2c HTTP/1.1" 200 None 2026-03-31 01:16:35,335 - lava_callback - INFO - Completed processing callback for node 69cb203a8c06c2799bf57c2c INFO: 172.20.0.1:43178 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58720 - "GET / HTTP/1.0" 200 OK 2026-03-31 01:16:40,735 - lava_callback - INFO - Received callback for node 69cb1f798c06c2799bf57b2b INFO: 172.20.0.1:58730 - "POST /node/69cb1f798c06c2799bf57b2b HTTP/1.0" 202 Accepted 2026-03-31 01:16:40,743 - lava_callback - INFO - Starting async job submit for node 69cb1f798c06c2799bf57b2b 2026-03-31 01:16:40,786 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:16:40,810 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1f798c06c2799bf57b2b HTTP/1.1" 200 None 2026-03-31 01:16:40,811 - lava_callback - DEBUG - Processing node 69cb1f798c06c2799bf57b2b: baseline-arm64-foundriesio 2026-03-31 01:16:41,588 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 01:16:41,735 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 01:16:41,737 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69cb1f798c06c2799bf57b2b/log.txt.gz 2026-03-31 01:16:42,205 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 01:16:42,416 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 01:16:42,419 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69cb1f798c06c2799bf57b2b/lava-logs.yaml 2026-03-31 01:16:42,438 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 01:16:42,607 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 01:16:42,609 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69cb1f798c06c2799bf57b2b/lava_callback.json.gz DEBUG: 69cb1f798c06c2799bf57b2b Transitting job node result: pass -> incomplete 2026-03-31 01:16:42,648 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:16:42,663 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1f798c06c2799bf57b2b HTTP/1.1" 200 None 2026-03-31 01:16:42,665 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:16:42,677 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12e08c06c2799bf5502b HTTP/1.1" 200 None 2026-03-31 01:16:42,679 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:16:42,703 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb1f798c06c2799bf57b2b HTTP/1.1" 200 None 2026-03-31 01:16:42,704 - lava_callback - INFO - Completed processing callback for node 69cb1f798c06c2799bf57b2b INFO: 172.20.0.1:58742 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54288 - "GET / HTTP/1.0" 200 OK 2026-03-31 01:17:00,933 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:17:00,948 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:40960 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45142 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45146 - "GET / HTTP/1.0" 200 OK 2026-03-31 01:17:22,197 - lava_callback - INFO - Received callback for node 69c87d90f41e4fb6a3785aac INFO: 172.20.0.1:54594 - "POST /node/69c87d90f41e4fb6a3785aac HTTP/1.0" 202 Accepted 2026-03-31 01:17:22,203 - lava_callback - INFO - Starting async job submit for node 69c87d90f41e4fb6a3785aac 2026-03-31 01:17:22,204 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:17:22,217 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c87d90f41e4fb6a3785aac HTTP/1.1" 200 None 2026-03-31 01:17:22,218 - lava_callback - DEBUG - Processing node 69c87d90f41e4fb6a3785aac: baseline-arm-baylibre 2026-03-31 01:17:22,218 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c87d90f41e4fb6a3785aac 2026-03-31 01:17:22,219 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:17:22,232 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c87d90f41e4fb6a3785aac HTTP/1.1" 200 None 2026-03-31 01:17:22,234 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:17:22,247 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086a5ce0c9bc43bccb62b HTTP/1.1" 200 None 2026-03-31 01:17:22,248 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:17:22,269 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c87d90f41e4fb6a3785aac HTTP/1.1" 200 None 2026-03-31 01:17:22,274 - lava_callback - ERROR - Error processing callback for node 69c87d90f41e4fb6a3785aac: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:54610 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53578 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37620 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37628 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44508 - "GET / HTTP/1.0" 200 OK 2026-03-31 01:18:02,161 - lava_callback - INFO - Received callback for node 69c87db5f41e4fb6a3785ab8 INFO: 172.20.0.1:48762 - "POST /node/69c87db5f41e4fb6a3785ab8 HTTP/1.0" 202 Accepted 2026-03-31 01:18:02,167 - lava_callback - INFO - Starting async job submit for node 69c87db5f41e4fb6a3785ab8 2026-03-31 01:18:02,167 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:18:02,182 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c87db5f41e4fb6a3785ab8 HTTP/1.1" 200 None 2026-03-31 01:18:02,183 - lava_callback - DEBUG - Processing node 69c87db5f41e4fb6a3785ab8: baseline-arm-baylibre 2026-03-31 01:18:02,183 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c87db5f41e4fb6a3785ab8 2026-03-31 01:18:02,184 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:18:02,197 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c87db5f41e4fb6a3785ab8 HTTP/1.1" 200 None 2026-03-31 01:18:02,198 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:18:02,211 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086a5ce0c9bc43bccb62b HTTP/1.1" 200 None 2026-03-31 01:18:02,213 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:18:02,234 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c87db5f41e4fb6a3785ab8 HTTP/1.1" 200 None 2026-03-31 01:18:02,239 - lava_callback - ERROR - Error processing callback for node 69c87db5f41e4fb6a3785ab8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:48778 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49570 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49578 - "GET / HTTP/1.0" 200 OK 2026-03-31 01:18:22,128 - lava_callback - INFO - Received callback for node 69c87dd9f41e4fb6a3785ac3 INFO: 172.20.0.1:34622 - "POST /node/69c87dd9f41e4fb6a3785ac3 HTTP/1.0" 202 Accepted 2026-03-31 01:18:22,134 - lava_callback - INFO - Starting async job submit for node 69c87dd9f41e4fb6a3785ac3 2026-03-31 01:18:22,135 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:18:22,154 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c87dd9f41e4fb6a3785ac3 HTTP/1.1" 200 None 2026-03-31 01:18:22,155 - lava_callback - DEBUG - Processing node 69c87dd9f41e4fb6a3785ac3: baseline-arm-baylibre 2026-03-31 01:18:22,155 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c87dd9f41e4fb6a3785ac3 2026-03-31 01:18:22,156 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:18:22,169 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c87dd9f41e4fb6a3785ac3 HTTP/1.1" 200 None 2026-03-31 01:18:22,170 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:18:22,184 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086a5ce0c9bc43bccb62b HTTP/1.1" 200 None 2026-03-31 01:18:22,186 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:18:22,209 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c87dd9f41e4fb6a3785ac3 HTTP/1.1" 200 None 2026-03-31 01:18:22,211 - lava_callback - ERROR - Error processing callback for node 69c87dd9f41e4fb6a3785ac3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:34626 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46420 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52094 - "GET / HTTP/1.0" 200 OK 2026-03-31 01:18:42,086 - lava_callback - INFO - Received callback for node 69cb1f3c8c06c2799bf57a6a INFO: 172.20.0.1:52108 - "POST /node/69cb1f3c8c06c2799bf57a6a HTTP/1.0" 202 Accepted 2026-03-31 01:18:42,093 - lava_callback - INFO - Starting async job submit for node 69cb1f3c8c06c2799bf57a6a 2026-03-31 01:18:42,129 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:18:42,149 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1f3c8c06c2799bf57a6a HTTP/1.1" 200 None 2026-03-31 01:18:42,149 - lava_callback - DEBUG - Processing node 69cb1f3c8c06c2799bf57a6a: baseline-arm64 2026-03-31 01:18:42,546 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 01:18:42,696 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 01:18:42,698 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb1f3c8c06c2799bf57a6a/log.txt.gz 2026-03-31 01:18:42,908 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 01:18:43,147 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 01:18:43,149 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb1f3c8c06c2799bf57a6a/lava-logs.yaml 2026-03-31 01:18:43,158 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 01:18:43,308 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 01:18:43,310 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb1f3c8c06c2799bf57a6a/lava_callback.json.gz DEBUG: Setup failed for 69cb1f3c8c06c2799bf57a6a. Transitting job node result: incomplete -> fail 2026-03-31 01:18:43,383 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:18:43,397 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1f3c8c06c2799bf57a6a HTTP/1.1" 200 None 2026-03-31 01:18:43,398 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:18:43,413 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12e08c06c2799bf5502b HTTP/1.1" 200 None 2026-03-31 01:18:43,414 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:18:43,437 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb1f3c8c06c2799bf57a6a HTTP/1.1" 200 None 2026-03-31 01:18:43,527 - lava_callback - INFO - Completed processing callback for node 69cb1f3c8c06c2799bf57a6a INFO: 172.20.0.1:52124 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37812 - "GET / HTTP/1.0" 200 OK 2026-03-31 01:19:00,950 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:19:00,966 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:32948 - "GET / HTTP/1.0" 200 OK 2026-03-31 01:19:07,822 - lava_callback - INFO - Received callback for node 69c87dfef41e4fb6a3785ad3 INFO: 172.20.0.1:32962 - "POST /node/69c87dfef41e4fb6a3785ad3 HTTP/1.0" 202 Accepted 2026-03-31 01:19:07,828 - lava_callback - INFO - Starting async job submit for node 69c87dfef41e4fb6a3785ad3 2026-03-31 01:19:07,828 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:19:07,854 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c87dfef41e4fb6a3785ad3 HTTP/1.1" 200 None 2026-03-31 01:19:07,856 - lava_callback - DEBUG - Processing node 69c87dfef41e4fb6a3785ad3: baseline-arm-clabbe 2026-03-31 01:19:07,856 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c87dfef41e4fb6a3785ad3 2026-03-31 01:19:07,857 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:19:07,870 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c87dfef41e4fb6a3785ad3 HTTP/1.1" 200 None 2026-03-31 01:19:07,872 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:19:07,886 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086a5ce0c9bc43bccb62b HTTP/1.1" 200 None 2026-03-31 01:19:07,887 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:19:07,909 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c87dfef41e4fb6a3785ad3 HTTP/1.1" 200 None 2026-03-31 01:19:07,915 - lava_callback - ERROR - Error processing callback for node 69c87dfef41e4fb6a3785ad3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:37008 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-31 01:19:16,032 - lava_callback - INFO - Received callback for node 69cb1d058c06c2799bf553d6 INFO: 172.20.0.1:37022 - "POST /node/69cb1d058c06c2799bf553d6 HTTP/1.0" 202 Accepted 2026-03-31 01:19:16,038 - lava_callback - INFO - Starting async job submit for node 69cb1d058c06c2799bf553d6 2026-03-31 01:19:16,096 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:19:16,110 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1d058c06c2799bf553d6 HTTP/1.1" 200 None 2026-03-31 01:19:16,111 - lava_callback - DEBUG - Processing node 69cb1d058c06c2799bf553d6: ltp-smoke INFO: 172.20.0.1:37024 - "GET / HTTP/1.0" 200 OK 2026-03-31 01:19:16,615 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 01:19:16,729 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 01:19:16,731 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoke-69cb1d058c06c2799bf553d6/log.txt.gz 2026-03-31 01:19:17,009 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 01:19:17,196 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 01:19:17,198 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-smoke-69cb1d058c06c2799bf553d6/lava-logs.yaml 2026-03-31 01:19:17,209 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 01:19:17,383 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 01:19:17,385 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoke-69cb1d058c06c2799bf553d6/lava_callback.json.gz 2026-03-31 01:19:17,572 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:19:17,589 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1d058c06c2799bf553d6 HTTP/1.1" 200 None 2026-03-31 01:19:17,591 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:19:17,606 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12d38c06c2799bf55022 HTTP/1.1" 200 None 2026-03-31 01:19:17,608 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:19:17,632 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb1d058c06c2799bf553d6 HTTP/1.1" 200 None 2026-03-31 01:19:17,762 - lava_callback - INFO - Completed processing callback for node 69cb1d058c06c2799bf553d6 INFO: 172.20.0.1:42262 - "GET / HTTP/1.0" 200 OK 2026-03-31 01:19:30,969 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:19:30,985 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-31 01:19:32,160 - lava_callback - INFO - Received callback for node 69cb1f4b8c06c2799bf57a91 INFO: 172.20.0.1:59620 - "POST /node/69cb1f4b8c06c2799bf57a91 HTTP/1.0" 202 Accepted 2026-03-31 01:19:32,168 - lava_callback - INFO - Starting async job submit for node 69cb1f4b8c06c2799bf57a91 2026-03-31 01:19:32,201 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:19:32,215 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1f4b8c06c2799bf57a91 HTTP/1.1" 200 None 2026-03-31 01:19:32,216 - lava_callback - DEBUG - Processing node 69cb1f4b8c06c2799bf57a91: baseline-nfs-arm64 2026-03-31 01:19:32,918 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 01:19:33,393 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 01:19:33,395 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cb1f4b8c06c2799bf57a91/log.txt.gz 2026-03-31 01:19:33,783 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 01:19:34,028 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 01:19:34,030 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cb1f4b8c06c2799bf57a91/lava-logs.yaml 2026-03-31 01:19:34,045 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 01:19:34,195 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 01:19:34,197 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cb1f4b8c06c2799bf57a91/lava_callback.json.gz 2026-03-31 01:19:34,230 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:19:34,245 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1f4b8c06c2799bf57a91 HTTP/1.1" 200 None 2026-03-31 01:19:34,246 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:19:34,259 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12e08c06c2799bf5502b HTTP/1.1" 200 None 2026-03-31 01:19:34,261 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:19:34,285 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb1f4b8c06c2799bf57a91 HTTP/1.1" 200 None 2026-03-31 01:19:34,287 - lava_callback - INFO - Completed processing callback for node 69cb1f4b8c06c2799bf57a91 INFO: 172.20.0.1:59636 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37442 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37446 - "GET / HTTP/1.0" 200 OK 2026-03-31 01:19:47,791 - lava_callback - INFO - Received callback for node 69c87e22f41e4fb6a3785adf INFO: 172.20.0.1:37454 - "POST /node/69c87e22f41e4fb6a3785adf HTTP/1.0" 202 Accepted 2026-03-31 01:19:47,797 - lava_callback - INFO - Starting async job submit for node 69c87e22f41e4fb6a3785adf 2026-03-31 01:19:47,797 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:19:47,812 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c87e22f41e4fb6a3785adf HTTP/1.1" 200 None 2026-03-31 01:19:47,812 - lava_callback - DEBUG - Processing node 69c87e22f41e4fb6a3785adf: baseline-arm-clabbe 2026-03-31 01:19:47,813 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c87e22f41e4fb6a3785adf 2026-03-31 01:19:47,813 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:19:47,827 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c87e22f41e4fb6a3785adf HTTP/1.1" 200 None 2026-03-31 01:19:47,828 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:19:47,841 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086a5ce0c9bc43bccb62b HTTP/1.1" 200 None 2026-03-31 01:19:47,843 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:19:47,864 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c87e22f41e4fb6a3785adf HTTP/1.1" 200 None 2026-03-31 01:19:47,868 - lava_callback - ERROR - Error processing callback for node 69c87e22f41e4fb6a3785adf: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:48132 - "GET / HTTP/1.0" 200 OK 2026-03-31 01:20:00,986 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:20:01,003 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:54066 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40342 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40350 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53112 - "GET / HTTP/1.0" 200 OK 2026-03-31 01:20:27,843 - lava_callback - INFO - Received callback for node 69c87e47f41e4fb6a3785aec INFO: 172.20.0.1:53118 - "POST /node/69c87e47f41e4fb6a3785aec HTTP/1.0" 202 Accepted 2026-03-31 01:20:27,849 - lava_callback - INFO - Starting async job submit for node 69c87e47f41e4fb6a3785aec 2026-03-31 01:20:27,850 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:20:27,864 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c87e47f41e4fb6a3785aec HTTP/1.1" 200 None 2026-03-31 01:20:27,865 - lava_callback - DEBUG - Processing node 69c87e47f41e4fb6a3785aec: baseline-arm-clabbe 2026-03-31 01:20:27,865 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c87e47f41e4fb6a3785aec 2026-03-31 01:20:27,866 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:20:27,886 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c87e47f41e4fb6a3785aec HTTP/1.1" 200 None 2026-03-31 01:20:27,888 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:20:27,908 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086a5ce0c9bc43bccb62b HTTP/1.1" 200 None 2026-03-31 01:20:27,910 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:20:27,931 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c87e47f41e4fb6a3785aec HTTP/1.1" 200 None 2026-03-31 01:20:27,932 - lava_callback - ERROR - Error processing callback for node 69c87e47f41e4fb6a3785aec: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:48046 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49394 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49402 - "GET / HTTP/1.0" 200 OK 2026-03-31 01:20:47,843 - lava_callback - INFO - Received callback for node 69c87e6cf41e4fb6a3785af8 INFO: 172.20.0.1:49406 - "POST /node/69c87e6cf41e4fb6a3785af8 HTTP/1.0" 202 Accepted 2026-03-31 01:20:47,849 - lava_callback - INFO - Starting async job submit for node 69c87e6cf41e4fb6a3785af8 2026-03-31 01:20:47,850 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:20:47,864 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c87e6cf41e4fb6a3785af8 HTTP/1.1" 200 None 2026-03-31 01:20:47,865 - lava_callback - DEBUG - Processing node 69c87e6cf41e4fb6a3785af8: baseline-arm-clabbe 2026-03-31 01:20:47,865 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c87e6cf41e4fb6a3785af8 2026-03-31 01:20:47,866 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:20:47,881 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c87e6cf41e4fb6a3785af8 HTTP/1.1" 200 None 2026-03-31 01:20:47,883 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:20:47,898 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086a5ce0c9bc43bccb62b HTTP/1.1" 200 None 2026-03-31 01:20:47,900 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:20:47,922 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c87e6cf41e4fb6a3785af8 HTTP/1.1" 200 None 2026-03-31 01:20:47,923 - lava_callback - ERROR - Error processing callback for node 69c87e6cf41e4fb6a3785af8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:55090 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59350 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55412 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55414 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34950 - "GET / HTTP/1.0" 200 OK 2026-03-31 01:21:27,933 - lava_callback - INFO - Received callback for node 69c87e91f41e4fb6a3785b09 INFO: 172.20.0.1:34964 - "POST /node/69c87e91f41e4fb6a3785b09 HTTP/1.0" 202 Accepted 2026-03-31 01:21:27,941 - lava_callback - INFO - Starting async job submit for node 69c87e91f41e4fb6a3785b09 2026-03-31 01:21:27,942 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:21:27,957 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c87e91f41e4fb6a3785b09 HTTP/1.1" 200 None 2026-03-31 01:21:27,958 - lava_callback - DEBUG - Processing node 69c87e91f41e4fb6a3785b09: baseline-arm-clabbe 2026-03-31 01:21:27,959 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c87e91f41e4fb6a3785b09 2026-03-31 01:21:27,959 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:21:27,980 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c87e91f41e4fb6a3785b09 HTTP/1.1" 200 None 2026-03-31 01:21:27,981 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:21:27,999 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086a5ce0c9bc43bccb62b HTTP/1.1" 200 None 2026-03-31 01:21:28,001 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:21:28,032 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c87e91f41e4fb6a3785b09 HTTP/1.1" 200 None 2026-03-31 01:21:28,033 - lava_callback - ERROR - Error processing callback for node 69c87e91f41e4fb6a3785b09: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:36690 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47896 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47904 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48248 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38030 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58486 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58496 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41354 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34888 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42336 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42340 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49396 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53398 - "GET / HTTP/1.0" 200 OK 2026-03-31 01:23:13,187 - lava_callback - INFO - Received callback for node 69cb21f78c06c2799bf57cb1 INFO: 172.20.0.1:48874 - "POST /node/69cb21f78c06c2799bf57cb1 HTTP/1.0" 202 Accepted 2026-03-31 01:23:13,193 - lava_callback - INFO - Starting async job submit for node 69cb21f78c06c2799bf57cb1 2026-03-31 01:23:13,195 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:23:13,213 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb21f78c06c2799bf57cb1 HTTP/1.1" 200 None 2026-03-31 01:23:13,214 - lava_callback - DEBUG - Processing node 69cb21f78c06c2799bf57cb1: baseline-nfs-arm64 2026-03-31 01:23:13,216 - lava_callback - WARNING - Failed to upload log 2026-03-31 01:23:13,218 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 01:23:13,386 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 01:23:13,389 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cb21f78c06c2799bf57cb1/lava-logs.yaml 2026-03-31 01:23:13,390 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 01:23:13,584 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 01:23:13,586 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cb21f78c06c2799bf57cb1/lava_callback.json.gz 2026-03-31 01:23:13,588 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:23:13,603 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb21f78c06c2799bf57cb1 HTTP/1.1" 200 None 2026-03-31 01:23:13,605 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:23:13,619 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12e08c06c2799bf5502b HTTP/1.1" 200 None 2026-03-31 01:23:13,621 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:23:13,643 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb21f78c06c2799bf57cb1 HTTP/1.1" 200 None 2026-03-31 01:23:13,647 - lava_callback - INFO - Completed processing callback for node 69cb21f78c06c2799bf57cb1 INFO: 172.20.0.1:48880 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48894 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54208 - "GET / HTTP/1.0" 200 OK 2026-03-31 01:23:31,006 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:23:31,025 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:58120 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60622 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60630 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41864 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46014 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52660 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52670 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60440 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57080 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57188 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57204 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46024 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58540 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55322 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55330 - "GET / HTTP/1.0" 200 OK 2026-03-31 01:25:17,057 - lava_callback - INFO - Received callback for node 69cb223f8c06c2799bf57cbe INFO: 172.20.0.1:55334 - "POST /node/69cb223f8c06c2799bf57cbe HTTP/1.0" 202 Accepted 2026-03-31 01:25:17,066 - lava_callback - INFO - Starting async job submit for node 69cb223f8c06c2799bf57cbe 2026-03-31 01:25:17,103 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:25:17,123 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb223f8c06c2799bf57cbe HTTP/1.1" 200 None 2026-03-31 01:25:17,124 - lava_callback - DEBUG - Processing node 69cb223f8c06c2799bf57cbe: baseline-arm64 2026-03-31 01:25:17,276 - lava_callback - WARNING - Failed to upload log 2026-03-31 01:25:17,394 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 01:25:17,600 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 01:25:17,602 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb223f8c06c2799bf57cbe/lava-logs.yaml 2026-03-31 01:25:17,606 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 01:25:17,734 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 01:25:17,736 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb223f8c06c2799bf57cbe/lava_callback.json.gz 2026-03-31 01:25:17,769 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:25:17,784 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb223f8c06c2799bf57cbe HTTP/1.1" 200 None 2026-03-31 01:25:17,786 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:25:17,799 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12e08c06c2799bf5502b HTTP/1.1" 200 None 2026-03-31 01:25:17,801 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:25:17,822 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb223f8c06c2799bf57cbe HTTP/1.1" 200 None 2026-03-31 01:25:17,885 - lava_callback - INFO - Completed processing callback for node 69cb223f8c06c2799bf57cbe INFO: 172.20.0.1:37930 - "GET / HTTP/1.0" 200 OK 2026-03-31 01:25:31,027 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:25:31,044 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:45362 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56752 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56754 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35428 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55220 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55074 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55090 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44446 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34184 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36072 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36074 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41052 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50958 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39788 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39792 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50742 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55554 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59744 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59754 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55480 - "GET / HTTP/1.0" 200 OK 2026-03-31 01:27:57,469 - lava_callback - INFO - Received callback for node 69cb22e48c06c2799bf57cdc INFO: 172.20.0.1:55482 - "POST /node/69cb22e48c06c2799bf57cdc HTTP/1.0" 202 Accepted 2026-03-31 01:27:57,480 - lava_callback - INFO - Starting async job submit for node 69cb22e48c06c2799bf57cdc 2026-03-31 01:27:57,540 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:27:57,570 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb22e48c06c2799bf57cdc HTTP/1.1" 200 None 2026-03-31 01:27:57,571 - lava_callback - DEBUG - Processing node 69cb22e48c06c2799bf57cdc: baseline-arm64 2026-03-31 01:27:57,798 - lava_callback - WARNING - Failed to upload log 2026-03-31 01:27:57,894 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 01:27:58,170 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 01:27:58,173 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb22e48c06c2799bf57cdc/lava-logs.yaml 2026-03-31 01:27:58,177 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 01:27:58,583 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 01:27:58,585 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb22e48c06c2799bf57cdc/lava_callback.json.gz 2026-03-31 01:27:58,634 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:27:58,651 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb22e48c06c2799bf57cdc HTTP/1.1" 200 None 2026-03-31 01:27:58,653 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:27:58,671 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12e08c06c2799bf5502b HTTP/1.1" 200 None 2026-03-31 01:27:58,673 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:27:58,711 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb22e48c06c2799bf57cdc HTTP/1.1" 200 None 2026-03-31 01:27:58,818 - lava_callback - INFO - Completed processing callback for node 69cb22e48c06c2799bf57cdc 2026-03-31 01:28:01,046 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:28:01,062 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:37516 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53488 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53500 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38968 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35804 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47866 - "GET / HTTP/1.0" 200 OK 2026-03-31 01:28:43,287 - lava_callback - INFO - Received callback for node 69c88035f41e4fb6a3785b1d INFO: 172.20.0.1:47878 - "POST /node/69c88035f41e4fb6a3785b1d HTTP/1.0" 202 Accepted 2026-03-31 01:28:43,293 - lava_callback - INFO - Starting async job submit for node 69c88035f41e4fb6a3785b1d 2026-03-31 01:28:43,294 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:28:43,309 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c88035f41e4fb6a3785b1d HTTP/1.1" 200 None 2026-03-31 01:28:43,310 - lava_callback - DEBUG - Processing node 69c88035f41e4fb6a3785b1d: baseline-arm64-baylibre 2026-03-31 01:28:43,310 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c88035f41e4fb6a3785b1d 2026-03-31 01:28:43,311 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:28:43,326 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c88035f41e4fb6a3785b1d HTTP/1.1" 200 None 2026-03-31 01:28:43,328 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:28:43,342 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3295318a3717aa65d01c3 HTTP/1.1" 200 None 2026-03-31 01:28:43,344 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:28:43,372 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c88035f41e4fb6a3785b1d HTTP/1.1" 200 None 2026-03-31 01:28:43,373 - lava_callback - ERROR - Error processing callback for node 69c88035f41e4fb6a3785b1d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:47888 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51184 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57072 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45590 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45592 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56234 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39996 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41016 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41026 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41922 - "GET / HTTP/1.0" 200 OK 2026-03-31 01:30:02,810 - lava_callback - INFO - Received callback for node 69c88085f41e4fb6a3785b2a INFO: 172.20.0.1:59266 - "POST /node/69c88085f41e4fb6a3785b2a HTTP/1.0" 202 Accepted 2026-03-31 01:30:02,816 - lava_callback - INFO - Starting async job submit for node 69c88085f41e4fb6a3785b2a 2026-03-31 01:30:02,817 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:30:02,831 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c88085f41e4fb6a3785b2a HTTP/1.1" 200 None 2026-03-31 01:30:02,832 - lava_callback - DEBUG - Processing node 69c88085f41e4fb6a3785b2a: baseline-arm64-baylibre 2026-03-31 01:30:02,832 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c88085f41e4fb6a3785b2a 2026-03-31 01:30:02,833 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:30:02,846 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c88085f41e4fb6a3785b2a HTTP/1.1" 200 None 2026-03-31 01:30:02,847 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:30:02,861 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3295318a3717aa65d01c3 HTTP/1.1" 200 None 2026-03-31 01:30:02,863 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:30:02,884 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c88085f41e4fb6a3785b2a HTTP/1.1" 200 None 2026-03-31 01:30:02,890 - lava_callback - ERROR - Error processing callback for node 69c88085f41e4fb6a3785b2a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:59276 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54436 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54444 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59764 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58740 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38056 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38064 - "GET / HTTP/1.0" 200 OK 2026-03-31 01:30:47,052 - lava_callback - INFO - Received callback for node 69cb238b8c06c2799bf57cf8 INFO: 172.20.0.1:38068 - "POST /node/69cb238b8c06c2799bf57cf8 HTTP/1.0" 202 Accepted 2026-03-31 01:30:47,058 - lava_callback - INFO - Starting async job submit for node 69cb238b8c06c2799bf57cf8 2026-03-31 01:30:47,079 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:30:47,095 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb238b8c06c2799bf57cf8 HTTP/1.1" 200 None 2026-03-31 01:30:47,095 - lava_callback - DEBUG - Processing node 69cb238b8c06c2799bf57cf8: baseline-arm64 2026-03-31 01:30:47,541 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 01:30:47,726 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 01:30:47,728 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb238b8c06c2799bf57cf8/log.txt.gz 2026-03-31 01:30:47,987 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 01:30:48,379 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 01:30:48,382 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb238b8c06c2799bf57cf8/lava-logs.yaml 2026-03-31 01:30:48,390 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 01:30:48,595 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 01:30:48,597 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb238b8c06c2799bf57cf8/lava_callback.json.gz 2026-03-31 01:30:48,619 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:30:48,633 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb238b8c06c2799bf57cf8 HTTP/1.1" 200 None 2026-03-31 01:30:48,635 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:30:48,648 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12e08c06c2799bf5502b HTTP/1.1" 200 None 2026-03-31 01:30:48,650 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:30:48,674 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb238b8c06c2799bf57cf8 HTTP/1.1" 200 None 2026-03-31 01:30:48,675 - lava_callback - INFO - Completed processing callback for node 69cb238b8c06c2799bf57cf8 INFO: 172.20.0.1:41574 - "GET / HTTP/1.0" 200 OK 2026-03-31 01:31:01,065 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:31:01,081 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:37440 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58892 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58908 - "GET / HTTP/1.0" 200 OK 2026-03-31 01:31:23,038 - lava_callback - INFO - Received callback for node 69c880d5f41e4fb6a3785b3b INFO: 172.20.0.1:43858 - "POST /node/69c880d5f41e4fb6a3785b3b HTTP/1.0" 202 Accepted 2026-03-31 01:31:23,047 - lava_callback - INFO - Starting async job submit for node 69c880d5f41e4fb6a3785b3b 2026-03-31 01:31:23,048 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:31:23,073 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c880d5f41e4fb6a3785b3b HTTP/1.1" 200 None 2026-03-31 01:31:23,074 - lava_callback - DEBUG - Processing node 69c880d5f41e4fb6a3785b3b: baseline-arm64-baylibre 2026-03-31 01:31:23,075 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c880d5f41e4fb6a3785b3b 2026-03-31 01:31:23,075 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:31:23,098 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c880d5f41e4fb6a3785b3b HTTP/1.1" 200 None 2026-03-31 01:31:23,100 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:31:23,121 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3295318a3717aa65d01c3 HTTP/1.1" 200 None 2026-03-31 01:31:23,123 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:31:23,157 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c880d5f41e4fb6a3785b3b HTTP/1.1" 200 None 2026-03-31 01:31:23,158 - lava_callback - ERROR - Error processing callback for node 69c880d5f41e4fb6a3785b3b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:43870 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34972 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35248 - "GET / HTTP/1.0" 200 OK 2026-03-31 01:31:43,417 - lava_callback - INFO - Received callback for node 69cb1f418c06c2799bf57a76 INFO: 172.20.0.1:35264 - "POST /node/69cb1f418c06c2799bf57a76 HTTP/1.0" 202 Accepted 2026-03-31 01:31:43,437 - lava_callback - INFO - Starting async job submit for node 69cb1f418c06c2799bf57a76 2026-03-31 01:31:43,523 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:31:43,547 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1f418c06c2799bf57a76 HTTP/1.1" 200 None 2026-03-31 01:31:43,549 - lava_callback - DEBUG - Processing node 69cb1f418c06c2799bf57a76: baseline-nfs-arm64 2026-03-31 01:31:45,270 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 01:31:45,737 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 01:31:45,739 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cb1f418c06c2799bf57a76/log.txt.gz INFO: 172.20.0.1:35270 - "GET / HTTP/1.0" 200 OK 2026-03-31 01:31:46,645 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 01:31:47,107 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 01:31:47,110 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cb1f418c06c2799bf57a76/lava-logs.yaml 2026-03-31 01:31:47,131 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 01:31:47,564 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 01:31:47,566 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cb1f418c06c2799bf57a76/lava_callback.json.gz DEBUG: Setup failed for 69cb1f418c06c2799bf57a76. Transitting job node result: incomplete -> fail 2026-03-31 01:31:47,713 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:31:47,727 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1f418c06c2799bf57a76 HTTP/1.1" 200 None 2026-03-31 01:31:47,729 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:31:47,742 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12e08c06c2799bf5502b HTTP/1.1" 200 None 2026-03-31 01:31:47,744 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:31:47,767 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb1f418c06c2799bf57a76 HTTP/1.1" 200 None 2026-03-31 01:31:47,862 - lava_callback - INFO - Completed processing callback for node 69cb1f418c06c2799bf57a76 INFO: 172.20.0.1:59372 - "GET / HTTP/1.0" 200 OK 2026-03-31 01:32:01,084 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:32:01,115 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:37744 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42090 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42094 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59924 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58278 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40714 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40720 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42154 - "GET / HTTP/1.0" 200 OK 2026-03-31 01:33:03,010 - lava_callback - INFO - Received callback for node 69c8813af41e4fb6a3785b49 INFO: 172.20.0.1:35140 - "POST /node/69c8813af41e4fb6a3785b49 HTTP/1.0" 202 Accepted 2026-03-31 01:33:03,016 - lava_callback - INFO - Starting async job submit for node 69c8813af41e4fb6a3785b49 2026-03-31 01:33:03,017 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:33:03,032 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8813af41e4fb6a3785b49 HTTP/1.1" 200 None 2026-03-31 01:33:03,033 - lava_callback - DEBUG - Processing node 69c8813af41e4fb6a3785b49: baseline-arm64-baylibre 2026-03-31 01:33:03,033 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8813af41e4fb6a3785b49 2026-03-31 01:33:03,033 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:33:03,047 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8813af41e4fb6a3785b49 HTTP/1.1" 200 None 2026-03-31 01:33:03,048 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:33:03,061 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3295318a3717aa65d01c3 HTTP/1.1" 200 None 2026-03-31 01:33:03,062 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:33:03,090 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8813af41e4fb6a3785b49 HTTP/1.1" 200 None 2026-03-31 01:33:03,092 - lava_callback - ERROR - Error processing callback for node 69c8813af41e4fb6a3785b49: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:35156 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57330 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57336 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57390 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39730 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54264 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54278 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43530 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51492 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46306 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46322 - "GET / HTTP/1.0" 200 OK 2026-03-31 01:34:23,113 - lava_callback - INFO - Received callback for node 69c8818ff41e4fb6a3785b5e INFO: 172.20.0.1:59986 - "POST /node/69c8818ff41e4fb6a3785b5e HTTP/1.0" 202 Accepted 2026-03-31 01:34:23,119 - lava_callback - INFO - Starting async job submit for node 69c8818ff41e4fb6a3785b5e 2026-03-31 01:34:23,120 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:34:23,137 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8818ff41e4fb6a3785b5e HTTP/1.1" 200 None 2026-03-31 01:34:23,138 - lava_callback - DEBUG - Processing node 69c8818ff41e4fb6a3785b5e: baseline-arm64-baylibre 2026-03-31 01:34:23,139 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8818ff41e4fb6a3785b5e 2026-03-31 01:34:23,140 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:34:23,155 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8818ff41e4fb6a3785b5e HTTP/1.1" 200 None 2026-03-31 01:34:23,156 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:34:23,169 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3295318a3717aa65d01c3 HTTP/1.1" 200 None 2026-03-31 01:34:23,171 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:34:23,205 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8818ff41e4fb6a3785b5e HTTP/1.1" 200 None 2026-03-31 01:34:23,206 - lava_callback - ERROR - Error processing callback for node 69c8818ff41e4fb6a3785b5e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:59988 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45604 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37554 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37556 - "GET / HTTP/1.0" 200 OK 2026-03-31 01:34:49,972 - lava_callback - INFO - Received callback for node 69cb246c8c06c2799bf57d31 INFO: 172.20.0.1:51602 - "POST /node/69cb246c8c06c2799bf57d31 HTTP/1.0" 202 Accepted 2026-03-31 01:34:49,980 - lava_callback - INFO - Starting async job submit for node 69cb246c8c06c2799bf57d31 2026-03-31 01:34:50,012 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:34:50,026 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb246c8c06c2799bf57d31 HTTP/1.1" 200 None 2026-03-31 01:34:50,027 - lava_callback - DEBUG - Processing node 69cb246c8c06c2799bf57d31: baseline-arm64-foundriesio 2026-03-31 01:34:50,440 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 01:34:50,608 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 01:34:50,610 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69cb246c8c06c2799bf57d31/log.txt.gz 2026-03-31 01:34:50,862 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 01:34:51,108 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 01:34:51,110 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69cb246c8c06c2799bf57d31/lava-logs.yaml 2026-03-31 01:34:51,121 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 01:34:51,286 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 01:34:51,288 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69cb246c8c06c2799bf57d31/lava_callback.json.gz 2026-03-31 01:34:51,319 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:34:51,333 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb246c8c06c2799bf57d31 HTTP/1.1" 200 None 2026-03-31 01:34:51,335 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:34:51,348 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12e08c06c2799bf5502b HTTP/1.1" 200 None 2026-03-31 01:34:51,350 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:34:51,373 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb246c8c06c2799bf57d31 HTTP/1.1" 200 None 2026-03-31 01:34:51,385 - lava_callback - INFO - Completed processing callback for node 69cb246c8c06c2799bf57d31 INFO: 172.20.0.1:51604 - "GET / HTTP/1.0" 200 OK 2026-03-31 01:35:01,118 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:35:01,136 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:41776 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50832 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50844 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39870 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46130 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53038 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53052 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57332 - "GET / HTTP/1.0" 200 OK 2026-03-31 01:36:03,182 - lava_callback - INFO - Received callback for node 69c88201f41e4fb6a3785b81 INFO: 172.20.0.1:41610 - "POST /node/69c88201f41e4fb6a3785b81 HTTP/1.0" 202 Accepted 2026-03-31 01:36:03,188 - lava_callback - INFO - Starting async job submit for node 69c88201f41e4fb6a3785b81 2026-03-31 01:36:03,189 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:36:03,203 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c88201f41e4fb6a3785b81 HTTP/1.1" 200 None 2026-03-31 01:36:03,204 - lava_callback - DEBUG - Processing node 69c88201f41e4fb6a3785b81: baseline-arm64-baylibre 2026-03-31 01:36:03,204 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c88201f41e4fb6a3785b81 2026-03-31 01:36:03,205 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:36:03,218 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c88201f41e4fb6a3785b81 HTTP/1.1" 200 None 2026-03-31 01:36:03,219 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:36:03,233 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3295318a3717aa65d01c3 HTTP/1.1" 200 None 2026-03-31 01:36:03,234 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:36:03,260 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c88201f41e4fb6a3785b81 HTTP/1.1" 200 None 2026-03-31 01:36:03,265 - lava_callback - ERROR - Error processing callback for node 69c88201f41e4fb6a3785b81: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:41616 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44736 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44748 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43238 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48952 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35394 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35398 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55772 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49946 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38542 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38554 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50806 - "GET / HTTP/1.0" 200 OK 2026-03-31 01:37:33,578 - lava_callback - INFO - Received callback for node 69cb25508c06c2799bf57d57 INFO: 172.20.0.1:39058 - "POST /node/69cb25508c06c2799bf57d57 HTTP/1.0" 202 Accepted 2026-03-31 01:37:33,586 - lava_callback - INFO - Starting async job submit for node 69cb25508c06c2799bf57d57 2026-03-31 01:37:33,588 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:37:33,608 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb25508c06c2799bf57d57 HTTP/1.1" 200 None 2026-03-31 01:37:33,609 - lava_callback - DEBUG - Processing node 69cb25508c06c2799bf57d57: baseline-nfs-arm64 2026-03-31 01:37:33,611 - lava_callback - WARNING - Failed to upload log 2026-03-31 01:37:33,612 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 01:37:33,736 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 01:37:33,738 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cb25508c06c2799bf57d57/lava-logs.yaml 2026-03-31 01:37:33,739 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 01:37:33,856 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 01:37:33,858 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cb25508c06c2799bf57d57/lava_callback.json.gz 2026-03-31 01:37:33,860 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:37:33,875 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb25508c06c2799bf57d57 HTTP/1.1" 200 None 2026-03-31 01:37:33,876 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:37:33,890 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12e08c06c2799bf5502b HTTP/1.1" 200 None 2026-03-31 01:37:33,891 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:37:33,912 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb25508c06c2799bf57d57 HTTP/1.1" 200 None 2026-03-31 01:37:33,915 - lava_callback - INFO - Completed processing callback for node 69cb25508c06c2799bf57d57 INFO: 172.20.0.1:39060 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58238 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58244 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50590 - "GET / HTTP/1.0" 200 OK 2026-03-31 01:38:01,139 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:38:01,159 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:60740 - "GET / HTTP/1.0" 200 OK 2026-03-31 01:38:08,656 - lava_callback - INFO - Received callback for node 69c88277f41e4fb6a3785ba0 INFO: 172.20.0.1:60748 - "POST /node/69c88277f41e4fb6a3785ba0 HTTP/1.0" 202 Accepted 2026-03-31 01:38:08,662 - lava_callback - INFO - Starting async job submit for node 69c88277f41e4fb6a3785ba0 2026-03-31 01:38:08,663 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:38:08,684 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c88277f41e4fb6a3785ba0 HTTP/1.1" 200 None 2026-03-31 01:38:08,685 - lava_callback - DEBUG - Processing node 69c88277f41e4fb6a3785ba0: baseline-arm64-clabbe 2026-03-31 01:38:08,685 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c88277f41e4fb6a3785ba0 2026-03-31 01:38:08,686 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:38:08,699 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c88277f41e4fb6a3785ba0 HTTP/1.1" 200 None 2026-03-31 01:38:08,701 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:38:08,716 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3295318a3717aa65d01c3 HTTP/1.1" 200 None 2026-03-31 01:38:08,718 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:38:08,739 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c88277f41e4fb6a3785ba0 HTTP/1.1" 200 None 2026-03-31 01:38:08,741 - lava_callback - ERROR - Error processing callback for node 69c88277f41e4fb6a3785ba0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:58836 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58840 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44794 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59382 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45150 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45160 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51240 - "GET / HTTP/1.0" 200 OK 2026-03-31 01:39:03,397 - lava_callback - INFO - Received callback for node 69c882b0f41e4fb6a3785c3c INFO: 172.20.0.1:55760 - "POST /node/69c882b0f41e4fb6a3785c3c HTTP/1.0" 202 Accepted 2026-03-31 01:39:03,403 - lava_callback - INFO - Starting async job submit for node 69c882b0f41e4fb6a3785c3c 2026-03-31 01:39:03,403 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:39:03,421 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c882b0f41e4fb6a3785c3c HTTP/1.1" 200 None 2026-03-31 01:39:03,422 - lava_callback - DEBUG - Processing node 69c882b0f41e4fb6a3785c3c: baseline-arm64-baylibre 2026-03-31 01:39:03,422 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c882b0f41e4fb6a3785c3c 2026-03-31 01:39:03,423 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:39:03,437 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c882b0f41e4fb6a3785c3c HTTP/1.1" 200 None 2026-03-31 01:39:03,438 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:39:03,452 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f7ef41e4fb6a3782f4d HTTP/1.1" 200 None 2026-03-31 01:39:03,454 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:39:03,477 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c882b0f41e4fb6a3785c3c HTTP/1.1" 200 None 2026-03-31 01:39:03,479 - lava_callback - ERROR - Error processing callback for node 69c882b0f41e4fb6a3785c3c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:55772 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58884 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58892 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49428 - "GET / HTTP/1.0" 200 OK 2026-03-31 01:39:28,755 - lava_callback - INFO - Received callback for node 69c882bff41e4fb6a3785c60 INFO: 172.20.0.1:49440 - "POST /node/69c882bff41e4fb6a3785c60 HTTP/1.0" 202 Accepted 2026-03-31 01:39:28,761 - lava_callback - INFO - Starting async job submit for node 69c882bff41e4fb6a3785c60 2026-03-31 01:39:28,762 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:39:28,781 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c882bff41e4fb6a3785c60 HTTP/1.1" 200 None 2026-03-31 01:39:28,781 - lava_callback - DEBUG - Processing node 69c882bff41e4fb6a3785c60: baseline-arm64-clabbe 2026-03-31 01:39:28,782 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c882bff41e4fb6a3785c60 2026-03-31 01:39:28,782 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:39:28,796 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c882bff41e4fb6a3785c60 HTTP/1.1" 200 None 2026-03-31 01:39:28,797 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:39:28,810 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f7ef41e4fb6a3782f4d HTTP/1.1" 200 None 2026-03-31 01:39:28,812 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:39:28,848 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c882bff41e4fb6a3785c60 HTTP/1.1" 200 None 2026-03-31 01:39:28,850 - lava_callback - ERROR - Error processing callback for node 69c882bff41e4fb6a3785c60: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-31 01:39:29,446 - lava_callback - INFO - Received callback for node 69c882c1f41e4fb6a3785c63 INFO: 172.20.0.1:49446 - "POST /node/69c882c1f41e4fb6a3785c63 HTTP/1.0" 202 Accepted 2026-03-31 01:39:29,452 - lava_callback - INFO - Starting async job submit for node 69c882c1f41e4fb6a3785c63 2026-03-31 01:39:29,453 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:39:29,473 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c882c1f41e4fb6a3785c63 HTTP/1.1" 200 None 2026-03-31 01:39:29,474 - lava_callback - DEBUG - Processing node 69c882c1f41e4fb6a3785c63: baseline-arm64-clabbe 2026-03-31 01:39:29,475 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c882c1f41e4fb6a3785c63 2026-03-31 01:39:29,475 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:39:29,489 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c882c1f41e4fb6a3785c63 HTTP/1.1" 200 None 2026-03-31 01:39:29,491 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:39:29,504 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f7ef41e4fb6a3782f4d HTTP/1.1" 200 None 2026-03-31 01:39:29,506 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:39:29,527 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c882c1f41e4fb6a3785c63 HTTP/1.1" 200 None 2026-03-31 01:39:29,535 - lava_callback - ERROR - Error processing callback for node 69c882c1f41e4fb6a3785c63: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-31 01:39:30,135 - lava_callback - INFO - Received callback for node 69c882c2f41e4fb6a3785c66 INFO: 172.20.0.1:45676 - "POST /node/69c882c2f41e4fb6a3785c66 HTTP/1.0" 202 Accepted 2026-03-31 01:39:30,142 - lava_callback - INFO - Starting async job submit for node 69c882c2f41e4fb6a3785c66 2026-03-31 01:39:30,142 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:39:30,157 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c882c2f41e4fb6a3785c66 HTTP/1.1" 200 None 2026-03-31 01:39:30,159 - lava_callback - DEBUG - Processing node 69c882c2f41e4fb6a3785c66: baseline-arm64-clabbe 2026-03-31 01:39:30,159 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c882c2f41e4fb6a3785c66 2026-03-31 01:39:30,160 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:39:30,175 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c882c2f41e4fb6a3785c66 HTTP/1.1" 200 None 2026-03-31 01:39:30,177 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:39:30,193 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f7ef41e4fb6a3782f4d HTTP/1.1" 200 None 2026-03-31 01:39:30,195 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:39:30,221 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c882c2f41e4fb6a3785c66 HTTP/1.1" 200 None 2026-03-31 01:39:30,223 - lava_callback - ERROR - Error processing callback for node 69c882c2f41e4fb6a3785c66: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-31 01:39:30,835 - lava_callback - INFO - Received callback for node 69c882c3f41e4fb6a3785c69 INFO: 172.20.0.1:45690 - "POST /node/69c882c3f41e4fb6a3785c69 HTTP/1.0" 202 Accepted 2026-03-31 01:39:30,843 - lava_callback - INFO - Starting async job submit for node 69c882c3f41e4fb6a3785c69 2026-03-31 01:39:30,844 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:39:30,861 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c882c3f41e4fb6a3785c69 HTTP/1.1" 200 None 2026-03-31 01:39:30,862 - lava_callback - DEBUG - Processing node 69c882c3f41e4fb6a3785c69: baseline-arm64-clabbe 2026-03-31 01:39:30,862 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c882c3f41e4fb6a3785c69 2026-03-31 01:39:30,863 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:39:30,886 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c882c3f41e4fb6a3785c69 HTTP/1.1" 200 None 2026-03-31 01:39:30,887 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:39:30,902 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f7ef41e4fb6a3782f4d HTTP/1.1" 200 None 2026-03-31 01:39:30,903 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:39:30,926 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c882c3f41e4fb6a3785c69 HTTP/1.1" 200 None 2026-03-31 01:39:30,927 - lava_callback - ERROR - Error processing callback for node 69c882c3f41e4fb6a3785c69: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-31 01:39:31,530 - lava_callback - INFO - Received callback for node 69c882c4f41e4fb6a3785c6e INFO: 172.20.0.1:45706 - "POST /node/69c882c4f41e4fb6a3785c6e HTTP/1.0" 202 Accepted 2026-03-31 01:39:31,536 - lava_callback - INFO - Starting async job submit for node 69c882c4f41e4fb6a3785c6e 2026-03-31 01:39:31,537 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:39:31,553 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c882c4f41e4fb6a3785c6e HTTP/1.1" 200 None 2026-03-31 01:39:31,554 - lava_callback - DEBUG - Processing node 69c882c4f41e4fb6a3785c6e: baseline-arm64-clabbe 2026-03-31 01:39:31,555 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c882c4f41e4fb6a3785c6e 2026-03-31 01:39:31,555 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:39:31,569 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c882c4f41e4fb6a3785c6e HTTP/1.1" 200 None 2026-03-31 01:39:31,570 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:39:31,587 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3295318a3717aa65d01c3 HTTP/1.1" 200 None 2026-03-31 01:39:31,589 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:39:31,616 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c882c4f41e4fb6a3785c6e HTTP/1.1" 200 None 2026-03-31 01:39:31,617 - lava_callback - ERROR - Error processing callback for node 69c882c4f41e4fb6a3785c6e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-31 01:39:32,219 - lava_callback - INFO - Received callback for node 69c882c5f41e4fb6a3785c71 INFO: 172.20.0.1:45720 - "POST /node/69c882c5f41e4fb6a3785c71 HTTP/1.0" 202 Accepted 2026-03-31 01:39:32,227 - lava_callback - INFO - Starting async job submit for node 69c882c5f41e4fb6a3785c71 2026-03-31 01:39:32,228 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:39:32,245 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c882c5f41e4fb6a3785c71 HTTP/1.1" 200 None 2026-03-31 01:39:32,246 - lava_callback - DEBUG - Processing node 69c882c5f41e4fb6a3785c71: baseline-arm64-clabbe 2026-03-31 01:39:32,246 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c882c5f41e4fb6a3785c71 2026-03-31 01:39:32,247 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:39:32,263 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c882c5f41e4fb6a3785c71 HTTP/1.1" 200 None 2026-03-31 01:39:32,265 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:39:32,282 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f7ef41e4fb6a3782f4d HTTP/1.1" 200 None 2026-03-31 01:39:32,284 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:39:32,304 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c882c5f41e4fb6a3785c71 HTTP/1.1" 200 None 2026-03-31 01:39:32,310 - lava_callback - ERROR - Error processing callback for node 69c882c5f41e4fb6a3785c71: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-31 01:39:32,920 - lava_callback - INFO - Received callback for node 69c882c6f41e4fb6a3785c96 INFO: 172.20.0.1:45736 - "POST /node/69c882c6f41e4fb6a3785c96 HTTP/1.0" 202 Accepted 2026-03-31 01:39:32,928 - lava_callback - INFO - Starting async job submit for node 69c882c6f41e4fb6a3785c96 2026-03-31 01:39:32,929 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:39:32,946 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c882c6f41e4fb6a3785c96 HTTP/1.1" 200 None 2026-03-31 01:39:32,947 - lava_callback - DEBUG - Processing node 69c882c6f41e4fb6a3785c96: baseline-arm64-clabbe 2026-03-31 01:39:32,948 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c882c6f41e4fb6a3785c96 2026-03-31 01:39:32,948 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:39:32,962 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c882c6f41e4fb6a3785c96 HTTP/1.1" 200 None 2026-03-31 01:39:32,964 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:39:32,979 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f7ef41e4fb6a3782f4d HTTP/1.1" 200 None 2026-03-31 01:39:32,980 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:39:33,008 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c882c6f41e4fb6a3785c96 HTTP/1.1" 200 None 2026-03-31 01:39:33,013 - lava_callback - ERROR - Error processing callback for node 69c882c6f41e4fb6a3785c96: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-31 01:39:34,944 - lava_callback - INFO - Received callback for node 69cb25998c06c2799bf57d67 INFO: 172.20.0.1:45752 - "POST /node/69cb25998c06c2799bf57d67 HTTP/1.0" 202 Accepted 2026-03-31 01:39:34,952 - lava_callback - INFO - Starting async job submit for node 69cb25998c06c2799bf57d67 2026-03-31 01:39:34,985 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:39:35,002 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb25998c06c2799bf57d67 HTTP/1.1" 200 None 2026-03-31 01:39:35,003 - lava_callback - DEBUG - Processing node 69cb25998c06c2799bf57d67: baseline-arm64 2026-03-31 01:39:35,161 - lava_callback - WARNING - Failed to upload log 2026-03-31 01:39:35,234 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 01:39:35,422 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 01:39:35,424 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb25998c06c2799bf57d67/lava-logs.yaml 2026-03-31 01:39:35,427 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 01:39:35,628 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 01:39:35,630 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb25998c06c2799bf57d67/lava_callback.json.gz 2026-03-31 01:39:35,671 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:39:35,686 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb25998c06c2799bf57d67 HTTP/1.1" 200 None 2026-03-31 01:39:35,689 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:39:35,717 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12e08c06c2799bf5502b HTTP/1.1" 200 None 2026-03-31 01:39:35,720 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:39:35,756 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb25998c06c2799bf57d67 HTTP/1.1" 200 None 2026-03-31 01:39:35,845 - lava_callback - INFO - Completed processing callback for node 69cb25998c06c2799bf57d67 INFO: 172.20.0.1:45766 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56854 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56858 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43832 - "GET / HTTP/1.0" 200 OK 2026-03-31 01:40:01,162 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:40:01,193 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:47388 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47988 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48002 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44894 - "GET / HTTP/1.0" 200 OK 2026-03-31 01:40:27,935 - lava_callback - INFO - Received callback for node 69cb24aa8c06c2799bf57d39 INFO: 172.20.0.1:44896 - "POST /node/69cb24aa8c06c2799bf57d39 HTTP/1.0" 202 Accepted 2026-03-31 01:40:27,943 - lava_callback - INFO - Starting async job submit for node 69cb24aa8c06c2799bf57d39 2026-03-31 01:40:27,986 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:40:28,004 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb24aa8c06c2799bf57d39 HTTP/1.1" 200 None 2026-03-31 01:40:28,004 - lava_callback - DEBUG - Processing node 69cb24aa8c06c2799bf57d39: baseline-arm64 2026-03-31 01:40:28,437 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 01:40:28,642 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 01:40:28,644 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb24aa8c06c2799bf57d39/log.txt.gz 2026-03-31 01:40:28,880 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 01:40:29,153 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 01:40:29,155 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb24aa8c06c2799bf57d39/lava-logs.yaml 2026-03-31 01:40:29,164 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 01:40:29,371 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 01:40:29,373 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb24aa8c06c2799bf57d39/lava_callback.json.gz DEBUG: Setup failed for 69cb24aa8c06c2799bf57d39. Transitting job node result: incomplete -> fail 2026-03-31 01:40:29,451 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:40:29,466 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb24aa8c06c2799bf57d39 HTTP/1.1" 200 None 2026-03-31 01:40:29,468 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:40:29,482 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12e08c06c2799bf5502b HTTP/1.1" 200 None 2026-03-31 01:40:29,484 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:40:29,509 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb24aa8c06c2799bf57d39 HTTP/1.1" 200 None 2026-03-31 01:40:29,604 - lava_callback - INFO - Completed processing callback for node 69cb24aa8c06c2799bf57d39 2026-03-31 01:40:31,195 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:40:31,210 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:58686 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55628 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55634 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40168 - "GET / HTTP/1.0" 200 OK 2026-03-31 01:41:03,057 - lava_callback - INFO - Received callback for node 69cb25ea8c06c2799bf57d82 INFO: 172.20.0.1:33722 - "POST /node/69cb25ea8c06c2799bf57d82 HTTP/1.0" 202 Accepted 2026-03-31 01:41:03,064 - lava_callback - INFO - Starting async job submit for node 69cb25ea8c06c2799bf57d82 2026-03-31 01:41:03,095 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:41:03,123 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb25ea8c06c2799bf57d82 HTTP/1.1" 200 None 2026-03-31 01:41:03,124 - lava_callback - DEBUG - Processing node 69cb25ea8c06c2799bf57d82: baseline-arm64 2026-03-31 01:41:03,336 - lava_callback - WARNING - Failed to upload log 2026-03-31 01:41:03,437 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 01:41:03,631 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 01:41:03,633 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb25ea8c06c2799bf57d82/lava-logs.yaml 2026-03-31 01:41:03,637 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 01:41:03,782 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 01:41:03,784 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb25ea8c06c2799bf57d82/lava_callback.json.gz 2026-03-31 01:41:03,816 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:41:03,830 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb25ea8c06c2799bf57d82 HTTP/1.1" 200 None 2026-03-31 01:41:03,831 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:41:03,849 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12e08c06c2799bf5502b HTTP/1.1" 200 None 2026-03-31 01:41:03,850 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:41:03,870 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb25ea8c06c2799bf57d82 HTTP/1.1" 200 None 2026-03-31 01:41:03,933 - lava_callback - INFO - Completed processing callback for node 69cb25ea8c06c2799bf57d82 INFO: 172.20.0.1:33732 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59290 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59292 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49504 - "GET / HTTP/1.0" 200 OK 2026-03-31 01:41:31,212 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:41:31,233 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:52602 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47000 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47002 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33182 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53782 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33758 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33760 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55240 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44836 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49622 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49632 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45496 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52750 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45360 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45366 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46232 - "GET / HTTP/1.0" 200 OK 2026-03-31 01:43:29,126 - lava_callback - INFO - Received callback for node 69c883abf41e4fb6a3785dc9 INFO: 172.20.0.1:46246 - "POST /node/69c883abf41e4fb6a3785dc9 HTTP/1.0" 202 Accepted 2026-03-31 01:43:29,132 - lava_callback - INFO - Starting async job submit for node 69c883abf41e4fb6a3785dc9 2026-03-31 01:43:29,133 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:43:29,151 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c883abf41e4fb6a3785dc9 HTTP/1.1" 200 None 2026-03-31 01:43:29,152 - lava_callback - DEBUG - Processing node 69c883abf41e4fb6a3785dc9: baseline-arm64-clabbe 2026-03-31 01:43:29,153 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c883abf41e4fb6a3785dc9 2026-03-31 01:43:29,153 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:43:29,168 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c883abf41e4fb6a3785dc9 HTTP/1.1" 200 None 2026-03-31 01:43:29,170 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:43:29,184 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3295318a3717aa65d01c3 HTTP/1.1" 200 None 2026-03-31 01:43:29,186 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:43:29,227 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c883abf41e4fb6a3785dc9 HTTP/1.1" 200 None 2026-03-31 01:43:29,229 - lava_callback - ERROR - Error processing callback for node 69c883abf41e4fb6a3785dc9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:57698 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47120 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47136 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38194 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35854 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35314 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35330 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35342 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38320 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48166 - "GET /favicon.ico HTTP/1.0" 404 Not Found INFO: 172.20.0.1:48172 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43496 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43510 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36646 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44676 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51792 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51804 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49726 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45262 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43920 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43932 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45788 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58936 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51420 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51432 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48232 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55232 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54676 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54684 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53542 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46158 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41958 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41974 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52692 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41652 - "GET / HTTP/1.0" 200 OK 2026-03-31 01:47:36,802 - lava_callback - INFO - Received callback for node 69cb1bde8c06c2799bf552b9 INFO: 172.20.0.1:41660 - "POST /node/69cb1bde8c06c2799bf552b9 HTTP/1.0" 202 Accepted 2026-03-31 01:47:36,851 - lava_callback - INFO - Starting async job submit for node 69cb1bde8c06c2799bf552b9 2026-03-31 01:47:36,878 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:47:36,896 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1bde8c06c2799bf552b9 HTTP/1.1" 200 None 2026-03-31 01:47:36,897 - lava_callback - DEBUG - Processing node 69cb1bde8c06c2799bf552b9: baseline-arm-pengutronix 2026-03-31 01:47:37,033 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 01:47:37,190 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 01:47:37,192 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cb1bde8c06c2799bf552b9/log.txt.gz 2026-03-31 01:47:37,269 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 01:47:37,450 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 01:47:37,452 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cb1bde8c06c2799bf552b9/lava-logs.yaml 2026-03-31 01:47:37,455 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 01:47:37,580 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 01:47:37,582 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cb1bde8c06c2799bf552b9/lava_callback.json.gz DEBUG: Setup failed for 69cb1bde8c06c2799bf552b9. Transitting job node result: incomplete -> fail 2026-03-31 01:47:37,630 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:47:37,651 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1bde8c06c2799bf552b9 HTTP/1.1" 200 None 2026-03-31 01:47:37,652 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:47:37,665 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12cb8c06c2799bf55018 HTTP/1.1" 200 None 2026-03-31 01:47:37,667 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:47:37,690 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb1bde8c06c2799bf552b9 HTTP/1.1" 200 None 2026-03-31 01:47:37,743 - lava_callback - INFO - Completed processing callback for node 69cb1bde8c06c2799bf552b9 INFO: 172.20.0.1:39506 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39522 - "GET / HTTP/1.0" 200 OK 2026-03-31 01:47:50,609 - lava_callback - INFO - Received callback for node 69cb23d88c06c2799bf57d0e INFO: 172.20.0.1:58666 - "POST /node/69cb23d88c06c2799bf57d0e HTTP/1.0" 202 Accepted 2026-03-31 01:47:50,616 - lava_callback - INFO - Starting async job submit for node 69cb23d88c06c2799bf57d0e 2026-03-31 01:47:50,668 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:47:50,683 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb23d88c06c2799bf57d0e HTTP/1.1" 200 None 2026-03-31 01:47:50,684 - lava_callback - DEBUG - Processing node 69cb23d88c06c2799bf57d0e: baseline-arm 2026-03-31 01:47:51,316 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 01:47:51,469 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 01:47:51,471 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69cb23d88c06c2799bf57d0e/log.txt.gz 2026-03-31 01:47:51,867 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 01:47:52,132 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 01:47:52,134 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69cb23d88c06c2799bf57d0e/lava-logs.yaml 2026-03-31 01:47:52,152 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 01:47:52,350 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 01:47:52,352 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69cb23d88c06c2799bf57d0e/lava_callback.json.gz DEBUG: Setup failed for 69cb23d88c06c2799bf57d0e. Transitting job node result: incomplete -> fail 2026-03-31 01:47:52,467 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:47:52,487 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb23d88c06c2799bf57d0e HTTP/1.1" 200 None 2026-03-31 01:47:52,488 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:47:52,501 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12cb8c06c2799bf55018 HTTP/1.1" 200 None 2026-03-31 01:47:52,503 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:47:52,527 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb23d88c06c2799bf57d0e HTTP/1.1" 200 None 2026-03-31 01:47:52,633 - lava_callback - INFO - Completed processing callback for node 69cb23d88c06c2799bf57d0e INFO: 172.20.0.1:58672 - "GET / HTTP/1.0" 200 OK 2026-03-31 01:48:01,236 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:48:01,252 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:42154 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45764 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45772 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52682 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42282 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55282 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55284 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45528 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60594 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51274 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51282 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34930 - "GET / HTTP/1.0" 200 OK 2026-03-31 01:49:29,114 - lava_callback - INFO - Received callback for node 69c88521f41e4fb6a3785e3a INFO: 172.20.0.1:34942 - "POST /node/69c88521f41e4fb6a3785e3a HTTP/1.0" 202 Accepted 2026-03-31 01:49:29,120 - lava_callback - INFO - Starting async job submit for node 69c88521f41e4fb6a3785e3a 2026-03-31 01:49:29,120 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:49:29,144 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c88521f41e4fb6a3785e3a HTTP/1.1" 200 None 2026-03-31 01:49:29,145 - lava_callback - DEBUG - Processing node 69c88521f41e4fb6a3785e3a: baseline-arm64-clabbe 2026-03-31 01:49:29,145 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c88521f41e4fb6a3785e3a 2026-03-31 01:49:29,146 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:49:29,159 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c88521f41e4fb6a3785e3a HTTP/1.1" 200 None 2026-03-31 01:49:29,160 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:49:29,173 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3295318a3717aa65d01c3 HTTP/1.1" 200 None 2026-03-31 01:49:29,174 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:49:29,195 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c88521f41e4fb6a3785e3a HTTP/1.1" 200 None 2026-03-31 01:49:29,201 - lava_callback - ERROR - Error processing callback for node 69c88521f41e4fb6a3785e3a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:43526 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36960 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36974 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37828 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36176 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38010 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38026 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39284 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44316 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35334 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35336 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35608 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46228 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40744 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40760 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38308 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41532 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42254 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42260 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53458 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48298 - "GET / HTTP/1.0" 200 OK 2026-03-31 01:52:09,212 - lava_callback - INFO - Received callback for node 69c885baf41e4fb6a3785e5e INFO: 172.20.0.1:48302 - "POST /node/69c885baf41e4fb6a3785e5e HTTP/1.0" 202 Accepted 2026-03-31 01:52:09,218 - lava_callback - INFO - Starting async job submit for node 69c885baf41e4fb6a3785e5e 2026-03-31 01:52:09,218 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:52:09,272 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c885baf41e4fb6a3785e5e HTTP/1.1" 200 None 2026-03-31 01:52:09,273 - lava_callback - DEBUG - Processing node 69c885baf41e4fb6a3785e5e: baseline-arm64-clabbe 2026-03-31 01:52:09,273 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c885baf41e4fb6a3785e5e 2026-03-31 01:52:09,274 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:52:09,287 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c885baf41e4fb6a3785e5e HTTP/1.1" 200 None 2026-03-31 01:52:09,289 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:52:09,302 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3295318a3717aa65d01c3 HTTP/1.1" 200 None 2026-03-31 01:52:09,304 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:52:09,326 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c885baf41e4fb6a3785e5e HTTP/1.1" 200 None 2026-03-31 01:52:09,327 - lava_callback - ERROR - Error processing callback for node 69c885baf41e4fb6a3785e5e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:41098 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41102 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52272 - "GET / HTTP/1.0" 200 OK 2026-03-31 01:52:32,891 - lava_callback - INFO - Received callback for node 69cb28d68c06c2799bf57e01 INFO: 172.20.0.1:39382 - "POST /node/69cb28d68c06c2799bf57e01 HTTP/1.0" 202 Accepted 2026-03-31 01:52:32,897 - lava_callback - INFO - Starting async job submit for node 69cb28d68c06c2799bf57e01 2026-03-31 01:52:32,899 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:52:32,913 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb28d68c06c2799bf57e01 HTTP/1.1" 200 None 2026-03-31 01:52:32,914 - lava_callback - DEBUG - Processing node 69cb28d68c06c2799bf57e01: baseline-nfs-arm64 2026-03-31 01:52:32,915 - lava_callback - WARNING - Failed to upload log 2026-03-31 01:52:32,917 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 01:52:33,088 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 01:52:33,090 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cb28d68c06c2799bf57e01/lava-logs.yaml 2026-03-31 01:52:33,091 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 01:52:33,387 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 01:52:33,388 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cb28d68c06c2799bf57e01/lava_callback.json.gz 2026-03-31 01:52:33,390 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:52:33,405 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb28d68c06c2799bf57e01 HTTP/1.1" 200 None 2026-03-31 01:52:33,406 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:52:33,420 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12e08c06c2799bf5502b HTTP/1.1" 200 None 2026-03-31 01:52:33,421 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:52:33,447 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb28d68c06c2799bf57e01 HTTP/1.1" 200 None 2026-03-31 01:52:33,452 - lava_callback - INFO - Completed processing callback for node 69cb28d68c06c2799bf57e01 INFO: 172.20.0.1:39386 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33084 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33086 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49596 - "GET / HTTP/1.0" 200 OK 2026-03-31 01:53:01,255 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:53:01,271 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:42790 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49310 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49316 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52750 - "GET / HTTP/1.0" 200 OK 2026-03-31 01:53:29,223 - lava_callback - INFO - Received callback for node 69c88605f41e4fb6a3785e64 INFO: 172.20.0.1:52758 - "POST /node/69c88605f41e4fb6a3785e64 HTTP/1.0" 202 Accepted 2026-03-31 01:53:29,228 - lava_callback - INFO - Starting async job submit for node 69c88605f41e4fb6a3785e64 2026-03-31 01:53:29,229 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:53:29,244 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c88605f41e4fb6a3785e64 HTTP/1.1" 200 None 2026-03-31 01:53:29,245 - lava_callback - DEBUG - Processing node 69c88605f41e4fb6a3785e64: baseline-arm64-clabbe 2026-03-31 01:53:29,245 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c88605f41e4fb6a3785e64 2026-03-31 01:53:29,245 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:53:29,259 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c88605f41e4fb6a3785e64 HTTP/1.1" 200 None 2026-03-31 01:53:29,261 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:53:29,285 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3295318a3717aa65d01c3 HTTP/1.1" 200 None 2026-03-31 01:53:29,287 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:53:29,318 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c88605f41e4fb6a3785e64 HTTP/1.1" 200 None 2026-03-31 01:53:29,319 - lava_callback - ERROR - Error processing callback for node 69c88605f41e4fb6a3785e64: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-31 01:53:35,525 - lava_callback - INFO - Received callback for node 69cb24f58c06c2799bf57d48 INFO: 172.20.0.1:50962 - "POST /node/69cb24f58c06c2799bf57d48 HTTP/1.0" 202 Accepted 2026-03-31 01:53:35,532 - lava_callback - INFO - Starting async job submit for node 69cb24f58c06c2799bf57d48 2026-03-31 01:53:35,590 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:53:35,605 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb24f58c06c2799bf57d48 HTTP/1.1" 200 None 2026-03-31 01:53:35,606 - lava_callback - DEBUG - Processing node 69cb24f58c06c2799bf57d48: ltp-smoke 2026-03-31 01:53:36,097 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 01:53:36,264 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 01:53:36,266 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoke-69cb24f58c06c2799bf57d48/log.txt.gz INFO: 172.20.0.1:50968 - "GET / HTTP/1.0" 200 OK 2026-03-31 01:53:36,549 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 01:53:36,782 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 01:53:36,784 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-smoke-69cb24f58c06c2799bf57d48/lava-logs.yaml 2026-03-31 01:53:36,796 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 01:53:36,983 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 01:53:36,985 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoke-69cb24f58c06c2799bf57d48/lava_callback.json.gz 2026-03-31 01:53:37,112 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:53:37,129 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb24f58c06c2799bf57d48 HTTP/1.1" 200 None 2026-03-31 01:53:37,131 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:53:37,144 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12d38c06c2799bf55022 HTTP/1.1" 200 None 2026-03-31 01:53:37,146 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:53:37,170 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb24f58c06c2799bf57d48 HTTP/1.1" 200 None 2026-03-31 01:53:37,291 - lava_callback - INFO - Completed processing callback for node 69cb24f58c06c2799bf57d48 INFO: 172.20.0.1:58120 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58126 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47110 - "GET / HTTP/1.0" 200 OK 2026-03-31 01:54:01,273 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 01:54:01,295 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:34712 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44482 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44496 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58596 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36636 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51742 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51750 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33306 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57496 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45044 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45052 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47130 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49022 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49808 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49818 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55302 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58408 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47952 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47956 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37118 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57982 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53574 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53586 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56546 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38214 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32874 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:32890 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36626 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43684 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38056 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38068 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54560 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46166 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59384 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59396 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50288 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40712 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58996 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59006 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55056 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46120 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54614 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54616 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55174 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37250 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40680 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40694 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56208 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36036 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38496 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38502 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59386 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41354 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44352 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44356 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54226 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54610 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58572 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58588 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34674 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52484 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36142 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36150 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46534 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49660 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55996 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56008 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52490 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35466 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40472 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40480 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51444 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43234 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41128 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41130 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37294 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46778 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52574 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52578 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56874 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40828 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51866 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51876 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56794 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44336 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33392 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33394 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54644 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57606 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57450 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57454 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43234 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36648 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41578 - "GET / HTTP/1.0" 200 OK 2026-03-31 02:05:41,396 - lava_callback - INFO - Received callback for node 69cb273a8c06c2799bf57dbe INFO: 172.20.0.1:41590 - "POST /node/69cb273a8c06c2799bf57dbe HTTP/1.0" 202 Accepted 2026-03-31 02:05:41,405 - lava_callback - INFO - Starting async job submit for node 69cb273a8c06c2799bf57dbe 2026-03-31 02:05:41,470 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:05:41,485 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb273a8c06c2799bf57dbe HTTP/1.1" 200 None 2026-03-31 02:05:41,486 - lava_callback - DEBUG - Processing node 69cb273a8c06c2799bf57dbe: baseline-nfs-arm64 2026-03-31 02:05:42,882 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 02:05:43,072 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 02:05:43,074 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cb273a8c06c2799bf57dbe/log.txt.gz 2026-03-31 02:05:43,714 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 02:05:43,939 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 02:05:43,942 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cb273a8c06c2799bf57dbe/lava-logs.yaml 2026-03-31 02:05:43,965 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 02:05:44,160 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 02:05:44,162 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cb273a8c06c2799bf57dbe/lava_callback.json.gz DEBUG: Setup failed for 69cb273a8c06c2799bf57dbe. Transitting job node result: incomplete -> fail 2026-03-31 02:05:44,276 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:05:44,291 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb273a8c06c2799bf57dbe HTTP/1.1" 200 None 2026-03-31 02:05:44,292 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:05:44,307 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12e08c06c2799bf5502b HTTP/1.1" 200 None 2026-03-31 02:05:44,308 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:05:44,336 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb273a8c06c2799bf57dbe HTTP/1.1" 200 None 2026-03-31 02:05:44,441 - lava_callback - INFO - Completed processing callback for node 69cb273a8c06c2799bf57dbe INFO: 172.20.0.1:41606 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56938 - "GET / HTTP/1.0" 200 OK 2026-03-31 02:06:01,299 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:06:01,328 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:57802 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55434 - "GET /node/69a15fe3a41e9451dcf411fa%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:55450 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55466 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60474 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53134 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42702 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42712 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43112 - "GET / HTTP/1.0" 200 OK 2026-03-31 02:06:58,627 - lava_callback - INFO - Received callback for node 69cb2c098c06c2799bf57e76 INFO: 172.20.0.1:43120 - "POST /node/69cb2c098c06c2799bf57e76 HTTP/1.0" 202 Accepted 2026-03-31 02:06:58,635 - lava_callback - INFO - Starting async job submit for node 69cb2c098c06c2799bf57e76 2026-03-31 02:06:58,663 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:06:58,677 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb2c098c06c2799bf57e76 HTTP/1.1" 200 None 2026-03-31 02:06:58,678 - lava_callback - DEBUG - Processing node 69cb2c098c06c2799bf57e76: baseline-arm64 2026-03-31 02:06:58,827 - lava_callback - WARNING - Failed to upload log 2026-03-31 02:06:58,903 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 02:06:59,083 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 02:06:59,085 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb2c098c06c2799bf57e76/lava-logs.yaml 2026-03-31 02:06:59,088 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 02:06:59,227 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 02:06:59,228 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb2c098c06c2799bf57e76/lava_callback.json.gz 2026-03-31 02:06:59,258 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:06:59,278 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb2c098c06c2799bf57e76 HTTP/1.1" 200 None 2026-03-31 02:06:59,280 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:06:59,293 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12e08c06c2799bf5502b HTTP/1.1" 200 None 2026-03-31 02:06:59,295 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:06:59,315 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb2c098c06c2799bf57e76 HTTP/1.1" 200 None 2026-03-31 02:06:59,356 - lava_callback - INFO - Completed processing callback for node 69cb2c098c06c2799bf57e76 2026-03-31 02:07:01,330 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:07:01,346 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:38380 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54052 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54062 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60488 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58516 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42032 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42046 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58422 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40220 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47872 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47884 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49420 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50624 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48548 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48556 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36402 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41074 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52690 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52704 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35746 - "GET / HTTP/1.0" 200 OK 2026-03-31 02:09:36,213 - lava_callback - INFO - Received callback for node 69cb2caa8c06c2799bf57e8d INFO: 172.20.0.1:40510 - "POST /node/69cb2caa8c06c2799bf57e8d HTTP/1.0" 202 Accepted 2026-03-31 02:09:36,220 - lava_callback - INFO - Starting async job submit for node 69cb2caa8c06c2799bf57e8d 2026-03-31 02:09:36,252 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:09:36,268 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb2caa8c06c2799bf57e8d HTTP/1.1" 200 None 2026-03-31 02:09:36,269 - lava_callback - DEBUG - Processing node 69cb2caa8c06c2799bf57e8d: baseline-arm64 2026-03-31 02:09:36,428 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:40514 - "GET / HTTP/1.0" 200 OK 2026-03-31 02:09:36,591 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 02:09:36,593 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb2caa8c06c2799bf57e8d/log.txt.gz 2026-03-31 02:09:36,671 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 02:09:36,839 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 02:09:36,841 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb2caa8c06c2799bf57e8d/lava-logs.yaml 2026-03-31 02:09:36,846 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 02:09:36,986 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 02:09:36,987 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb2caa8c06c2799bf57e8d/lava_callback.json.gz 2026-03-31 02:09:37,018 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:09:37,036 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb2caa8c06c2799bf57e8d HTTP/1.1" 200 None 2026-03-31 02:09:37,038 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:09:37,061 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12e08c06c2799bf5502b HTTP/1.1" 200 None 2026-03-31 02:09:37,063 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:09:37,096 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb2caa8c06c2799bf57e8d HTTP/1.1" 200 None 2026-03-31 02:09:37,161 - lava_callback - INFO - Completed processing callback for node 69cb2caa8c06c2799bf57e8d INFO: 172.20.0.1:56108 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56118 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58114 - "GET / HTTP/1.0" 200 OK 2026-03-31 02:10:01,348 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:10:01,367 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:34606 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44456 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44472 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50584 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47388 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48108 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48122 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35090 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43912 - "GET /node/6993b22acebe99dd8e07aae0%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:43926 - "GET /node/69876b97e31b58f834f1982d%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:43938 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40364 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40368 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46176 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59512 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43916 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43928 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57954 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41176 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56196 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56206 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51084 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43084 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45302 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45306 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33906 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37612 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60616 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60626 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38478 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53436 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59922 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59928 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56058 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59006 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56288 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56300 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39614 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55782 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55652 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55664 - "GET / HTTP/1.0" 200 OK 2026-03-31 02:14:49,580 - lava_callback - INFO - Received callback for node 69cb2c5e8c06c2799bf57e86 INFO: 172.20.0.1:55680 - "POST /node/69cb2c5e8c06c2799bf57e86 HTTP/1.0" 202 Accepted 2026-03-31 02:14:49,587 - lava_callback - INFO - Starting async job submit for node 69cb2c5e8c06c2799bf57e86 2026-03-31 02:14:49,625 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:14:49,641 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb2c5e8c06c2799bf57e86 HTTP/1.1" 200 None 2026-03-31 02:14:49,642 - lava_callback - DEBUG - Processing node 69cb2c5e8c06c2799bf57e86: baseline-arm64 2026-03-31 02:14:50,005 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 02:14:50,158 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 02:14:50,159 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb2c5e8c06c2799bf57e86/log.txt.gz 2026-03-31 02:14:50,391 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 02:14:50,613 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 02:14:50,615 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb2c5e8c06c2799bf57e86/lava-logs.yaml 2026-03-31 02:14:50,623 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 02:14:50,820 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 02:14:50,822 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb2c5e8c06c2799bf57e86/lava_callback.json.gz DEBUG: Setup failed for 69cb2c5e8c06c2799bf57e86. Transitting job node result: incomplete -> fail 2026-03-31 02:14:50,897 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:14:50,910 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb2c5e8c06c2799bf57e86 HTTP/1.1" 200 None 2026-03-31 02:14:50,912 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:14:50,925 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12e08c06c2799bf5502b HTTP/1.1" 200 None 2026-03-31 02:14:50,927 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:14:50,954 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb2c5e8c06c2799bf57e86 HTTP/1.1" 200 None 2026-03-31 02:14:51,032 - lava_callback - INFO - Completed processing callback for node 69cb2c5e8c06c2799bf57e86 INFO: 172.20.0.1:44810 - "GET / HTTP/1.0" 200 OK 2026-03-31 02:15:01,370 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:15:01,396 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:37694 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35326 - "GET /node/699113fa85f550420785db5d%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:35334 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35342 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40430 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44446 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38590 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38606 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55670 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56588 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41786 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41796 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47630 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50996 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34118 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34120 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42866 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37204 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47346 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47360 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58486 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47430 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39180 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39188 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33996 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46282 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37324 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37336 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43092 - "GET / HTTP/1.0" 200 OK 2026-03-31 02:18:36,480 - lava_callback - INFO - Received callback for node 69cb1bc88c06c2799bf55245 INFO: 172.20.0.1:38928 - "POST /node/69cb1bc88c06c2799bf55245 HTTP/1.0" 202 Accepted 2026-03-31 02:18:36,485 - lava_callback - INFO - Starting async job submit for node 69cb1bc88c06c2799bf55245 2026-03-31 02:18:36,508 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.20.0.1:38944 - "GET / HTTP/1.0" 200 OK 2026-03-31 02:18:36,523 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1bc88c06c2799bf55245 HTTP/1.1" 200 None 2026-03-31 02:18:36,523 - lava_callback - DEBUG - Processing node 69cb1bc88c06c2799bf55245: baseline-arm-broonie 2026-03-31 02:18:36,723 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 02:18:36,885 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 02:18:36,887 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69cb1bc88c06c2799bf55245/log.txt.gz 2026-03-31 02:18:37,008 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 02:18:37,219 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 02:18:37,221 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69cb1bc88c06c2799bf55245/lava-logs.yaml 2026-03-31 02:18:37,227 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 02:18:37,393 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 02:18:37,394 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69cb1bc88c06c2799bf55245/lava_callback.json.gz 2026-03-31 02:18:37,417 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:18:37,433 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1bc88c06c2799bf55245 HTTP/1.1" 200 None 2026-03-31 02:18:37,434 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:18:37,449 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12cb8c06c2799bf55018 HTTP/1.1" 200 None 2026-03-31 02:18:37,450 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:18:37,475 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb1bc88c06c2799bf55245 HTTP/1.1" 200 None 2026-03-31 02:18:37,476 - lava_callback - INFO - Completed processing callback for node 69cb1bc88c06c2799bf55245 INFO: 172.20.0.1:38948 - "GET /node/693ae80efee8383e92b6562c%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:38956 - "GET /node/69876cb5e31b58f834f199d8%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:53414 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53422 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34148 - "GET / HTTP/1.0" 200 OK 2026-03-31 02:19:01,399 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:19:01,415 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:36048 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57248 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57260 - "GET / HTTP/1.0" 200 OK 2026-03-31 02:19:25,631 - lava_callback - INFO - Received callback for node 69c88c2bf41e4fb6a3785f87 INFO: 172.20.0.1:34722 - "POST /node/69c88c2bf41e4fb6a3785f87 HTTP/1.0" 202 Accepted 2026-03-31 02:19:25,637 - lava_callback - INFO - Starting async job submit for node 69c88c2bf41e4fb6a3785f87 2026-03-31 02:19:25,638 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:19:25,652 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c88c2bf41e4fb6a3785f87 HTTP/1.1" 200 None 2026-03-31 02:19:25,653 - lava_callback - DEBUG - Processing node 69c88c2bf41e4fb6a3785f87: baseline-arm64-baylibre 2026-03-31 02:19:25,653 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c88c2bf41e4fb6a3785f87 2026-03-31 02:19:25,653 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:19:25,667 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c88c2bf41e4fb6a3785f87 HTTP/1.1" 200 None 2026-03-31 02:19:25,668 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:19:25,681 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086b1ce0c9bc43bccb63a HTTP/1.1" 200 None 2026-03-31 02:19:25,682 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:19:25,706 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c88c2bf41e4fb6a3785f87 HTTP/1.1" 200 None 2026-03-31 02:19:25,708 - lava_callback - ERROR - Error processing callback for node 69c88c2bf41e4fb6a3785f87: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:34728 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48614 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34814 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34820 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45764 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34074 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44114 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44120 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50830 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45948 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37864 - "GET /node/693e2ce8407f32424eb592b7%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:37876 - "GET /node/694ac6746c96ed51d316f2de%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:37882 - "GET / HTTP/1.0" 200 OK 2026-03-31 02:20:45,717 - lava_callback - INFO - Received callback for node 69c88c7af41e4fb6a3785f95 INFO: 172.20.0.1:37884 - "POST /node/69c88c7af41e4fb6a3785f95 HTTP/1.0" 202 Accepted 2026-03-31 02:20:45,723 - lava_callback - INFO - Starting async job submit for node 69c88c7af41e4fb6a3785f95 2026-03-31 02:20:45,724 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:20:45,738 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c88c7af41e4fb6a3785f95 HTTP/1.1" 200 None 2026-03-31 02:20:45,739 - lava_callback - DEBUG - Processing node 69c88c7af41e4fb6a3785f95: baseline-arm64-baylibre 2026-03-31 02:20:45,739 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c88c7af41e4fb6a3785f95 2026-03-31 02:20:45,740 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:20:45,753 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c88c7af41e4fb6a3785f95 HTTP/1.1" 200 None 2026-03-31 02:20:45,754 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:20:45,767 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086b1ce0c9bc43bccb63a HTTP/1.1" 200 None 2026-03-31 02:20:45,769 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:20:45,789 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c88c7af41e4fb6a3785f95 HTTP/1.1" 200 None 2026-03-31 02:20:45,797 - lava_callback - ERROR - Error processing callback for node 69c88c7af41e4fb6a3785f95: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:37890 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48664 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35390 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42030 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42038 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52340 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47594 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43220 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43234 - "GET / HTTP/1.0" 200 OK 2026-03-31 02:21:55,271 - lava_callback - INFO - Received callback for node 69cb1bf48c06c2799bf552e5 INFO: 172.20.0.1:34780 - "POST /node/69cb1bf48c06c2799bf552e5 HTTP/1.0" 202 Accepted 2026-03-31 02:21:55,277 - lava_callback - INFO - Starting async job submit for node 69cb1bf48c06c2799bf552e5 2026-03-31 02:21:55,303 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:21:55,323 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1bf48c06c2799bf552e5 HTTP/1.1" 200 None 2026-03-31 02:21:55,325 - lava_callback - DEBUG - Processing node 69cb1bf48c06c2799bf552e5: ltp-cap_bounds 2026-03-31 02:21:55,673 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 02:21:55,788 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 02:21:55,790 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-cap_bounds-69cb1bf48c06c2799bf552e5/log.txt.gz 2026-03-31 02:21:56,019 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 02:21:56,225 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 02:21:56,227 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-cap_bounds-69cb1bf48c06c2799bf552e5/lava-logs.yaml 2026-03-31 02:21:56,236 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 02:21:56,385 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 02:21:56,387 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-cap_bounds-69cb1bf48c06c2799bf552e5/lava_callback.json.gz 2026-03-31 02:21:56,413 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:21:56,427 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1bf48c06c2799bf552e5 HTTP/1.1" 200 None 2026-03-31 02:21:56,429 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:21:56,442 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12cb8c06c2799bf55018 HTTP/1.1" 200 None 2026-03-31 02:21:56,444 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:21:56,468 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb1bf48c06c2799bf552e5 HTTP/1.1" 200 None 2026-03-31 02:21:56,469 - lava_callback - INFO - Completed processing callback for node 69cb1bf48c06c2799bf552e5 INFO: 172.20.0.1:34796 - "GET / HTTP/1.0" 200 OK 2026-03-31 02:21:56,909 - lava_callback - INFO - Received callback for node 69cb1bf68c06c2799bf552e8 INFO: 172.20.0.1:34800 - "POST /node/69cb1bf68c06c2799bf552e8 HTTP/1.0" 202 Accepted 2026-03-31 02:21:56,917 - lava_callback - INFO - Starting async job submit for node 69cb1bf68c06c2799bf552e8 2026-03-31 02:21:56,958 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:21:56,980 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1bf68c06c2799bf552e8 HTTP/1.1" 200 None 2026-03-31 02:21:56,982 - lava_callback - DEBUG - Processing node 69cb1bf68c06c2799bf552e8: ltp-containers 2026-03-31 02:21:57,406 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 02:21:57,571 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 02:21:57,573 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-containers-69cb1bf68c06c2799bf552e8/log.txt.gz 2026-03-31 02:21:57,850 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 02:21:58,054 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 02:21:58,057 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-containers-69cb1bf68c06c2799bf552e8/lava-logs.yaml 2026-03-31 02:21:58,068 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 02:21:58,248 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 02:21:58,251 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-containers-69cb1bf68c06c2799bf552e8/lava_callback.json.gz 2026-03-31 02:21:58,277 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:21:58,307 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1bf68c06c2799bf552e8 HTTP/1.1" 200 None 2026-03-31 02:21:58,309 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:21:58,323 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12cb8c06c2799bf55018 HTTP/1.1" 200 None 2026-03-31 02:21:58,325 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:21:58,350 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb1bf68c06c2799bf552e8 HTTP/1.1" 200 None 2026-03-31 02:21:58,351 - lava_callback - INFO - Completed processing callback for node 69cb1bf68c06c2799bf552e8 2026-03-31 02:22:01,418 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:22:01,435 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:49864 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36756 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36758 - "GET / HTTP/1.0" 200 OK 2026-03-31 02:22:25,818 - lava_callback - INFO - Received callback for node 69c88ce0f41e4fb6a3785fa2 INFO: 172.20.0.1:44050 - "POST /node/69c88ce0f41e4fb6a3785fa2 HTTP/1.0" 202 Accepted 2026-03-31 02:22:25,823 - lava_callback - INFO - Starting async job submit for node 69c88ce0f41e4fb6a3785fa2 2026-03-31 02:22:25,824 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:22:25,839 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c88ce0f41e4fb6a3785fa2 HTTP/1.1" 200 None 2026-03-31 02:22:25,840 - lava_callback - DEBUG - Processing node 69c88ce0f41e4fb6a3785fa2: baseline-arm64-baylibre 2026-03-31 02:22:25,840 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c88ce0f41e4fb6a3785fa2 2026-03-31 02:22:25,841 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:22:25,854 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c88ce0f41e4fb6a3785fa2 HTTP/1.1" 200 None 2026-03-31 02:22:25,856 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:22:25,874 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086b1ce0c9bc43bccb63a HTTP/1.1" 200 None 2026-03-31 02:22:25,875 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:22:25,898 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c88ce0f41e4fb6a3785fa2 HTTP/1.1" 200 None 2026-03-31 02:22:25,899 - lava_callback - ERROR - Error processing callback for node 69c88ce0f41e4fb6a3785fa2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:44054 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50720 - "GET / HTTP/1.0" 200 OK 2026-03-31 02:22:36,544 - lava_callback - INFO - Received callback for node 69cb1bf78c06c2799bf552eb INFO: 172.20.0.1:50724 - "POST /node/69cb1bf78c06c2799bf552eb HTTP/1.0" 202 Accepted 2026-03-31 02:22:36,550 - lava_callback - INFO - Starting async job submit for node 69cb1bf78c06c2799bf552eb 2026-03-31 02:22:36,578 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:22:36,597 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1bf78c06c2799bf552eb HTTP/1.1" 200 None 2026-03-31 02:22:36,598 - lava_callback - DEBUG - Processing node 69cb1bf78c06c2799bf552eb: ltp-controllers 2026-03-31 02:22:37,095 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 02:22:37,240 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 02:22:37,242 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-controllers-69cb1bf78c06c2799bf552eb/log.txt.gz 2026-03-31 02:22:37,510 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 02:22:37,714 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 02:22:37,717 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-controllers-69cb1bf78c06c2799bf552eb/lava-logs.yaml 2026-03-31 02:22:37,727 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 02:22:37,898 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 02:22:37,899 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-controllers-69cb1bf78c06c2799bf552eb/lava_callback.json.gz 2026-03-31 02:22:37,925 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:22:37,941 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1bf78c06c2799bf552eb HTTP/1.1" 200 None 2026-03-31 02:22:37,943 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:22:37,966 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12cb8c06c2799bf55018 HTTP/1.1" 200 None 2026-03-31 02:22:37,968 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:22:38,001 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb1bf78c06c2799bf552eb HTTP/1.1" 200 None 2026-03-31 02:22:38,002 - lava_callback - INFO - Completed processing callback for node 69cb1bf78c06c2799bf552eb INFO: 172.20.0.1:55184 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55194 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47840 - "GET / HTTP/1.0" 200 OK 2026-03-31 02:23:01,437 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:23:01,454 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:33328 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42770 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42780 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46970 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55246 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58696 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58700 - "GET / HTTP/1.0" 200 OK 2026-03-31 02:23:55,607 - lava_callback - INFO - Received callback for node 69cb2ee48c06c2799bf57ee8 INFO: 172.20.0.1:42960 - "POST /node/69cb2ee48c06c2799bf57ee8 HTTP/1.0" 202 Accepted 2026-03-31 02:23:55,615 - lava_callback - INFO - Starting async job submit for node 69cb2ee48c06c2799bf57ee8 2026-03-31 02:23:55,663 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:23:55,685 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb2ee48c06c2799bf57ee8 HTTP/1.1" 200 None 2026-03-31 02:23:55,686 - lava_callback - DEBUG - Processing node 69cb2ee48c06c2799bf57ee8: baseline-arm64 2026-03-31 02:23:56,107 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 02:23:56,235 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 02:23:56,237 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb2ee48c06c2799bf57ee8/log.txt.gz 2026-03-31 02:23:56,446 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:42968 - "GET / HTTP/1.0" 200 OK 2026-03-31 02:23:56,647 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 02:23:56,649 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb2ee48c06c2799bf57ee8/lava-logs.yaml 2026-03-31 02:23:56,657 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 02:23:56,807 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 02:23:56,809 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb2ee48c06c2799bf57ee8/lava_callback.json.gz DEBUG: Setup failed for 69cb2ee48c06c2799bf57ee8. Transitting job node result: incomplete -> fail 2026-03-31 02:23:56,880 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:23:56,894 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb2ee48c06c2799bf57ee8 HTTP/1.1" 200 None 2026-03-31 02:23:56,896 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:23:56,909 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12e08c06c2799bf5502b HTTP/1.1" 200 None 2026-03-31 02:23:56,911 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:23:56,934 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb2ee48c06c2799bf57ee8 HTTP/1.1" 200 None 2026-03-31 02:23:57,007 - lava_callback - INFO - Completed processing callback for node 69cb2ee48c06c2799bf57ee8 2026-03-31 02:24:01,456 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:24:01,472 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:34232 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60964 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60970 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56792 - "GET /node/693b86b541fb4c95097665ff%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:56802 - "GET /node/69937f43cebe99dd8e06d0a7%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:56808 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53074 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42058 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42064 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50596 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35760 - "GET /node/69938055cebe99dd8e06d7b8%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:35770 - "GET /node/69839c9966dc8a56088a6396%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:35774 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45000 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45004 - "GET / HTTP/1.0" 200 OK 2026-03-31 02:25:26,009 - lava_callback - INFO - Received callback for node 69c88d81f41e4fb6a3785fb3 INFO: 172.20.0.1:53130 - "POST /node/69c88d81f41e4fb6a3785fb3 HTTP/1.0" 202 Accepted 2026-03-31 02:25:26,014 - lava_callback - INFO - Starting async job submit for node 69c88d81f41e4fb6a3785fb3 2026-03-31 02:25:26,015 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:25:26,030 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c88d81f41e4fb6a3785fb3 HTTP/1.1" 200 None 2026-03-31 02:25:26,031 - lava_callback - DEBUG - Processing node 69c88d81f41e4fb6a3785fb3: baseline-arm64-baylibre 2026-03-31 02:25:26,032 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c88d81f41e4fb6a3785fb3 2026-03-31 02:25:26,032 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:25:26,049 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c88d81f41e4fb6a3785fb3 HTTP/1.1" 200 None 2026-03-31 02:25:26,051 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:25:26,067 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086b1ce0c9bc43bccb63a HTTP/1.1" 200 None 2026-03-31 02:25:26,070 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:25:26,101 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c88d81f41e4fb6a3785fb3 HTTP/1.1" 200 None 2026-03-31 02:25:26,102 - lava_callback - ERROR - Error processing callback for node 69c88d81f41e4fb6a3785fb3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:53144 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40508 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40330 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40342 - "GET / HTTP/1.0" 200 OK 2026-03-31 02:25:55,413 - lava_callback - INFO - Received callback for node 69cb1bfa8c06c2799bf552f7 INFO: 172.20.0.1:47728 - "POST /node/69cb1bfa8c06c2799bf552f7 HTTP/1.0" 202 Accepted 2026-03-31 02:25:55,420 - lava_callback - INFO - Starting async job submit for node 69cb1bfa8c06c2799bf552f7 2026-03-31 02:25:55,449 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:25:55,464 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1bfa8c06c2799bf552f7 HTTP/1.1" 200 None 2026-03-31 02:25:55,465 - lava_callback - DEBUG - Processing node 69cb1bfa8c06c2799bf552f7: ltp-cve 2026-03-31 02:25:55,874 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 02:25:56,019 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 02:25:56,021 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-cve-69cb1bfa8c06c2799bf552f7/log.txt.gz 2026-03-31 02:25:56,234 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 02:25:56,444 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 02:25:56,446 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-cve-69cb1bfa8c06c2799bf552f7/lava-logs.yaml 2026-03-31 02:25:56,455 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:47732 - "GET / HTTP/1.0" 200 OK 2026-03-31 02:25:56,618 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 02:25:56,620 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-cve-69cb1bfa8c06c2799bf552f7/lava_callback.json.gz 2026-03-31 02:25:56,647 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:25:56,661 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1bfa8c06c2799bf552f7 HTTP/1.1" 200 None 2026-03-31 02:25:56,663 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:25:56,676 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12cb8c06c2799bf55018 HTTP/1.1" 200 None 2026-03-31 02:25:56,678 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:25:56,704 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb1bfa8c06c2799bf552f7 HTTP/1.1" 200 None 2026-03-31 02:25:56,710 - lava_callback - INFO - Completed processing callback for node 69cb1bfa8c06c2799bf552f7 2026-03-31 02:25:58,248 - lava_callback - INFO - Received callback for node 69cb1bfe8c06c2799bf552fd INFO: 172.20.0.1:47736 - "POST /node/69cb1bfe8c06c2799bf552fd HTTP/1.0" 202 Accepted 2026-03-31 02:25:58,255 - lava_callback - INFO - Starting async job submit for node 69cb1bfe8c06c2799bf552fd 2026-03-31 02:25:58,298 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:25:58,316 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1bfe8c06c2799bf552fd HTTP/1.1" 200 None 2026-03-31 02:25:58,317 - lava_callback - DEBUG - Processing node 69cb1bfe8c06c2799bf552fd: ltp-fs_bind 2026-03-31 02:25:58,684 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 02:25:58,835 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 02:25:58,837 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-fs_bind-69cb1bfe8c06c2799bf552fd/log.txt.gz 2026-03-31 02:25:59,054 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 02:25:59,272 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 02:25:59,274 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-fs_bind-69cb1bfe8c06c2799bf552fd/lava-logs.yaml 2026-03-31 02:25:59,283 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 02:25:59,444 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 02:25:59,446 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-fs_bind-69cb1bfe8c06c2799bf552fd/lava_callback.json.gz 2026-03-31 02:25:59,472 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:25:59,487 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1bfe8c06c2799bf552fd HTTP/1.1" 200 None 2026-03-31 02:25:59,488 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:25:59,502 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12cb8c06c2799bf55018 HTTP/1.1" 200 None 2026-03-31 02:25:59,504 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:25:59,528 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb1bfe8c06c2799bf552fd HTTP/1.1" 200 None 2026-03-31 02:25:59,529 - lava_callback - INFO - Completed processing callback for node 69cb1bfe8c06c2799bf552fd 2026-03-31 02:26:01,474 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:26:01,490 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:52720 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33842 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33856 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38914 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36300 - "GET / HTTP/1.0" 200 OK 2026-03-31 02:26:36,671 - lava_callback - INFO - Received callback for node 69cb1c038c06c2799bf55303 INFO: 172.20.0.1:36314 - "POST /node/69cb1c038c06c2799bf55303 HTTP/1.0" 202 Accepted 2026-03-31 02:26:36,678 - lava_callback - INFO - Starting async job submit for node 69cb1c038c06c2799bf55303 2026-03-31 02:26:36,702 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:26:36,717 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1c038c06c2799bf55303 HTTP/1.1" 200 None 2026-03-31 02:26:36,718 - lava_callback - DEBUG - Processing node 69cb1c038c06c2799bf55303: ltp-input 2026-03-31 02:26:37,093 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 02:26:37,236 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 02:26:37,238 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-input-69cb1c038c06c2799bf55303/log.txt.gz 2026-03-31 02:26:37,488 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 02:26:37,720 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 02:26:37,723 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-input-69cb1c038c06c2799bf55303/lava-logs.yaml 2026-03-31 02:26:37,732 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 02:26:37,905 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 02:26:37,907 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-input-69cb1c038c06c2799bf55303/lava_callback.json.gz 2026-03-31 02:26:37,935 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:26:37,949 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1c038c06c2799bf55303 HTTP/1.1" 200 None 2026-03-31 02:26:37,951 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:26:37,971 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12cb8c06c2799bf55018 HTTP/1.1" 200 None 2026-03-31 02:26:37,973 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:26:38,005 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb1c038c06c2799bf55303 HTTP/1.1" 200 None 2026-03-31 02:26:38,008 - lava_callback - INFO - Completed processing callback for node 69cb1c038c06c2799bf55303 INFO: 172.20.0.1:34648 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34650 - "GET / HTTP/1.0" 200 OK 2026-03-31 02:26:55,383 - lava_callback - INFO - Received callback for node 69cb1c018c06c2799bf55300 INFO: 172.20.0.1:34770 - "POST /node/69cb1c018c06c2799bf55300 HTTP/1.0" 202 Accepted 2026-03-31 02:26:55,390 - lava_callback - INFO - Starting async job submit for node 69cb1c018c06c2799bf55300 2026-03-31 02:26:55,416 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:26:55,431 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1c018c06c2799bf55300 HTTP/1.1" 200 None 2026-03-31 02:26:55,432 - lava_callback - DEBUG - Processing node 69cb1c018c06c2799bf55300: ltp-fsx 2026-03-31 02:26:55,796 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 02:26:55,933 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 02:26:55,934 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-fsx-69cb1c018c06c2799bf55300/log.txt.gz 2026-03-31 02:26:56,161 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 02:26:56,370 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 02:26:56,373 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-fsx-69cb1c018c06c2799bf55300/lava-logs.yaml 2026-03-31 02:26:56,381 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:34786 - "GET / HTTP/1.0" 200 OK 2026-03-31 02:26:56,534 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 02:26:56,535 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-fsx-69cb1c018c06c2799bf55300/lava_callback.json.gz 2026-03-31 02:26:56,561 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:26:56,576 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1c018c06c2799bf55300 HTTP/1.1" 200 None 2026-03-31 02:26:56,577 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:26:56,591 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12cb8c06c2799bf55018 HTTP/1.1" 200 None 2026-03-31 02:26:56,593 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:26:56,623 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb1c018c06c2799bf55300 HTTP/1.1" 200 None 2026-03-31 02:26:56,625 - lava_callback - INFO - Completed processing callback for node 69cb1c018c06c2799bf55300 2026-03-31 02:27:01,492 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:27:01,509 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:34092 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45340 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45356 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33184 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41992 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47930 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47944 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60458 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43102 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51490 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51502 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43168 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58650 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37680 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37694 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52376 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45030 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37090 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37092 - "GET / HTTP/1.0" 200 OK 2026-03-31 02:29:26,288 - lava_callback - INFO - Received callback for node 69c88e71f41e4fb6a3785fd2 INFO: 172.20.0.1:37786 - "POST /node/69c88e71f41e4fb6a3785fd2 HTTP/1.0" 202 Accepted 2026-03-31 02:29:26,294 - lava_callback - INFO - Starting async job submit for node 69c88e71f41e4fb6a3785fd2 2026-03-31 02:29:26,295 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:29:26,309 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c88e71f41e4fb6a3785fd2 HTTP/1.1" 200 None 2026-03-31 02:29:26,310 - lava_callback - DEBUG - Processing node 69c88e71f41e4fb6a3785fd2: baseline-arm64-baylibre 2026-03-31 02:29:26,310 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c88e71f41e4fb6a3785fd2 2026-03-31 02:29:26,311 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:29:26,326 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c88e71f41e4fb6a3785fd2 HTTP/1.1" 200 None 2026-03-31 02:29:26,327 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:29:26,345 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086b1ce0c9bc43bccb63a HTTP/1.1" 200 None 2026-03-31 02:29:26,347 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:29:26,368 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c88e71f41e4fb6a3785fd2 HTTP/1.1" 200 None 2026-03-31 02:29:26,369 - lava_callback - ERROR - Error processing callback for node 69c88e71f41e4fb6a3785fd2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:37800 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42122 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39870 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39880 - "GET / HTTP/1.0" 200 OK 2026-03-31 02:29:55,514 - lava_callback - INFO - Received callback for node 69cb1c058c06c2799bf55306 INFO: 172.20.0.1:50194 - "POST /node/69cb1c058c06c2799bf55306 HTTP/1.0" 202 Accepted 2026-03-31 02:29:55,522 - lava_callback - INFO - Starting async job submit for node 69cb1c058c06c2799bf55306 2026-03-31 02:29:55,548 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:29:55,562 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1c058c06c2799bf55306 HTTP/1.1" 200 None 2026-03-31 02:29:55,563 - lava_callback - DEBUG - Processing node 69cb1c058c06c2799bf55306: ltp-ipc 2026-03-31 02:29:55,953 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 02:29:56,083 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 02:29:56,084 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-ipc-69cb1c058c06c2799bf55306/log.txt.gz 2026-03-31 02:29:56,292 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:50200 - "GET / HTTP/1.0" 200 OK 2026-03-31 02:29:56,511 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 02:29:56,513 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-ipc-69cb1c058c06c2799bf55306/lava-logs.yaml 2026-03-31 02:29:56,521 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 02:29:56,719 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 02:29:56,721 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-ipc-69cb1c058c06c2799bf55306/lava_callback.json.gz 2026-03-31 02:29:56,747 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:29:56,761 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1c058c06c2799bf55306 HTTP/1.1" 200 None 2026-03-31 02:29:56,762 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:29:56,778 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12cb8c06c2799bf55018 HTTP/1.1" 200 None 2026-03-31 02:29:56,779 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:29:56,804 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb1c058c06c2799bf55306 HTTP/1.1" 200 None 2026-03-31 02:29:56,805 - lava_callback - INFO - Completed processing callback for node 69cb1c058c06c2799bf55306 2026-03-31 02:29:57,438 - lava_callback - INFO - Received callback for node 69cb1c088c06c2799bf55309 INFO: 172.20.0.1:50206 - "POST /node/69cb1c088c06c2799bf55309 HTTP/1.0" 202 Accepted 2026-03-31 02:29:57,445 - lava_callback - INFO - Starting async job submit for node 69cb1c088c06c2799bf55309 2026-03-31 02:29:57,479 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:29:57,494 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1c088c06c2799bf55309 HTTP/1.1" 200 None 2026-03-31 02:29:57,495 - lava_callback - DEBUG - Processing node 69cb1c088c06c2799bf55309: ltp-pty 2026-03-31 02:29:57,899 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 02:29:58,068 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 02:29:58,069 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-pty-69cb1c088c06c2799bf55309/log.txt.gz 2026-03-31 02:29:58,346 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 02:29:58,585 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 02:29:58,587 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-pty-69cb1c088c06c2799bf55309/lava-logs.yaml 2026-03-31 02:29:58,595 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 02:29:58,767 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 02:29:58,769 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-pty-69cb1c088c06c2799bf55309/lava_callback.json.gz 2026-03-31 02:29:58,794 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:29:58,808 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1c088c06c2799bf55309 HTTP/1.1" 200 None 2026-03-31 02:29:58,809 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:29:58,824 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12cb8c06c2799bf55018 HTTP/1.1" 200 None 2026-03-31 02:29:58,825 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:29:58,853 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb1c088c06c2799bf55309 HTTP/1.1" 200 None 2026-03-31 02:29:58,855 - lava_callback - INFO - Completed processing callback for node 69cb1c088c06c2799bf55309 2026-03-31 02:30:01,511 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:30:01,533 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:55314 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54074 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54076 - "GET / HTTP/1.0" 200 OK 2026-03-31 02:30:26,212 - lava_callback - INFO - Received callback for node 69c88ebcf41e4fb6a3785fdf INFO: 172.20.0.1:49666 - "POST /node/69c88ebcf41e4fb6a3785fdf HTTP/1.0" 202 Accepted 2026-03-31 02:30:26,218 - lava_callback - INFO - Starting async job submit for node 69c88ebcf41e4fb6a3785fdf 2026-03-31 02:30:26,219 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:30:26,233 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c88ebcf41e4fb6a3785fdf HTTP/1.1" 200 None 2026-03-31 02:30:26,234 - lava_callback - DEBUG - Processing node 69c88ebcf41e4fb6a3785fdf: baseline-arm64-baylibre 2026-03-31 02:30:26,234 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c88ebcf41e4fb6a3785fdf 2026-03-31 02:30:26,235 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:30:26,249 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c88ebcf41e4fb6a3785fdf HTTP/1.1" 200 None 2026-03-31 02:30:26,250 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:30:26,264 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086b1ce0c9bc43bccb63a HTTP/1.1" 200 None 2026-03-31 02:30:26,266 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:30:26,288 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c88ebcf41e4fb6a3785fdf HTTP/1.1" 200 None 2026-03-31 02:30:26,289 - lava_callback - ERROR - Error processing callback for node 69c88ebcf41e4fb6a3785fdf: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:49682 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44874 - "GET / HTTP/1.0" 200 OK 2026-03-31 02:30:36,893 - lava_callback - INFO - Received callback for node 69cb1c0a8c06c2799bf55310 INFO: 172.20.0.1:44884 - "POST /node/69cb1c0a8c06c2799bf55310 HTTP/1.0" 202 Accepted 2026-03-31 02:30:36,901 - lava_callback - INFO - Starting async job submit for node 69cb1c0a8c06c2799bf55310 2026-03-31 02:30:36,929 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:30:36,946 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1c0a8c06c2799bf55310 HTTP/1.1" 200 None 2026-03-31 02:30:36,947 - lava_callback - DEBUG - Processing node 69cb1c0a8c06c2799bf55310: ltp-sched 2026-03-31 02:30:37,384 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 02:30:37,499 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 02:30:37,501 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-sched-69cb1c0a8c06c2799bf55310/log.txt.gz 2026-03-31 02:30:37,727 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 02:30:37,952 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 02:30:37,954 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-sched-69cb1c0a8c06c2799bf55310/lava-logs.yaml 2026-03-31 02:30:37,963 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 02:30:38,125 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 02:30:38,127 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-sched-69cb1c0a8c06c2799bf55310/lava_callback.json.gz 2026-03-31 02:30:38,153 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:30:38,172 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1c0a8c06c2799bf55310 HTTP/1.1" 200 None 2026-03-31 02:30:38,174 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:30:38,188 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12cb8c06c2799bf55018 HTTP/1.1" 200 None 2026-03-31 02:30:38,190 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:30:38,214 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb1c0a8c06c2799bf55310 HTTP/1.1" 200 None 2026-03-31 02:30:38,219 - lava_callback - INFO - Completed processing callback for node 69cb1c0a8c06c2799bf55310 INFO: 172.20.0.1:56104 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56106 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53682 - "GET / HTTP/1.0" 200 OK 2026-03-31 02:30:59,770 - lava_callback - INFO - Received callback for node 69cb2d568c06c2799bf57ea6 INFO: 172.20.0.1:41380 - "POST /node/69cb2d568c06c2799bf57ea6 HTTP/1.0" 202 Accepted 2026-03-31 02:30:59,820 - lava_callback - INFO - Starting async job submit for node 69cb2d568c06c2799bf57ea6 2026-03-31 02:30:59,858 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:30:59,885 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb2d568c06c2799bf57ea6 HTTP/1.1" 200 None 2026-03-31 02:30:59,886 - lava_callback - DEBUG - Processing node 69cb2d568c06c2799bf57ea6: baseline-arm-pengutronix 2026-03-31 02:31:00,037 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 02:31:00,160 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 02:31:00,162 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cb2d568c06c2799bf57ea6/log.txt.gz 2026-03-31 02:31:00,222 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 02:31:00,405 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 02:31:00,407 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cb2d568c06c2799bf57ea6/lava-logs.yaml 2026-03-31 02:31:00,410 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 02:31:00,539 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 02:31:00,540 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cb2d568c06c2799bf57ea6/lava_callback.json.gz DEBUG: Setup failed for 69cb2d568c06c2799bf57ea6. Transitting job node result: incomplete -> fail 2026-03-31 02:31:00,589 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:31:00,604 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb2d568c06c2799bf57ea6 HTTP/1.1" 200 None 2026-03-31 02:31:00,605 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:31:00,618 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12cb8c06c2799bf55018 HTTP/1.1" 200 None 2026-03-31 02:31:00,620 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:31:00,643 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb2d568c06c2799bf57ea6 HTTP/1.1" 200 None 2026-03-31 02:31:00,698 - lava_callback - INFO - Completed processing callback for node 69cb2d568c06c2799bf57ea6 2026-03-31 02:31:01,536 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:31:01,557 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:41382 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43094 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-31 02:31:15,901 - lava_callback - INFO - Received callback for node 69cb1c0c8c06c2799bf55313 INFO: 172.20.0.1:43104 - "POST /node/69cb1c0c8c06c2799bf55313 HTTP/1.0" 202 Accepted 2026-03-31 02:31:15,911 - lava_callback - INFO - Starting async job submit for node 69cb1c0c8c06c2799bf55313 2026-03-31 02:31:15,952 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:31:15,967 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1c0c8c06c2799bf55313 HTTP/1.1" 200 None 2026-03-31 02:31:15,968 - lava_callback - DEBUG - Processing node 69cb1c0c8c06c2799bf55313: ltp-smoke 2026-03-31 02:31:16,474 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:43114 - "GET / HTTP/1.0" 200 OK 2026-03-31 02:31:16,626 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 02:31:16,628 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoke-69cb1c0c8c06c2799bf55313/log.txt.gz 2026-03-31 02:31:16,894 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 02:31:17,091 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 02:31:17,093 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-smoke-69cb1c0c8c06c2799bf55313/lava-logs.yaml 2026-03-31 02:31:17,101 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 02:31:17,279 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 02:31:17,281 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoke-69cb1c0c8c06c2799bf55313/lava_callback.json.gz 2026-03-31 02:31:17,309 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:31:17,324 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1c0c8c06c2799bf55313 HTTP/1.1" 200 None 2026-03-31 02:31:17,325 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:31:17,339 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12cb8c06c2799bf55018 HTTP/1.1" 200 None 2026-03-31 02:31:17,341 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:31:17,366 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb1c0c8c06c2799bf55313 HTTP/1.1" 200 None 2026-03-31 02:31:17,368 - lava_callback - INFO - Completed processing callback for node 69cb1c0c8c06c2799bf55313 INFO: 172.20.0.1:47294 - "GET / HTTP/1.0" 200 OK 2026-03-31 02:31:31,560 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:31:31,586 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:32886 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49608 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49618 - "GET / HTTP/1.0" 200 OK 2026-03-31 02:31:51,019 - lava_callback - INFO - Received callback for node 69c88f0af41e4fb6a3785fe7 INFO: 172.20.0.1:60668 - "POST /node/69c88f0af41e4fb6a3785fe7 HTTP/1.0" 202 Accepted 2026-03-31 02:31:51,025 - lava_callback - INFO - Starting async job submit for node 69c88f0af41e4fb6a3785fe7 2026-03-31 02:31:51,025 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:31:51,046 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c88f0af41e4fb6a3785fe7 HTTP/1.1" 200 None 2026-03-31 02:31:51,047 - lava_callback - DEBUG - Processing node 69c88f0af41e4fb6a3785fe7: baseline-arm64-clabbe 2026-03-31 02:31:51,047 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c88f0af41e4fb6a3785fe7 2026-03-31 02:31:51,048 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:31:51,063 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c88f0af41e4fb6a3785fe7 HTTP/1.1" 200 None 2026-03-31 02:31:51,065 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:31:51,085 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086b1ce0c9bc43bccb63a HTTP/1.1" 200 None 2026-03-31 02:31:51,088 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:31:51,120 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c88f0af41e4fb6a3785fe7 HTTP/1.1" 200 None 2026-03-31 02:31:51,122 - lava_callback - ERROR - Error processing callback for node 69c88f0af41e4fb6a3785fe7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:60676 - "GET / HTTP/1.0" 200 OK 2026-03-31 02:32:05,457 - lava_callback - INFO - Received callback for node 69cb2d728c06c2799bf57ead INFO: 172.20.0.1:47234 - "POST /node/69cb2d728c06c2799bf57ead HTTP/1.0" 202 Accepted 2026-03-31 02:32:05,468 - lava_callback - INFO - Starting async job submit for node 69cb2d728c06c2799bf57ead 2026-03-31 02:32:05,530 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:32:05,570 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb2d728c06c2799bf57ead HTTP/1.1" 200 None 2026-03-31 02:32:05,571 - lava_callback - DEBUG - Processing node 69cb2d728c06c2799bf57ead: baseline-arm 2026-03-31 02:32:06,337 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 02:32:06,481 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 02:32:06,483 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69cb2d728c06c2799bf57ead/log.txt.gz INFO: 172.20.0.1:47246 - "GET / HTTP/1.0" 200 OK 2026-03-31 02:32:06,855 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 02:32:07,085 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 02:32:07,087 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69cb2d728c06c2799bf57ead/lava-logs.yaml 2026-03-31 02:32:07,103 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 02:32:07,261 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 02:32:07,263 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69cb2d728c06c2799bf57ead/lava_callback.json.gz DEBUG: Setup failed for 69cb2d728c06c2799bf57ead. Transitting job node result: incomplete -> fail 2026-03-31 02:32:07,369 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:32:07,408 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb2d728c06c2799bf57ead HTTP/1.1" 200 None 2026-03-31 02:32:07,410 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:32:07,424 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12cb8c06c2799bf55018 HTTP/1.1" 200 None 2026-03-31 02:32:07,425 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:32:07,450 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb2d728c06c2799bf57ead HTTP/1.1" 200 None 2026-03-31 02:32:07,560 - lava_callback - INFO - Completed processing callback for node 69cb2d728c06c2799bf57ead INFO: 172.20.0.1:43748 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43754 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42370 - "GET / HTTP/1.0" 200 OK 2026-03-31 02:32:31,588 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:32:31,605 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:54120 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35562 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35564 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35578 - "GET /node/69402d7a575bc060ce155c04%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:35580 - "GET /node/6940b9b61e3c24e95681b4b2%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:48610 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54550 - "GET / HTTP/1.0" 200 OK 2026-03-31 02:33:11,038 - lava_callback - INFO - Received callback for node 69c88f55f41e4fb6a3785ff3 INFO: 172.20.0.1:55590 - "POST /node/69c88f55f41e4fb6a3785ff3 HTTP/1.0" 202 Accepted 2026-03-31 02:33:11,044 - lava_callback - INFO - Starting async job submit for node 69c88f55f41e4fb6a3785ff3 2026-03-31 02:33:11,045 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:33:11,069 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c88f55f41e4fb6a3785ff3 HTTP/1.1" 200 None 2026-03-31 02:33:11,070 - lava_callback - DEBUG - Processing node 69c88f55f41e4fb6a3785ff3: baseline-arm64-clabbe 2026-03-31 02:33:11,070 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c88f55f41e4fb6a3785ff3 2026-03-31 02:33:11,071 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:33:11,086 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c88f55f41e4fb6a3785ff3 HTTP/1.1" 200 None 2026-03-31 02:33:11,088 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:33:11,103 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086b1ce0c9bc43bccb63a HTTP/1.1" 200 None 2026-03-31 02:33:11,105 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:33:11,146 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c88f55f41e4fb6a3785ff3 HTTP/1.1" 200 None 2026-03-31 02:33:11,148 - lava_callback - ERROR - Error processing callback for node 69c88f55f41e4fb6a3785ff3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:55600 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55612 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40618 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52118 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38344 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38360 - "GET / HTTP/1.0" 200 OK 2026-03-31 02:33:55,885 - lava_callback - INFO - Received callback for node 69cb1c0f8c06c2799bf5531a INFO: 172.20.0.1:38770 - "POST /node/69cb1c0f8c06c2799bf5531a HTTP/1.0" 202 Accepted 2026-03-31 02:33:55,893 - lava_callback - INFO - Starting async job submit for node 69cb1c0f8c06c2799bf5531a 2026-03-31 02:33:55,922 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:33:55,935 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1c0f8c06c2799bf5531a HTTP/1.1" 200 None 2026-03-31 02:33:55,936 - lava_callback - DEBUG - Processing node 69cb1c0f8c06c2799bf5531a: ltp-syscalls-ipc 2026-03-31 02:33:56,332 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 02:33:56,487 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 02:33:56,489 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-syscalls-ipc-69cb1c0f8c06c2799bf5531a/log.txt.gz INFO: 172.20.0.1:38774 - "GET / HTTP/1.0" 200 OK 2026-03-31 02:33:56,763 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 02:33:56,955 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 02:33:56,957 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-syscalls-ipc-69cb1c0f8c06c2799bf5531a/lava-logs.yaml 2026-03-31 02:33:56,966 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 02:33:57,113 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 02:33:57,114 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-syscalls-ipc-69cb1c0f8c06c2799bf5531a/lava_callback.json.gz 2026-03-31 02:33:57,140 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:33:57,155 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1c0f8c06c2799bf5531a HTTP/1.1" 200 None 2026-03-31 02:33:57,156 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:33:57,169 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12cb8c06c2799bf55018 HTTP/1.1" 200 None 2026-03-31 02:33:57,171 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:33:57,195 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb1c0f8c06c2799bf5531a HTTP/1.1" 200 None 2026-03-31 02:33:57,197 - lava_callback - INFO - Completed processing callback for node 69cb1c0f8c06c2799bf5531a 2026-03-31 02:33:57,459 - lava_callback - INFO - Received callback for node 69cb1c118c06c2799bf5532e INFO: 172.20.0.1:38776 - "POST /node/69cb1c118c06c2799bf5532e HTTP/1.0" 202 Accepted 2026-03-31 02:33:57,466 - lava_callback - INFO - Starting async job submit for node 69cb1c118c06c2799bf5532e 2026-03-31 02:33:57,492 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:33:57,506 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1c118c06c2799bf5532e HTTP/1.1" 200 None 2026-03-31 02:33:57,507 - lava_callback - DEBUG - Processing node 69cb1c118c06c2799bf5532e: ltp-timers 2026-03-31 02:33:57,844 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 02:33:57,957 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 02:33:57,959 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-timers-69cb1c118c06c2799bf5532e/log.txt.gz 2026-03-31 02:33:58,216 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 02:33:58,462 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 02:33:58,464 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-timers-69cb1c118c06c2799bf5532e/lava-logs.yaml 2026-03-31 02:33:58,472 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 02:33:58,633 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 02:33:58,636 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-timers-69cb1c118c06c2799bf5532e/lava_callback.json.gz 2026-03-31 02:33:58,664 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:33:58,678 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1c118c06c2799bf5532e HTTP/1.1" 200 None 2026-03-31 02:33:58,680 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:33:58,694 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12cb8c06c2799bf55018 HTTP/1.1" 200 None 2026-03-31 02:33:58,695 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:33:58,740 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb1c118c06c2799bf5532e HTTP/1.1" 200 None 2026-03-31 02:33:58,753 - lava_callback - INFO - Completed processing callback for node 69cb1c118c06c2799bf5532e 2026-03-31 02:34:01,608 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:34:01,633 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:54442 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33104 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33116 - "GET / HTTP/1.0" 200 OK 2026-03-31 02:34:16,932 - lava_callback - INFO - Received callback for node 69cb2e8c8c06c2799bf57ed9 INFO: 172.20.0.1:33132 - "POST /node/69cb2e8c8c06c2799bf57ed9 HTTP/1.0" 202 Accepted 2026-03-31 02:34:16,939 - lava_callback - INFO - Starting async job submit for node 69cb2e8c8c06c2799bf57ed9 2026-03-31 02:34:17,005 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:34:17,019 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb2e8c8c06c2799bf57ed9 HTTP/1.1" 200 None 2026-03-31 02:34:17,020 - lava_callback - DEBUG - Processing node 69cb2e8c8c06c2799bf57ed9: ltp-smoke 2026-03-31 02:34:17,623 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 02:34:17,771 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 02:34:17,773 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoke-69cb2e8c8c06c2799bf57ed9/log.txt.gz 2026-03-31 02:34:18,108 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 02:34:18,328 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 02:34:18,331 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-smoke-69cb2e8c8c06c2799bf57ed9/lava-logs.yaml 2026-03-31 02:34:18,345 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 02:34:18,530 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 02:34:18,531 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoke-69cb2e8c8c06c2799bf57ed9/lava_callback.json.gz 2026-03-31 02:34:18,667 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:34:18,682 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb2e8c8c06c2799bf57ed9 HTTP/1.1" 200 None 2026-03-31 02:34:18,683 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:34:18,699 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12d38c06c2799bf55022 HTTP/1.1" 200 None 2026-03-31 02:34:18,700 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:34:18,724 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb2e8c8c06c2799bf57ed9 HTTP/1.1" 200 None 2026-03-31 02:34:18,828 - lava_callback - INFO - Completed processing callback for node 69cb2e8c8c06c2799bf57ed9 INFO: 172.20.0.1:55732 - "GET / HTTP/1.0" 200 OK 2026-03-31 02:34:31,124 - lava_callback - INFO - Received callback for node 69c88fa1f41e4fb6a3785ffb INFO: 172.20.0.1:51374 - "POST /node/69c88fa1f41e4fb6a3785ffb HTTP/1.0" 202 Accepted 2026-03-31 02:34:31,131 - lava_callback - INFO - Starting async job submit for node 69c88fa1f41e4fb6a3785ffb 2026-03-31 02:34:31,131 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:34:31,146 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c88fa1f41e4fb6a3785ffb HTTP/1.1" 200 None 2026-03-31 02:34:31,147 - lava_callback - DEBUG - Processing node 69c88fa1f41e4fb6a3785ffb: baseline-arm64-clabbe 2026-03-31 02:34:31,148 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c88fa1f41e4fb6a3785ffb 2026-03-31 02:34:31,148 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:34:31,170 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c88fa1f41e4fb6a3785ffb HTTP/1.1" 200 None 2026-03-31 02:34:31,172 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:34:31,191 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086b1ce0c9bc43bccb63a HTTP/1.1" 200 None 2026-03-31 02:34:31,193 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:34:31,228 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c88fa1f41e4fb6a3785ffb HTTP/1.1" 200 None 2026-03-31 02:34:31,230 - lava_callback - ERROR - Error processing callback for node 69c88fa1f41e4fb6a3785ffb: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-31 02:34:31,635 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:34:31,652 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:51380 - "GET / HTTP/1.0" 200 OK 2026-03-31 02:34:36,969 - lava_callback - INFO - Received callback for node 69cb1c138c06c2799bf55331 INFO: 172.20.0.1:51382 - "POST /node/69cb1c138c06c2799bf55331 HTTP/1.0" 202 Accepted 2026-03-31 02:34:36,975 - lava_callback - INFO - Starting async job submit for node 69cb1c138c06c2799bf55331 2026-03-31 02:34:37,002 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:34:37,020 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1c138c06c2799bf55331 HTTP/1.1" 200 None 2026-03-31 02:34:37,021 - lava_callback - DEBUG - Processing node 69cb1c138c06c2799bf55331: ltp-watchqueue 2026-03-31 02:34:37,357 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 02:34:37,470 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 02:34:37,472 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-watchqueue-69cb1c138c06c2799bf55331/log.txt.gz 2026-03-31 02:34:37,679 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 02:34:37,851 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 02:34:37,853 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-watchqueue-69cb1c138c06c2799bf55331/lava-logs.yaml 2026-03-31 02:34:37,861 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 02:34:38,022 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 02:34:38,024 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-watchqueue-69cb1c138c06c2799bf55331/lava_callback.json.gz 2026-03-31 02:34:38,053 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:34:38,076 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1c138c06c2799bf55331 HTTP/1.1" 200 None 2026-03-31 02:34:38,078 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:34:38,093 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12cb8c06c2799bf55018 HTTP/1.1" 200 None 2026-03-31 02:34:38,095 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:34:38,131 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb1c138c06c2799bf55331 HTTP/1.1" 200 None 2026-03-31 02:34:38,132 - lava_callback - INFO - Completed processing callback for node 69cb1c138c06c2799bf55331 INFO: 172.20.0.1:52926 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52934 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33986 - "GET / HTTP/1.0" 200 OK 2026-03-31 02:35:01,653 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:35:01,671 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:56854 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55504 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55514 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48900 - "GET / HTTP/1.0" 200 OK 2026-03-31 02:35:31,141 - lava_callback - INFO - Received callback for node 69c88fecf41e4fb6a3786007 INFO: 172.20.0.1:54064 - "POST /node/69c88fecf41e4fb6a3786007 HTTP/1.0" 202 Accepted 2026-03-31 02:35:31,149 - lava_callback - INFO - Starting async job submit for node 69c88fecf41e4fb6a3786007 2026-03-31 02:35:31,150 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:35:31,164 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c88fecf41e4fb6a3786007 HTTP/1.1" 200 None 2026-03-31 02:35:31,165 - lava_callback - DEBUG - Processing node 69c88fecf41e4fb6a3786007: baseline-arm64-clabbe 2026-03-31 02:35:31,165 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c88fecf41e4fb6a3786007 2026-03-31 02:35:31,166 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:35:31,212 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c88fecf41e4fb6a3786007 HTTP/1.1" 200 None 2026-03-31 02:35:31,213 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:35:31,227 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086b1ce0c9bc43bccb63a HTTP/1.1" 200 None 2026-03-31 02:35:31,229 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:35:31,263 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c88fecf41e4fb6a3786007 HTTP/1.1" 200 None 2026-03-31 02:35:31,265 - lava_callback - ERROR - Error processing callback for node 69c88fecf41e4fb6a3786007: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:54066 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37954 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37966 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41732 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52356 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50010 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50014 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45046 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41014 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54252 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54254 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49688 - "GET / HTTP/1.0" 200 OK 2026-03-31 02:37:01,362 - lava_callback - INFO - Received callback for node 69cb2e958c06c2799bf57edc INFO: 172.20.0.1:54886 - "POST /node/69cb2e958c06c2799bf57edc HTTP/1.0" 202 Accepted 2026-03-31 02:37:01,369 - lava_callback - INFO - Starting async job submit for node 69cb2e958c06c2799bf57edc 2026-03-31 02:37:01,413 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:37:01,429 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb2e958c06c2799bf57edc HTTP/1.1" 200 None 2026-03-31 02:37:01,430 - lava_callback - DEBUG - Processing node 69cb2e958c06c2799bf57edc: baseline-nfs-arm64 2026-03-31 02:37:02,547 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 02:37:02,714 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 02:37:02,715 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cb2e958c06c2799bf57edc/log.txt.gz 2026-03-31 02:37:03,358 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 02:37:03,738 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 02:37:03,741 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cb2e958c06c2799bf57edc/lava-logs.yaml 2026-03-31 02:37:03,763 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 02:37:03,949 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 02:37:03,951 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cb2e958c06c2799bf57edc/lava_callback.json.gz DEBUG: Setup failed for 69cb2e958c06c2799bf57edc. Transitting job node result: incomplete -> fail 2026-03-31 02:37:04,040 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:37:04,055 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb2e958c06c2799bf57edc HTTP/1.1" 200 None 2026-03-31 02:37:04,057 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:37:04,071 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12e08c06c2799bf5502b HTTP/1.1" 200 None 2026-03-31 02:37:04,073 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:37:04,108 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb2e958c06c2799bf57edc HTTP/1.1" 200 None 2026-03-31 02:37:04,208 - lava_callback - INFO - Completed processing callback for node 69cb2e958c06c2799bf57edc INFO: 172.20.0.1:54902 - "GET / HTTP/1.0" 200 OK 2026-03-31 02:37:11,227 - lava_callback - INFO - Received callback for node 69c89042f41e4fb6a378600e INFO: 172.20.0.1:43980 - "POST /node/69c89042f41e4fb6a378600e HTTP/1.0" 202 Accepted 2026-03-31 02:37:11,233 - lava_callback - INFO - Starting async job submit for node 69c89042f41e4fb6a378600e 2026-03-31 02:37:11,234 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:37:11,249 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c89042f41e4fb6a378600e HTTP/1.1" 200 None 2026-03-31 02:37:11,250 - lava_callback - DEBUG - Processing node 69c89042f41e4fb6a378600e: baseline-arm64-clabbe 2026-03-31 02:37:11,250 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c89042f41e4fb6a378600e 2026-03-31 02:37:11,251 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:37:11,264 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c89042f41e4fb6a378600e HTTP/1.1" 200 None 2026-03-31 02:37:11,265 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:37:11,278 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086b1ce0c9bc43bccb63a HTTP/1.1" 200 None 2026-03-31 02:37:11,279 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:37:11,307 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c89042f41e4fb6a378600e HTTP/1.1" 200 None 2026-03-31 02:37:11,312 - lava_callback - ERROR - Error processing callback for node 69c89042f41e4fb6a378600e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:43984 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43990 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42462 - "GET / HTTP/1.0" 200 OK 2026-03-31 02:37:31,678 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:37:31,698 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:33274 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60402 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60406 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40878 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46090 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43152 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43162 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48498 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56818 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52134 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52136 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52266 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36874 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49520 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49532 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56610 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51316 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44540 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44550 - "GET / HTTP/1.0" 200 OK 2026-03-31 02:39:51,362 - lava_callback - INFO - Received callback for node 69c890e2f41e4fb6a378601b INFO: 172.20.0.1:42056 - "POST /node/69c890e2f41e4fb6a378601b HTTP/1.0" 202 Accepted 2026-03-31 02:39:51,368 - lava_callback - INFO - Starting async job submit for node 69c890e2f41e4fb6a378601b 2026-03-31 02:39:51,369 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:39:51,382 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c890e2f41e4fb6a378601b HTTP/1.1" 200 None 2026-03-31 02:39:51,383 - lava_callback - DEBUG - Processing node 69c890e2f41e4fb6a378601b: baseline-arm64-clabbe 2026-03-31 02:39:51,383 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c890e2f41e4fb6a378601b 2026-03-31 02:39:51,384 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:39:51,397 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c890e2f41e4fb6a378601b HTTP/1.1" 200 None 2026-03-31 02:39:51,399 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:39:51,413 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086b1ce0c9bc43bccb63a HTTP/1.1" 200 None 2026-03-31 02:39:51,414 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:39:51,443 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c890e2f41e4fb6a378601b HTTP/1.1" 200 None 2026-03-31 02:39:51,453 - lava_callback - ERROR - Error processing callback for node 69c890e2f41e4fb6a378601b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:42060 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50818 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36734 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36738 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51856 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42796 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56736 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56748 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45382 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41292 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46638 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46646 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53944 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44932 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53830 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53834 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51460 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59486 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51368 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51374 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33114 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58248 - "GET /node/694acb246c96ed51d316f67a%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:58256 - "GET /node/69411a76459ff5fc64d2a42b%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:58258 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52484 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52488 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60212 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34506 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49356 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49366 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36102 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44094 - "GET /node/6992e68131166481c8dd7460%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:44104 - "GET /node/698b684b0a16aea5a0dc76c0%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:44110 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46018 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46020 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38040 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45298 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60708 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60714 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52584 - "GET / HTTP/1.0" 200 OK 2026-03-31 02:44:31,491 - lava_callback - INFO - Received callback for node 69c891faf41e4fb6a378605f INFO: 172.20.0.1:53574 - "POST /node/69c891faf41e4fb6a378605f HTTP/1.0" 202 Accepted 2026-03-31 02:44:31,497 - lava_callback - INFO - Starting async job submit for node 69c891faf41e4fb6a378605f 2026-03-31 02:44:31,497 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:44:31,512 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c891faf41e4fb6a378605f HTTP/1.1" 200 None 2026-03-31 02:44:31,513 - lava_callback - DEBUG - Processing node 69c891faf41e4fb6a378605f: baseline-arm-clabbe 2026-03-31 02:44:31,513 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c891faf41e4fb6a378605f 2026-03-31 02:44:31,514 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:44:31,527 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c891faf41e4fb6a378605f HTTP/1.1" 200 None 2026-03-31 02:44:31,529 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:44:31,542 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eacbe430e18c5d159df HTTP/1.1" 200 None 2026-03-31 02:44:31,543 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:44:31,568 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c891faf41e4fb6a378605f HTTP/1.1" 200 None 2026-03-31 02:44:31,575 - lava_callback - ERROR - Error processing callback for node 69c891faf41e4fb6a378605f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:53590 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39334 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39346 - "GET / HTTP/1.0" 200 OK 2026-03-31 02:44:51,487 - lava_callback - INFO - Received callback for node 69c8921ff41e4fb6a378606e INFO: 172.20.0.1:47960 - "POST /node/69c8921ff41e4fb6a378606e HTTP/1.0" 202 Accepted 2026-03-31 02:44:51,493 - lava_callback - INFO - Starting async job submit for node 69c8921ff41e4fb6a378606e 2026-03-31 02:44:51,494 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:44:51,509 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8921ff41e4fb6a378606e HTTP/1.1" 200 None 2026-03-31 02:44:51,510 - lava_callback - DEBUG - Processing node 69c8921ff41e4fb6a378606e: baseline-arm-clabbe 2026-03-31 02:44:51,510 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8921ff41e4fb6a378606e 2026-03-31 02:44:51,510 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:44:51,524 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8921ff41e4fb6a378606e HTTP/1.1" 200 None 2026-03-31 02:44:51,526 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:44:51,543 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eacbe430e18c5d159df HTTP/1.1" 200 None 2026-03-31 02:44:51,545 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:44:51,578 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8921ff41e4fb6a378606e HTTP/1.1" 200 None 2026-03-31 02:44:51,580 - lava_callback - ERROR - Error processing callback for node 69c8921ff41e4fb6a378606e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:47976 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60598 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59530 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59536 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43244 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59974 - "GET /node/693f6eb50d8ce4018da92b50%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:59986 - "GET /node/69304996009d5f8c767ef049%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:59994 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46700 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46708 - "GET / HTTP/1.0" 200 OK 2026-03-31 02:45:51,555 - lava_callback - INFO - Received callback for node 69c8924ff41e4fb6a378607e INFO: 172.20.0.1:52402 - "POST /node/69c8924ff41e4fb6a378607e HTTP/1.0" 202 Accepted 2026-03-31 02:45:51,561 - lava_callback - INFO - Starting async job submit for node 69c8924ff41e4fb6a378607e 2026-03-31 02:45:51,562 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:45:51,576 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8924ff41e4fb6a378607e HTTP/1.1" 200 None 2026-03-31 02:45:51,577 - lava_callback - DEBUG - Processing node 69c8924ff41e4fb6a378607e: baseline-arm-clabbe 2026-03-31 02:45:51,577 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8924ff41e4fb6a378607e 2026-03-31 02:45:51,578 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:45:51,591 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8924ff41e4fb6a378607e HTTP/1.1" 200 None 2026-03-31 02:45:51,593 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:45:51,627 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eacbe430e18c5d159df HTTP/1.1" 200 None 2026-03-31 02:45:51,629 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:45:51,651 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8924ff41e4fb6a378607e HTTP/1.1" 200 None 2026-03-31 02:45:51,653 - lava_callback - ERROR - Error processing callback for node 69c8924ff41e4fb6a378607e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:52418 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40712 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40716 - "GET /node/698bd7731e517752bab8b4b6%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:40732 - "GET /node/69939907cebe99dd8e074d8f%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:42628 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42634 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46002 - "GET / HTTP/1.0" 200 OK 2026-03-31 02:46:31,772 - lava_callback - INFO - Received callback for node 69c89273f41e4fb6a378608c INFO: 172.20.0.1:58822 - "POST /node/69c89273f41e4fb6a378608c HTTP/1.0" 202 Accepted 2026-03-31 02:46:31,777 - lava_callback - INFO - Starting async job submit for node 69c89273f41e4fb6a378608c 2026-03-31 02:46:31,778 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:46:31,797 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c89273f41e4fb6a378608c HTTP/1.1" 200 None 2026-03-31 02:46:31,798 - lava_callback - DEBUG - Processing node 69c89273f41e4fb6a378608c: baseline-arm-clabbe 2026-03-31 02:46:31,799 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c89273f41e4fb6a378608c 2026-03-31 02:46:31,800 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:46:31,814 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c89273f41e4fb6a378608c HTTP/1.1" 200 None 2026-03-31 02:46:31,816 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:46:31,832 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eacbe430e18c5d159df HTTP/1.1" 200 None 2026-03-31 02:46:31,833 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:46:31,866 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c89273f41e4fb6a378608c HTTP/1.1" 200 None 2026-03-31 02:46:31,868 - lava_callback - ERROR - Error processing callback for node 69c89273f41e4fb6a378608c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:58824 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45134 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45146 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45152 - "GET /node/6939363e6e7b0b7f22e5211c%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:45160 - "GET /node/69bc9c9bb4744551b27fa6df%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:58970 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43534 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60212 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60222 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38458 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47626 - "GET /node/69830f3a409235c29c0de6fa%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:47630 - "GET /node/698b67480a16aea5a0dc7587%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:47646 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37102 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37112 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49730 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54172 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48362 - "GET /node/69936c23cebe99dd8e067773%5Cn HTTP/1.0" 405 Method Not Allowed 2026-03-31 02:48:11,665 - lava_callback - INFO - Received callback for node 69c892e1f41e4fb6a37860ae INFO: 172.20.0.1:48378 - "POST /node/69c892e1f41e4fb6a37860ae HTTP/1.0" 202 Accepted 2026-03-31 02:48:11,672 - lava_callback - INFO - Starting async job submit for node 69c892e1f41e4fb6a37860ae 2026-03-31 02:48:11,674 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:48:11,691 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c892e1f41e4fb6a37860ae HTTP/1.1" 200 None 2026-03-31 02:48:11,692 - lava_callback - DEBUG - Processing node 69c892e1f41e4fb6a37860ae: baseline-arm-clabbe 2026-03-31 02:48:11,692 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c892e1f41e4fb6a37860ae 2026-03-31 02:48:11,693 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:48:11,707 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c892e1f41e4fb6a37860ae HTTP/1.1" 200 None 2026-03-31 02:48:11,708 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:48:11,724 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eacbe430e18c5d159df HTTP/1.1" 200 None 2026-03-31 02:48:11,726 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:48:11,757 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c892e1f41e4fb6a37860ae HTTP/1.1" 200 None 2026-03-31 02:48:11,762 - lava_callback - ERROR - Error processing callback for node 69c892e1f41e4fb6a37860ae: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:48390 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48404 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40726 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40156 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40092 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40098 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45714 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40160 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33254 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33256 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58668 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44016 - "GET / HTTP/1.0" 200 OK 2026-03-31 02:49:40,011 - lava_callback - INFO - Received callback for node 69cb32288c06c2799bf57f66 INFO: 172.20.0.1:42022 - "POST /node/69cb32288c06c2799bf57f66 HTTP/1.0" 202 Accepted 2026-03-31 02:49:40,018 - lava_callback - INFO - Starting async job submit for node 69cb32288c06c2799bf57f66 2026-03-31 02:49:40,092 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:49:40,107 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb32288c06c2799bf57f66 HTTP/1.1" 200 None 2026-03-31 02:49:40,108 - lava_callback - DEBUG - Processing node 69cb32288c06c2799bf57f66: baseline-arm 2026-03-31 02:49:40,819 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 02:49:41,012 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 02:49:41,014 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69cb32288c06c2799bf57f66/log.txt.gz 2026-03-31 02:49:41,419 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:42024 - "GET / HTTP/1.0" 200 OK 2026-03-31 02:49:41,681 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 02:49:41,684 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69cb32288c06c2799bf57f66/lava-logs.yaml 2026-03-31 02:49:41,700 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 02:49:41,852 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 02:49:41,853 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69cb32288c06c2799bf57f66/lava_callback.json.gz DEBUG: Setup failed for 69cb32288c06c2799bf57f66. Transitting job node result: incomplete -> fail 2026-03-31 02:49:41,965 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:49:41,980 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb32288c06c2799bf57f66 HTTP/1.1" 200 None 2026-03-31 02:49:41,981 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:49:41,995 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12cb8c06c2799bf55018 HTTP/1.1" 200 None 2026-03-31 02:49:41,996 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:49:42,023 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb32288c06c2799bf57f66 HTTP/1.1" 200 None 2026-03-31 02:49:42,121 - lava_callback - INFO - Completed processing callback for node 69cb32288c06c2799bf57f66 INFO: 172.20.0.1:42040 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54562 - "GET / HTTP/1.0" 200 OK 2026-03-31 02:50:01,702 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:50:01,718 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:44598 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42764 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42780 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52992 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58490 - "GET / HTTP/1.0" 200 OK 2026-03-31 02:50:39,289 - lava_callback - INFO - Received callback for node 69cb31f28c06c2799bf57f50 INFO: 172.20.0.1:58506 - "POST /node/69cb31f28c06c2799bf57f50 HTTP/1.0" 202 Accepted 2026-03-31 02:50:39,337 - lava_callback - INFO - Starting async job submit for node 69cb31f28c06c2799bf57f50 2026-03-31 02:50:39,364 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:50:39,380 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb31f28c06c2799bf57f50 HTTP/1.1" 200 None 2026-03-31 02:50:39,381 - lava_callback - DEBUG - Processing node 69cb31f28c06c2799bf57f50: baseline-arm-pengutronix 2026-03-31 02:50:39,506 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 02:50:39,615 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 02:50:39,617 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cb31f28c06c2799bf57f50/log.txt.gz 2026-03-31 02:50:39,676 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 02:50:39,827 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 02:50:39,829 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cb31f28c06c2799bf57f50/lava-logs.yaml 2026-03-31 02:50:39,832 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 02:50:39,943 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 02:50:39,945 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cb31f28c06c2799bf57f50/lava_callback.json.gz DEBUG: Setup failed for 69cb31f28c06c2799bf57f50. Transitting job node result: incomplete -> fail 2026-03-31 02:50:39,994 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:50:40,008 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb31f28c06c2799bf57f50 HTTP/1.1" 200 None 2026-03-31 02:50:40,009 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:50:40,022 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12cb8c06c2799bf55018 HTTP/1.1" 200 None 2026-03-31 02:50:40,024 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:50:40,046 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb31f28c06c2799bf57f50 HTTP/1.1" 200 None 2026-03-31 02:50:40,091 - lava_callback - INFO - Completed processing callback for node 69cb31f28c06c2799bf57f50 INFO: 172.20.0.1:43796 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43802 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38464 - "GET / HTTP/1.0" 200 OK 2026-03-31 02:51:01,721 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:51:01,737 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:33980 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52290 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52302 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57548 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53712 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36112 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36126 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46878 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36028 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40486 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40488 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38664 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51976 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55192 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55202 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46400 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57836 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54750 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54752 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48254 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36676 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40120 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40122 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47624 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56868 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56674 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56680 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55112 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32788 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57934 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57938 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33530 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46946 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57416 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57430 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50318 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35968 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44102 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44112 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41008 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43624 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52226 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52228 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35434 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58076 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54986 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54994 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51162 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41682 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51742 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51758 - "GET / HTTP/1.0" 200 OK 2026-03-31 02:57:24,392 - lava_callback - INFO - Received callback for node 69cb33598c06c2799bf57f96 INFO: 172.20.0.1:36090 - "POST /node/69cb33598c06c2799bf57f96 HTTP/1.0" 202 Accepted 2026-03-31 02:57:24,401 - lava_callback - INFO - Starting async job submit for node 69cb33598c06c2799bf57f96 2026-03-31 02:57:24,460 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:57:24,474 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb33598c06c2799bf57f96 HTTP/1.1" 200 None 2026-03-31 02:57:24,475 - lava_callback - DEBUG - Processing node 69cb33598c06c2799bf57f96: baseline-nfs-arm64 2026-03-31 02:57:25,573 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 02:57:25,743 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 02:57:25,745 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cb33598c06c2799bf57f96/log.txt.gz 2026-03-31 02:57:26,438 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:36098 - "GET / HTTP/1.0" 200 OK 2026-03-31 02:57:26,708 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 02:57:26,711 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cb33598c06c2799bf57f96/lava-logs.yaml 2026-03-31 02:57:26,734 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 02:57:27,181 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 02:57:27,183 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cb33598c06c2799bf57f96/lava_callback.json.gz DEBUG: Setup failed for 69cb33598c06c2799bf57f96. Transitting job node result: incomplete -> fail 2026-03-31 02:57:27,271 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:57:27,290 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb33598c06c2799bf57f96 HTTP/1.1" 200 None 2026-03-31 02:57:27,292 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:57:27,308 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12e08c06c2799bf5502b HTTP/1.1" 200 None 2026-03-31 02:57:27,309 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:57:27,333 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb33598c06c2799bf57f96 HTTP/1.1" 200 None 2026-03-31 02:57:27,418 - lava_callback - INFO - Completed processing callback for node 69cb33598c06c2799bf57f96 2026-03-31 02:57:31,740 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 02:57:31,757 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:52226 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56506 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56514 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52120 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52574 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56032 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56042 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46316 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34768 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51498 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51500 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49610 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52116 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42562 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42576 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60522 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56848 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45030 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45036 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45628 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51364 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35674 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35678 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38558 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42526 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32954 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32956 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32962 - "GET /node/69af1b6dce46d8ec186acd91%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:44376 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39136 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59918 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59920 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50782 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46846 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40312 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40324 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49014 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49124 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56682 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56690 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54662 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42888 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35588 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35596 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58166 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47622 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44458 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44464 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55044 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58408 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50314 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50320 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36384 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58352 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52356 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52366 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49430 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58464 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38426 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38432 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34150 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56692 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57184 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57194 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33606 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39916 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53836 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53838 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56078 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58078 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34374 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34384 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45894 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59434 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57098 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57114 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55236 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33472 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59624 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59630 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42186 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37914 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45706 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45714 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59052 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46804 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55012 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55024 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48388 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57126 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36640 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36648 - "GET / HTTP/1.0" 200 OK 2026-03-31 03:08:48,556 - lava_callback - INFO - Received callback for node 69c897bef41e4fb6a3786125 INFO: 172.20.0.1:36656 - "POST /node/69c897bef41e4fb6a3786125 HTTP/1.0" 202 Accepted 2026-03-31 03:08:48,561 - lava_callback - INFO - Starting async job submit for node 69c897bef41e4fb6a3786125 2026-03-31 03:08:48,562 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 03:08:48,580 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c897bef41e4fb6a3786125 HTTP/1.1" 200 None 2026-03-31 03:08:48,581 - lava_callback - DEBUG - Processing node 69c897bef41e4fb6a3786125: baseline-arm64-baylibre 2026-03-31 03:08:48,581 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c897bef41e4fb6a3786125 2026-03-31 03:08:48,582 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 03:08:48,776 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c897bef41e4fb6a3786125 HTTP/1.1" 200 None 2026-03-31 03:08:48,777 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 03:08:48,790 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eb9be430e18c5d159ee HTTP/1.1" 200 None 2026-03-31 03:08:48,792 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 03:08:48,813 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c897bef41e4fb6a3786125 HTTP/1.1" 200 None 2026-03-31 03:08:48,818 - lava_callback - ERROR - Error processing callback for node 69c897bef41e4fb6a3786125: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:38562 - "GET / HTTP/1.0" 200 OK 2026-03-31 03:08:58,094 - lava_callback - INFO - Received callback for node 69cb1f588c06c2799bf57aaa INFO: 172.20.0.1:38574 - "POST /node/69cb1f588c06c2799bf57aaa HTTP/1.0" 202 Accepted 2026-03-31 03:08:58,100 - lava_callback - INFO - Starting async job submit for node 69cb1f588c06c2799bf57aaa 2026-03-31 03:08:58,123 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 03:08:58,138 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1f588c06c2799bf57aaa HTTP/1.1" 200 None 2026-03-31 03:08:58,138 - lava_callback - DEBUG - Processing node 69cb1f588c06c2799bf57aaa: baseline-arm64-broonie 2026-03-31 03:08:58,357 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 03:08:58,503 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 03:08:58,505 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cb1f588c06c2799bf57aaa/log.txt.gz 2026-03-31 03:08:58,639 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 03:08:58,870 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 03:08:58,872 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cb1f588c06c2799bf57aaa/lava-logs.yaml 2026-03-31 03:08:58,878 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 03:08:59,027 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 03:08:59,028 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cb1f588c06c2799bf57aaa/lava_callback.json.gz 2026-03-31 03:08:59,052 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 03:08:59,067 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1f588c06c2799bf57aaa HTTP/1.1" 200 None 2026-03-31 03:08:59,069 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 03:08:59,083 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12e08c06c2799bf5502b HTTP/1.1" 200 None 2026-03-31 03:08:59,084 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 03:08:59,109 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb1f588c06c2799bf57aaa HTTP/1.1" 200 None 2026-03-31 03:08:59,110 - lava_callback - INFO - Completed processing callback for node 69cb1f588c06c2799bf57aaa 2026-03-31 03:09:01,761 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 03:09:01,777 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:44646 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45070 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45074 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49478 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33310 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35796 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35802 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46548 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38446 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60200 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60202 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45956 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52398 - "GET / HTTP/1.0" 200 OK 2026-03-31 03:10:38,222 - lava_callback - INFO - Received callback for node 69cb1f618c06c2799bf57ae0 INFO: 172.20.0.1:52404 - "POST /node/69cb1f618c06c2799bf57ae0 HTTP/1.0" 202 Accepted 2026-03-31 03:10:38,229 - lava_callback - INFO - Starting async job submit for node 69cb1f618c06c2799bf57ae0 2026-03-31 03:10:38,252 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 03:10:38,267 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1f618c06c2799bf57ae0 HTTP/1.1" 200 None 2026-03-31 03:10:38,269 - lava_callback - DEBUG - Processing node 69cb1f618c06c2799bf57ae0: baseline-arm64-broonie 2026-03-31 03:10:38,560 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 03:10:38,696 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 03:10:38,698 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cb1f618c06c2799bf57ae0/log.txt.gz 2026-03-31 03:10:38,861 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 03:10:39,050 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 03:10:39,052 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cb1f618c06c2799bf57ae0/lava-logs.yaml 2026-03-31 03:10:39,059 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 03:10:39,224 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 03:10:39,226 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cb1f618c06c2799bf57ae0/lava_callback.json.gz 2026-03-31 03:10:39,268 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 03:10:39,290 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1f618c06c2799bf57ae0 HTTP/1.1" 200 None 2026-03-31 03:10:39,292 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 03:10:39,306 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12e08c06c2799bf5502b HTTP/1.1" 200 None 2026-03-31 03:10:39,307 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 03:10:39,331 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb1f618c06c2799bf57ae0 HTTP/1.1" 200 None 2026-03-31 03:10:39,334 - lava_callback - INFO - Completed processing callback for node 69cb1f618c06c2799bf57ae0 INFO: 172.20.0.1:35424 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35432 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51100 - "GET / HTTP/1.0" 200 OK 2026-03-31 03:11:01,780 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 03:11:01,796 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:35434 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45492 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45494 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54636 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54936 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56612 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56614 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59598 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43900 - "GET / HTTP/1.0" 200 OK 2026-03-31 03:12:08,553 - lava_callback - INFO - Received callback for node 69c89873f41e4fb6a3786130 INFO: 172.20.0.1:43914 - "POST /node/69c89873f41e4fb6a3786130 HTTP/1.0" 202 Accepted 2026-03-31 03:12:08,558 - lava_callback - INFO - Starting async job submit for node 69c89873f41e4fb6a3786130 2026-03-31 03:12:08,559 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 03:12:08,577 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c89873f41e4fb6a3786130 HTTP/1.1" 200 None 2026-03-31 03:12:08,577 - lava_callback - DEBUG - Processing node 69c89873f41e4fb6a3786130: baseline-arm64-baylibre 2026-03-31 03:12:08,578 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c89873f41e4fb6a3786130 2026-03-31 03:12:08,578 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 03:12:08,591 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c89873f41e4fb6a3786130 HTTP/1.1" 200 None 2026-03-31 03:12:08,593 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 03:12:08,606 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eb9be430e18c5d159ee HTTP/1.1" 200 None 2026-03-31 03:12:08,607 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 03:12:08,628 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c89873f41e4fb6a3786130 HTTP/1.1" 200 None 2026-03-31 03:12:08,635 - lava_callback - ERROR - Error processing callback for node 69c89873f41e4fb6a3786130: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:51640 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51654 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51828 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49176 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35678 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35688 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38992 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49906 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38090 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38100 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40472 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37250 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49556 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49560 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51980 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43310 - "GET / HTTP/1.0" 200 OK 2026-03-31 03:14:13,048 - lava_callback - INFO - Received callback for node 69c898f0f41e4fb6a3786140 INFO: 172.20.0.1:49636 - "POST /node/69c898f0f41e4fb6a3786140 HTTP/1.0" 202 Accepted 2026-03-31 03:14:13,054 - lava_callback - INFO - Starting async job submit for node 69c898f0f41e4fb6a3786140 2026-03-31 03:14:13,055 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 03:14:13,070 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c898f0f41e4fb6a3786140 HTTP/1.1" 200 None 2026-03-31 03:14:13,071 - lava_callback - DEBUG - Processing node 69c898f0f41e4fb6a3786140: baseline-arm64-clabbe 2026-03-31 03:14:13,071 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c898f0f41e4fb6a3786140 2026-03-31 03:14:13,072 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 03:14:13,086 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c898f0f41e4fb6a3786140 HTTP/1.1" 200 None 2026-03-31 03:14:13,088 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 03:14:13,103 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eb9be430e18c5d159ee HTTP/1.1" 200 None 2026-03-31 03:14:13,105 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 03:14:13,125 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c898f0f41e4fb6a3786140 HTTP/1.1" 200 None 2026-03-31 03:14:13,126 - lava_callback - ERROR - Error processing callback for node 69c898f0f41e4fb6a3786140: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:49640 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49654 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56136 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53780 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42880 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42894 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52746 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53582 - "GET / HTTP/1.0" 200 OK 2026-03-31 03:15:12,941 - lava_callback - INFO - Received callback for node 69c8993cf41e4fb6a3786149 INFO: 172.20.0.1:49856 - "POST /node/69c8993cf41e4fb6a3786149 HTTP/1.0" 202 Accepted 2026-03-31 03:15:12,948 - lava_callback - INFO - Starting async job submit for node 69c8993cf41e4fb6a3786149 2026-03-31 03:15:12,949 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 03:15:12,966 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8993cf41e4fb6a3786149 HTTP/1.1" 200 None 2026-03-31 03:15:12,967 - lava_callback - DEBUG - Processing node 69c8993cf41e4fb6a3786149: baseline-arm64-clabbe 2026-03-31 03:15:12,968 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8993cf41e4fb6a3786149 2026-03-31 03:15:12,968 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 03:15:12,984 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8993cf41e4fb6a3786149 HTTP/1.1" 200 None 2026-03-31 03:15:12,985 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 03:15:13,005 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eb9be430e18c5d159ee HTTP/1.1" 200 None 2026-03-31 03:15:13,007 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 03:15:13,029 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8993cf41e4fb6a3786149 HTTP/1.1" 200 None 2026-03-31 03:15:13,030 - lava_callback - ERROR - Error processing callback for node 69c8993cf41e4fb6a3786149: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:49872 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49888 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58648 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54360 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60304 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60318 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35712 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41864 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40660 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40676 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39906 - "GET / HTTP/1.0" 200 OK 2026-03-31 03:16:33,019 - lava_callback - INFO - Received callback for node 69c8998bf41e4fb6a3786159 INFO: 172.20.0.1:40510 - "POST /node/69c8998bf41e4fb6a3786159 HTTP/1.0" 202 Accepted 2026-03-31 03:16:33,025 - lava_callback - INFO - Starting async job submit for node 69c8998bf41e4fb6a3786159 2026-03-31 03:16:33,026 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 03:16:33,044 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8998bf41e4fb6a3786159 HTTP/1.1" 200 None 2026-03-31 03:16:33,045 - lava_callback - DEBUG - Processing node 69c8998bf41e4fb6a3786159: baseline-arm64-clabbe 2026-03-31 03:16:33,045 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8998bf41e4fb6a3786159 2026-03-31 03:16:33,046 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 03:16:33,059 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8998bf41e4fb6a3786159 HTTP/1.1" 200 None 2026-03-31 03:16:33,061 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 03:16:33,075 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eb9be430e18c5d159ee HTTP/1.1" 200 None 2026-03-31 03:16:33,076 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 03:16:33,097 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8998bf41e4fb6a3786159 HTTP/1.1" 200 None 2026-03-31 03:16:33,102 - lava_callback - ERROR - Error processing callback for node 69c8998bf41e4fb6a3786159: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:40514 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58152 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58160 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58988 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50212 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35216 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35222 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49288 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32814 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57610 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57620 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58124 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41784 - "GET / HTTP/1.0" 200 OK 2026-03-31 03:18:14,600 - lava_callback - INFO - Received callback for node 69c899e1f41e4fb6a3786165 INFO: 172.20.0.1:57458 - "POST /node/69c899e1f41e4fb6a3786165 HTTP/1.0" 202 Accepted 2026-03-31 03:18:14,606 - lava_callback - INFO - Starting async job submit for node 69c899e1f41e4fb6a3786165 2026-03-31 03:18:14,607 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 03:18:14,621 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c899e1f41e4fb6a3786165 HTTP/1.1" 200 None 2026-03-31 03:18:14,622 - lava_callback - DEBUG - Processing node 69c899e1f41e4fb6a3786165: baseline-arm64-clabbe 2026-03-31 03:18:14,622 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c899e1f41e4fb6a3786165 2026-03-31 03:18:14,623 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 03:18:14,637 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c899e1f41e4fb6a3786165 HTTP/1.1" 200 None 2026-03-31 03:18:14,638 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 03:18:14,652 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eb9be430e18c5d159ee HTTP/1.1" 200 None 2026-03-31 03:18:14,654 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 03:18:14,675 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c899e1f41e4fb6a3786165 HTTP/1.1" 200 None 2026-03-31 03:18:14,677 - lava_callback - ERROR - Error processing callback for node 69c899e1f41e4fb6a3786165: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:57470 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57486 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43342 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43564 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51008 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51022 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45182 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46160 - "GET / HTTP/1.0" 200 OK 2026-03-31 03:19:13,040 - lava_callback - INFO - Received callback for node 69c89a2cf41e4fb6a3786170 INFO: 172.20.0.1:49578 - "POST /node/69c89a2cf41e4fb6a3786170 HTTP/1.0" 202 Accepted 2026-03-31 03:19:13,045 - lava_callback - INFO - Starting async job submit for node 69c89a2cf41e4fb6a3786170 2026-03-31 03:19:13,047 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 03:19:13,066 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c89a2cf41e4fb6a3786170 HTTP/1.1" 200 None 2026-03-31 03:19:13,067 - lava_callback - DEBUG - Processing node 69c89a2cf41e4fb6a3786170: baseline-arm64-clabbe 2026-03-31 03:19:13,067 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c89a2cf41e4fb6a3786170 2026-03-31 03:19:13,068 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 03:19:13,083 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c89a2cf41e4fb6a3786170 HTTP/1.1" 200 None 2026-03-31 03:19:13,084 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 03:19:13,098 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eb9be430e18c5d159ee HTTP/1.1" 200 None 2026-03-31 03:19:13,100 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 03:19:13,120 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c89a2cf41e4fb6a3786170 HTTP/1.1" 200 None 2026-03-31 03:19:13,126 - lava_callback - ERROR - Error processing callback for node 69c89a2cf41e4fb6a3786170: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:49582 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49588 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54308 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53100 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34518 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34530 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37654 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53718 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59682 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59694 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49992 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53846 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37858 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37870 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47736 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50004 - "GET / HTTP/1.0" 200 OK 2026-03-31 03:21:13,157 - lava_callback - INFO - Received callback for node 69c89a95f41e4fb6a3786181 INFO: 172.20.0.1:38748 - "POST /node/69c89a95f41e4fb6a3786181 HTTP/1.0" 202 Accepted 2026-03-31 03:21:13,162 - lava_callback - INFO - Starting async job submit for node 69c89a95f41e4fb6a3786181 2026-03-31 03:21:13,163 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 03:21:13,177 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c89a95f41e4fb6a3786181 HTTP/1.1" 200 None 2026-03-31 03:21:13,178 - lava_callback - DEBUG - Processing node 69c89a95f41e4fb6a3786181: baseline-arm64-clabbe 2026-03-31 03:21:13,178 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c89a95f41e4fb6a3786181 2026-03-31 03:21:13,179 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 03:21:13,197 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c89a95f41e4fb6a3786181 HTTP/1.1" 200 None 2026-03-31 03:21:13,199 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 03:21:13,215 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eb9be430e18c5d159ee HTTP/1.1" 200 None 2026-03-31 03:21:13,216 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 03:21:13,278 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c89a95f41e4fb6a3786181 HTTP/1.1" 200 None 2026-03-31 03:21:13,285 - lava_callback - ERROR - Error processing callback for node 69c89a95f41e4fb6a3786181: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:38750 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38762 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33026 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39528 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41156 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41172 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47220 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55398 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38148 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38160 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41326 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41802 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39376 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39380 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37170 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45438 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41852 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41864 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45638 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59638 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41834 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41846 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54130 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45238 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53880 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53896 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41886 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56700 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58228 - "GET / HTTP/1.0" 200 OK 2026-03-31 03:24:45,333 - lava_callback - INFO - Received callback for node 69cb1f368c06c2799bf57a45 INFO: 172.20.0.1:58242 - "POST /node/69cb1f368c06c2799bf57a45 HTTP/1.0" 202 Accepted 2026-03-31 03:24:45,340 - lava_callback - INFO - Starting async job submit for node 69cb1f368c06c2799bf57a45 2026-03-31 03:24:45,377 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 03:24:45,418 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1f368c06c2799bf57a45 HTTP/1.1" 200 None 2026-03-31 03:24:45,419 - lava_callback - DEBUG - Processing node 69cb1f368c06c2799bf57a45: baseline-arm64 2026-03-31 03:24:46,002 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 03:24:46,149 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 03:24:46,150 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb1f368c06c2799bf57a45/log.txt.gz 2026-03-31 03:24:46,417 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:58256 - "GET / HTTP/1.0" 200 OK 2026-03-31 03:24:46,628 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 03:24:46,630 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb1f368c06c2799bf57a45/lava-logs.yaml 2026-03-31 03:24:46,641 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 03:24:46,795 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 03:24:46,797 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb1f368c06c2799bf57a45/lava_callback.json.gz 2026-03-31 03:24:46,822 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 03:24:46,837 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1f368c06c2799bf57a45 HTTP/1.1" 200 None 2026-03-31 03:24:46,838 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 03:24:46,856 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12e08c06c2799bf5502b HTTP/1.1" 200 None 2026-03-31 03:24:46,857 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 03:24:46,879 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb1f368c06c2799bf57a45 HTTP/1.1" 200 None 2026-03-31 03:24:46,941 - lava_callback - INFO - Completed processing callback for node 69cb1f368c06c2799bf57a45 INFO: 172.20.0.1:52116 - "GET / HTTP/1.0" 200 OK 2026-03-31 03:25:01,800 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 03:25:01,816 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:34540 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49014 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49030 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40148 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44468 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51122 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51124 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45858 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46692 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34634 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34640 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54864 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37700 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42992 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42994 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47410 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56014 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57336 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57342 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59834 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38032 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35370 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35382 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45970 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60374 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48754 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48760 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42830 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49464 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36850 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36856 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36090 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54144 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42274 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42280 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38992 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53580 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43510 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43514 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52944 - "GET / HTTP/1.0" 200 OK 2026-03-31 03:30:01,686 - lava_callback - INFO - Received callback for node 69cb36ec8c06c2799bf5800b INFO: 172.20.0.1:45256 - "POST /node/69cb36ec8c06c2799bf5800b HTTP/1.0" 202 Accepted 2026-03-31 03:30:01,733 - lava_callback - INFO - Starting async job submit for node 69cb36ec8c06c2799bf5800b 2026-03-31 03:30:01,757 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 03:30:01,775 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb36ec8c06c2799bf5800b HTTP/1.1" 200 None 2026-03-31 03:30:01,776 - lava_callback - DEBUG - Processing node 69cb36ec8c06c2799bf5800b: baseline-arm-pengutronix 2026-03-31 03:30:01,898 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 03:30:02,096 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 03:30:02,098 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cb36ec8c06c2799bf5800b/log.txt.gz 2026-03-31 03:30:02,156 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 03:30:02,495 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 03:30:02,497 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cb36ec8c06c2799bf5800b/lava-logs.yaml 2026-03-31 03:30:02,500 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 03:30:02,640 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 03:30:02,642 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cb36ec8c06c2799bf5800b/lava_callback.json.gz DEBUG: Setup failed for 69cb36ec8c06c2799bf5800b. Transitting job node result: incomplete -> fail 2026-03-31 03:30:02,690 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 03:30:02,704 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb36ec8c06c2799bf5800b HTTP/1.1" 200 None 2026-03-31 03:30:02,706 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 03:30:02,744 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12cb8c06c2799bf55018 HTTP/1.1" 200 None 2026-03-31 03:30:02,746 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 03:30:02,826 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb36ec8c06c2799bf5800b HTTP/1.1" 200 None 2026-03-31 03:30:02,887 - lava_callback - INFO - Completed processing callback for node 69cb36ec8c06c2799bf5800b INFO: 172.20.0.1:45266 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39944 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39960 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42256 - "GET / HTTP/1.0" 200 OK 2026-03-31 03:30:31,841 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 03:30:31,867 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:58582 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45052 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45064 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34516 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40598 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56650 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56652 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34888 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47934 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35602 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35612 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37576 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38120 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46820 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46826 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60664 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34364 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38616 - "GET / HTTP/1.0" 200 OK 2026-03-31 03:32:45,914 - lava_callback - INFO - Received callback for node 69cb1f478c06c2799bf57a88 INFO: 172.20.0.1:38618 - "POST /node/69cb1f478c06c2799bf57a88 HTTP/1.0" 202 Accepted 2026-03-31 03:32:45,920 - lava_callback - INFO - Starting async job submit for node 69cb1f478c06c2799bf57a88 2026-03-31 03:32:45,945 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 03:32:45,968 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1f478c06c2799bf57a88 HTTP/1.1" 200 None 2026-03-31 03:32:45,968 - lava_callback - DEBUG - Processing node 69cb1f478c06c2799bf57a88: baseline-nfs-arm64 2026-03-31 03:32:46,347 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 03:32:46,505 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 03:32:46,506 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cb1f478c06c2799bf57a88/log.txt.gz INFO: 172.20.0.1:38622 - "GET / HTTP/1.0" 200 OK 2026-03-31 03:32:46,698 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 03:32:46,920 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 03:32:46,922 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cb1f478c06c2799bf57a88/lava-logs.yaml 2026-03-31 03:32:46,931 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 03:32:47,071 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 03:32:47,073 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cb1f478c06c2799bf57a88/lava_callback.json.gz 2026-03-31 03:32:47,099 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 03:32:47,113 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1f478c06c2799bf57a88 HTTP/1.1" 200 None 2026-03-31 03:32:47,114 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 03:32:47,127 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12e08c06c2799bf5502b HTTP/1.1" 200 None 2026-03-31 03:32:47,129 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 03:32:47,154 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb1f478c06c2799bf57a88 HTTP/1.1" 200 None 2026-03-31 03:32:47,224 - lava_callback - INFO - Completed processing callback for node 69cb1f478c06c2799bf57a88 INFO: 172.20.0.1:51802 - "GET / HTTP/1.0" 200 OK 2026-03-31 03:33:01,870 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 03:33:01,886 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:39140 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46102 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46110 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46880 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59372 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57052 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57064 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33154 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49194 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36114 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36128 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52690 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54464 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41032 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41046 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59030 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57848 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36824 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36830 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52226 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60262 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37632 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37640 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46966 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50586 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49068 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49078 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51310 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52352 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48706 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48722 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39496 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46480 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35010 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35026 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38324 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49096 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57152 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57160 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34948 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50360 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52428 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52440 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45362 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45072 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55646 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55662 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52740 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52092 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45764 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45776 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40304 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42720 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35086 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35096 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41570 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40162 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49634 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49646 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43814 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57062 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48130 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48134 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41942 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52330 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36938 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36952 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59340 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45360 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33800 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33816 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39924 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56936 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52340 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52356 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40866 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48988 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58392 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58394 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44980 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47338 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49316 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49320 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34958 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39450 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46332 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46340 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35128 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53848 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36830 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36844 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38438 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45716 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51090 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51102 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50652 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46044 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45464 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-31 03:45:16,145 - lava_callback - INFO - Received callback for node 69cb3e848c06c2799bf58115 INFO: 172.20.0.1:45480 - "POST /node/69cb3e848c06c2799bf58115 HTTP/1.0" 202 Accepted 2026-03-31 03:45:16,152 - lava_callback - INFO - Starting async job submit for node 69cb3e848c06c2799bf58115 2026-03-31 03:45:16,176 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 03:45:16,201 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb3e848c06c2799bf58115 HTTP/1.1" 200 None 2026-03-31 03:45:16,202 - lava_callback - DEBUG - Processing node 69cb3e848c06c2799bf58115: baseline-arm64 INFO: 172.20.0.1:45494 - "GET / HTTP/1.0" 200 OK 2026-03-31 03:45:16,854 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 03:45:17,018 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 03:45:17,020 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb3e848c06c2799bf58115/log.txt.gz 2026-03-31 03:45:17,346 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 03:45:17,550 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 03:45:17,552 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb3e848c06c2799bf58115/lava-logs.yaml 2026-03-31 03:45:17,564 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 03:45:17,720 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 03:45:17,722 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb3e848c06c2799bf58115/lava_callback.json.gz 2026-03-31 03:45:17,747 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 03:45:17,761 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb3e848c06c2799bf58115 HTTP/1.1" 200 None 2026-03-31 03:45:17,763 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 03:45:17,789 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12e08c06c2799bf5502b HTTP/1.1" 200 None 2026-03-31 03:45:17,791 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 03:45:17,814 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb3e848c06c2799bf58115 HTTP/1.1" 200 None 2026-03-31 03:45:17,872 - lava_callback - INFO - Completed processing callback for node 69cb3e848c06c2799bf58115 INFO: 172.20.0.1:50686 - "GET / HTTP/1.0" 200 OK 2026-03-31 03:45:31,890 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 03:45:31,906 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:60588 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47854 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47870 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51710 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48450 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34784 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34794 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46042 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53752 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53030 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53034 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37118 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51152 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50976 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50984 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56922 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52056 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38462 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38476 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60032 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46276 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54118 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54124 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37546 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56398 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43902 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43910 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56034 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41186 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38124 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38138 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49022 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36456 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60484 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60490 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59236 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45354 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50494 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50502 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34114 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36456 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48132 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48136 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40164 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45774 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60796 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60810 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54848 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52608 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47664 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47668 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41680 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45168 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48712 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48728 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58252 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42184 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47192 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47202 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45710 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45568 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50840 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50848 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45634 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43410 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41246 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41262 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33850 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49970 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43378 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43394 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52576 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52872 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45724 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45736 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44818 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56802 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54546 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54554 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51586 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46884 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52056 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52060 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60406 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38362 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59656 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59672 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37120 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49870 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44278 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44290 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55376 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43508 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49816 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49830 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55210 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36126 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55170 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55184 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50496 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54982 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60622 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60626 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35118 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52118 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36276 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36290 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44138 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34162 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53306 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53318 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58160 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34486 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39632 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39636 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42710 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55790 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52514 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52528 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56892 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60350 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53938 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53942 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55782 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39516 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41244 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41256 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48244 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55380 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41080 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41086 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43206 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39968 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59482 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59486 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43094 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45068 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57492 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57498 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40318 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53450 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38452 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38454 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50078 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53362 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37438 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37446 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51924 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59126 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40838 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40854 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57568 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34184 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41936 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41950 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53486 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59434 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41404 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41412 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44482 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47666 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39988 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39998 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49598 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39290 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41846 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41858 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40720 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41348 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58392 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58398 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53642 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35728 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45552 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45568 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52292 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38772 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35038 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35054 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41968 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51760 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57914 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57916 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40074 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39718 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48998 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49014 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46352 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40332 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51732 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51748 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42944 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53886 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49958 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49968 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47016 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33818 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38884 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38888 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48818 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37376 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58948 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58950 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39116 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34368 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51356 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51366 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56972 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54352 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39648 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39654 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37912 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60190 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42332 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42344 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33932 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38062 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53810 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53812 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60248 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44660 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58834 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58838 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38546 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57182 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48194 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48210 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46688 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50296 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48450 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48456 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57164 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43414 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37328 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37342 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44078 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35116 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57036 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57046 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55188 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54162 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56734 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56750 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53164 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45378 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36346 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36350 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36880 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42812 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43332 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43340 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51140 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41510 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32790 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:32794 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48324 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48134 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45920 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45930 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54226 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54810 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40464 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40476 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37434 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33972 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49352 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49366 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48490 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37452 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41130 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41140 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32892 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48042 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43292 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43306 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37206 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40472 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54564 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54574 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41240 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51320 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40288 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40300 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59750 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54942 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58932 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58944 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46362 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59228 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48796 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48798 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49470 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38272 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33322 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33326 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32798 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37700 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54576 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54592 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51670 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58092 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59140 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59156 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39506 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42410 - "GET /node/6942e38ae43d2b76e9023aac%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:42412 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55212 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55228 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38700 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33462 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33474 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33490 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54812 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40114 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37306 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37316 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47998 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53182 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38724 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38726 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40444 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59958 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57376 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57378 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50934 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45000 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45436 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45446 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37206 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37398 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37410 - "GET /node/69a4e1227caa0b8e1b59dd07%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:48736 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48738 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54526 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35866 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57324 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57332 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51628 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49920 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54258 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54262 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50304 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58866 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50500 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50502 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35442 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54462 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40690 - "GET /node/69b140c7a870a1dc3343c8dc%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:40694 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40696 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57220 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40066 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59978 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59992 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51520 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45808 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32798 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32808 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57276 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54452 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48750 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48764 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47302 - "GET /node/69862ad2177ca0f097bd21ca%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:47316 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58620 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40082 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40092 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37040 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39388 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38202 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38206 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42190 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43086 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60878 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60888 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59786 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57572 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41918 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41930 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55578 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55556 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38328 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38344 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57294 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45512 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33490 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33494 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56050 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48784 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53098 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53100 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59088 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37670 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45960 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45970 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45980 - "GET /node/698de5d7594ec372d06988e3%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:51586 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45192 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44206 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44216 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48604 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34800 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58986 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58988 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36986 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51862 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36170 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36186 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55778 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34554 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45540 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45542 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38848 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50284 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36304 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36320 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57104 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41964 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50824 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50838 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56324 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52532 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40872 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40878 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40688 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46894 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60242 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60256 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36896 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41324 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36746 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36756 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58492 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58508 - "GET /node/698460f23a8748f2820864c5%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:58524 - "GET /node/69848d563a8748f282087995%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:34044 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47126 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47138 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49188 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37434 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34404 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34414 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47292 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38298 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36576 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36582 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36598 - "GET /node/6985c57e71989fb14149844c%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:35240 - "GET /node/69903797985e4efca431ceb3%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:35252 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40384 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41774 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41784 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42934 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43896 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40966 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40972 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45562 - "GET /node/6992db5631166481c8dd4a85%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:45566 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49864 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48096 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48108 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50286 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53760 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41378 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41384 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58426 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53222 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37330 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37336 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57744 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47782 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60510 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60512 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33122 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59452 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34662 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34678 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46538 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41942 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59296 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59300 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41352 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53526 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42392 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42400 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49596 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47508 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57090 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57094 - "GET / HTTP/1.0" 200 OK 2026-03-31 04:45:16,703 - lava_callback - INFO - Received callback for node 69c8ae4af41e4fb6a3786212 INFO: 172.20.0.1:57104 - "POST /node/69c8ae4af41e4fb6a3786212 HTTP/1.0" 202 Accepted 2026-03-31 04:45:16,708 - lava_callback - INFO - Starting async job submit for node 69c8ae4af41e4fb6a3786212 2026-03-31 04:45:16,709 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 04:45:16,734 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8ae4af41e4fb6a3786212 HTTP/1.1" 200 None 2026-03-31 04:45:16,735 - lava_callback - DEBUG - Processing node 69c8ae4af41e4fb6a3786212: baseline-arm-clabbe 2026-03-31 04:45:16,735 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8ae4af41e4fb6a3786212 2026-03-31 04:45:16,736 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 04:45:16,755 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8ae4af41e4fb6a3786212 HTTP/1.1" 200 None 2026-03-31 04:45:16,757 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 04:45:16,772 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7ff43123b47fd2376d74b HTTP/1.1" 200 None 2026-03-31 04:45:16,773 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 04:45:16,797 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8ae4af41e4fb6a3786212 HTTP/1.1" 200 None 2026-03-31 04:45:16,803 - lava_callback - ERROR - Error processing callback for node 69c8ae4af41e4fb6a3786212: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:36050 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43166 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56560 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56570 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38360 - "GET / HTTP/1.0" 200 OK 2026-03-31 04:45:56,668 - lava_callback - INFO - Received callback for node 69c8ae7cf41e4fb6a3786223 INFO: 172.20.0.1:38370 - "POST /node/69c8ae7cf41e4fb6a3786223 HTTP/1.0" 202 Accepted 2026-03-31 04:45:56,673 - lava_callback - INFO - Starting async job submit for node 69c8ae7cf41e4fb6a3786223 2026-03-31 04:45:56,674 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 04:45:56,688 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8ae7cf41e4fb6a3786223 HTTP/1.1" 200 None 2026-03-31 04:45:56,689 - lava_callback - DEBUG - Processing node 69c8ae7cf41e4fb6a3786223: baseline-arm-clabbe 2026-03-31 04:45:56,689 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8ae7cf41e4fb6a3786223 2026-03-31 04:45:56,690 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 04:45:56,711 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8ae7cf41e4fb6a3786223 HTTP/1.1" 200 None 2026-03-31 04:45:56,712 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 04:45:56,726 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7ff43123b47fd2376d74b HTTP/1.1" 200 None 2026-03-31 04:45:56,727 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 04:45:56,747 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8ae7cf41e4fb6a3786223 HTTP/1.1" 200 None 2026-03-31 04:45:56,754 - lava_callback - ERROR - Error processing callback for node 69c8ae7cf41e4fb6a3786223: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:41314 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36142 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36152 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56944 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58604 - "GET / HTTP/1.0" 200 OK 2026-03-31 04:46:36,753 - lava_callback - INFO - Received callback for node 69c8aea0f41e4fb6a3786230 INFO: 172.20.0.1:58620 - "POST /node/69c8aea0f41e4fb6a3786230 HTTP/1.0" 202 Accepted 2026-03-31 04:46:36,759 - lava_callback - INFO - Starting async job submit for node 69c8aea0f41e4fb6a3786230 2026-03-31 04:46:36,760 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 04:46:36,773 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8aea0f41e4fb6a3786230 HTTP/1.1" 200 None 2026-03-31 04:46:36,773 - lava_callback - DEBUG - Processing node 69c8aea0f41e4fb6a3786230: baseline-arm-clabbe 2026-03-31 04:46:36,774 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8aea0f41e4fb6a3786230 2026-03-31 04:46:36,774 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 04:46:36,794 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8aea0f41e4fb6a3786230 HTTP/1.1" 200 None 2026-03-31 04:46:36,796 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 04:46:36,809 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7ff43123b47fd2376d74b HTTP/1.1" 200 None 2026-03-31 04:46:36,810 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 04:46:36,830 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8aea0f41e4fb6a3786230 HTTP/1.1" 200 None 2026-03-31 04:46:36,831 - lava_callback - ERROR - Error processing callback for node 69c8aea0f41e4fb6a3786230: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:38662 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38664 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47696 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41582 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58706 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58718 - "GET / HTTP/1.0" 200 OK 2026-03-31 04:47:16,705 - lava_callback - INFO - Received callback for node 69c8aec4f41e4fb6a378623e INFO: 172.20.0.1:58732 - "POST /node/69c8aec4f41e4fb6a378623e HTTP/1.0" 202 Accepted 2026-03-31 04:47:16,710 - lava_callback - INFO - Starting async job submit for node 69c8aec4f41e4fb6a378623e 2026-03-31 04:47:16,711 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 04:47:16,725 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8aec4f41e4fb6a378623e HTTP/1.1" 200 None 2026-03-31 04:47:16,726 - lava_callback - DEBUG - Processing node 69c8aec4f41e4fb6a378623e: baseline-arm-clabbe 2026-03-31 04:47:16,726 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8aec4f41e4fb6a378623e 2026-03-31 04:47:16,727 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 04:47:16,741 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8aec4f41e4fb6a378623e HTTP/1.1" 200 None 2026-03-31 04:47:16,743 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 04:47:16,756 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7ff43123b47fd2376d74b HTTP/1.1" 200 None 2026-03-31 04:47:16,758 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 04:47:16,778 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8aec4f41e4fb6a378623e HTTP/1.1" 200 None 2026-03-31 04:47:16,788 - lava_callback - ERROR - Error processing callback for node 69c8aec4f41e4fb6a378623e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:45902 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52354 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35790 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35798 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57386 - "GET / HTTP/1.0" 200 OK 2026-03-31 04:47:56,810 - lava_callback - INFO - Received callback for node 69c8aee9f41e4fb6a3786248 INFO: 172.20.0.1:57390 - "POST /node/69c8aee9f41e4fb6a3786248 HTTP/1.0" 202 Accepted 2026-03-31 04:47:56,815 - lava_callback - INFO - Starting async job submit for node 69c8aee9f41e4fb6a3786248 2026-03-31 04:47:56,816 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 04:47:56,832 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8aee9f41e4fb6a3786248 HTTP/1.1" 200 None 2026-03-31 04:47:56,833 - lava_callback - DEBUG - Processing node 69c8aee9f41e4fb6a3786248: baseline-arm-clabbe 2026-03-31 04:47:56,833 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8aee9f41e4fb6a3786248 2026-03-31 04:47:56,834 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 04:47:56,847 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8aee9f41e4fb6a3786248 HTTP/1.1" 200 None 2026-03-31 04:47:56,849 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 04:47:56,869 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7ff43123b47fd2376d74b HTTP/1.1" 200 None 2026-03-31 04:47:56,870 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 04:47:56,895 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8aee9f41e4fb6a3786248 HTTP/1.1" 200 None 2026-03-31 04:47:56,897 - lava_callback - ERROR - Error processing callback for node 69c8aee9f41e4fb6a3786248: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:46446 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32948 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:32952 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55500 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48200 - "GET /node/6928eef1a7fbd8958ccef96c%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:48204 - "GET /node/69840cd7969566dcb0d2bcf1%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:48210 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38854 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38866 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42034 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39334 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39984 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40000 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35910 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53392 - "GET /node/698bda531e517752bab8b8f3%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:53404 - "GET /node/6928deb342a95a572a4f0689%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:53410 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47190 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47202 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59900 - "GET /node/69878b72e31b58f834f1bc37%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:59908 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43186 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43482 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43496 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58984 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50644 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34574 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34578 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54344 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32972 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57478 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57486 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54080 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60868 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46758 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46768 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45324 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38440 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48994 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49006 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54538 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47348 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49002 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49004 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43318 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51108 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60112 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60118 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54990 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50618 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33894 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33910 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48300 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60780 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48294 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48302 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35578 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33082 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49898 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49908 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40214 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60890 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51660 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51670 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45222 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54992 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57056 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57064 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52794 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32998 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54356 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54364 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42490 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34720 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57070 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57086 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42546 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49042 - "GET / HTTP/1.0" 200 OK 2026-03-31 04:57:14,431 - lava_callback - INFO - Received callback for node 69c8b118f41e4fb6a3786287 INFO: 172.20.0.1:35188 - "POST /node/69c8b118f41e4fb6a3786287 HTTP/1.0" 202 Accepted 2026-03-31 04:57:14,436 - lava_callback - INFO - Starting async job submit for node 69c8b118f41e4fb6a3786287 2026-03-31 04:57:14,437 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 04:57:14,467 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8b118f41e4fb6a3786287 HTTP/1.1" 200 None 2026-03-31 04:57:14,468 - lava_callback - DEBUG - Processing node 69c8b118f41e4fb6a3786287: baseline-arm64-baylibre 2026-03-31 04:57:14,468 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8b118f41e4fb6a3786287 2026-03-31 04:57:14,469 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 04:57:14,502 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8b118f41e4fb6a3786287 HTTP/1.1" 200 None 2026-03-31 04:57:14,503 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 04:57:14,517 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-03-31 04:57:14,519 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 04:57:14,540 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8b118f41e4fb6a3786287 HTTP/1.1" 200 None 2026-03-31 04:57:14,546 - lava_callback - ERROR - Error processing callback for node 69c8b118f41e4fb6a3786287: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:35204 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35218 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48884 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55424 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47876 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47886 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39204 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59332 - "GET / HTTP/1.0" 200 OK 2026-03-31 04:58:14,334 - lava_callback - INFO - Received callback for node 69c8b163f41e4fb6a3786290 INFO: 172.20.0.1:45396 - "POST /node/69c8b163f41e4fb6a3786290 HTTP/1.0" 202 Accepted 2026-03-31 04:58:14,339 - lava_callback - INFO - Starting async job submit for node 69c8b163f41e4fb6a3786290 2026-03-31 04:58:14,340 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 04:58:14,364 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8b163f41e4fb6a3786290 HTTP/1.1" 200 None 2026-03-31 04:58:14,365 - lava_callback - DEBUG - Processing node 69c8b163f41e4fb6a3786290: baseline-arm64-baylibre 2026-03-31 04:58:14,365 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8b163f41e4fb6a3786290 2026-03-31 04:58:14,366 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 04:58:14,380 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8b163f41e4fb6a3786290 HTTP/1.1" 200 None 2026-03-31 04:58:14,382 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 04:58:14,394 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-03-31 04:58:14,396 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 04:58:14,422 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8b163f41e4fb6a3786290 HTTP/1.1" 200 None 2026-03-31 04:58:14,423 - lava_callback - ERROR - Error processing callback for node 69c8b163f41e4fb6a3786290: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:45410 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45412 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47256 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51580 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42976 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42980 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38860 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48680 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39120 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39132 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34692 - "GET / HTTP/1.0" 200 OK 2026-03-31 04:59:34,441 - lava_callback - INFO - Received callback for node 69c8b1b1f41e4fb6a378629d INFO: 172.20.0.1:35742 - "POST /node/69c8b1b1f41e4fb6a378629d HTTP/1.0" 202 Accepted 2026-03-31 04:59:34,447 - lava_callback - INFO - Starting async job submit for node 69c8b1b1f41e4fb6a378629d 2026-03-31 04:59:34,448 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 04:59:34,465 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8b1b1f41e4fb6a378629d HTTP/1.1" 200 None 2026-03-31 04:59:34,466 - lava_callback - DEBUG - Processing node 69c8b1b1f41e4fb6a378629d: baseline-arm64-baylibre 2026-03-31 04:59:34,466 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8b1b1f41e4fb6a378629d 2026-03-31 04:59:34,467 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 04:59:34,482 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8b1b1f41e4fb6a378629d HTTP/1.1" 200 None 2026-03-31 04:59:34,483 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 04:59:34,506 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-03-31 04:59:34,507 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 04:59:34,530 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8b1b1f41e4fb6a378629d HTTP/1.1" 200 None 2026-03-31 04:59:34,537 - lava_callback - ERROR - Error processing callback for node 69c8b1b1f41e4fb6a378629d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:35748 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44358 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44374 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44058 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56296 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59040 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59048 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53256 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34636 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54666 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54678 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40100 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50170 - "GET / HTTP/1.0" 200 OK 2026-03-31 05:01:14,570 - lava_callback - INFO - Received callback for node 69c8b209f41e4fb6a37862a9 INFO: 172.20.0.1:53368 - "POST /node/69c8b209f41e4fb6a37862a9 HTTP/1.0" 202 Accepted 2026-03-31 05:01:14,576 - lava_callback - INFO - Starting async job submit for node 69c8b209f41e4fb6a37862a9 2026-03-31 05:01:14,577 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:01:14,603 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8b209f41e4fb6a37862a9 HTTP/1.1" 200 None 2026-03-31 05:01:14,604 - lava_callback - DEBUG - Processing node 69c8b209f41e4fb6a37862a9: baseline-arm64-baylibre 2026-03-31 05:01:14,604 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8b209f41e4fb6a37862a9 2026-03-31 05:01:14,605 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:01:14,617 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8b209f41e4fb6a37862a9 HTTP/1.1" 200 None 2026-03-31 05:01:14,619 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:01:14,631 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-03-31 05:01:14,633 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:01:14,653 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8b209f41e4fb6a37862a9 HTTP/1.1" 200 None 2026-03-31 05:01:14,661 - lava_callback - ERROR - Error processing callback for node 69c8b209f41e4fb6a37862a9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:53374 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53378 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52158 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34966 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38558 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38562 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47644 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55352 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41036 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41044 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52814 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44642 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33466 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33478 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56074 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52978 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33598 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33602 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41012 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58364 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43758 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43774 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46778 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52044 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44726 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44730 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58336 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49920 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41904 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41916 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43358 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49212 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49096 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49110 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42520 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44722 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37662 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37668 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41010 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43404 - "GET / HTTP/1.0" 200 OK 2026-03-31 05:06:15,056 - lava_callback - INFO - Received callback for node 69c8b342f41e4fb6a37862da INFO: 172.20.0.1:51150 - "POST /node/69c8b342f41e4fb6a37862da HTTP/1.0" 202 Accepted 2026-03-31 05:06:15,061 - lava_callback - INFO - Starting async job submit for node 69c8b342f41e4fb6a37862da 2026-03-31 05:06:15,062 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:06:15,076 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8b342f41e4fb6a37862da HTTP/1.1" 200 None 2026-03-31 05:06:15,077 - lava_callback - DEBUG - Processing node 69c8b342f41e4fb6a37862da: baseline-arm64-baylibre 2026-03-31 05:06:15,077 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8b342f41e4fb6a37862da 2026-03-31 05:06:15,077 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:06:15,090 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8b342f41e4fb6a37862da HTTP/1.1" 200 None 2026-03-31 05:06:15,092 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:06:15,105 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-03-31 05:06:15,106 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:06:15,126 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8b342f41e4fb6a37862da HTTP/1.1" 200 None 2026-03-31 05:06:15,131 - lava_callback - ERROR - Error processing callback for node 69c8b342f41e4fb6a37862da: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:51164 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51176 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52032 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58554 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34900 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34904 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37504 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45790 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50636 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50644 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60066 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33706 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51034 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51042 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59932 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60422 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60146 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60152 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50990 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40308 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40458 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40462 - "GET / HTTP/1.0" 200 OK 2026-03-31 05:08:55,275 - lava_callback - INFO - Received callback for node 69c8b3d6f41e4fb6a37862f1 INFO: 172.20.0.1:44628 - "POST /node/69c8b3d6f41e4fb6a37862f1 HTTP/1.0" 202 Accepted 2026-03-31 05:08:55,280 - lava_callback - INFO - Starting async job submit for node 69c8b3d6f41e4fb6a37862f1 2026-03-31 05:08:55,281 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:08:55,295 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8b3d6f41e4fb6a37862f1 HTTP/1.1" 200 None 2026-03-31 05:08:55,296 - lava_callback - DEBUG - Processing node 69c8b3d6f41e4fb6a37862f1: baseline-arm64-baylibre 2026-03-31 05:08:55,297 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8b3d6f41e4fb6a37862f1 2026-03-31 05:08:55,297 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:08:55,310 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8b3d6f41e4fb6a37862f1 HTTP/1.1" 200 None 2026-03-31 05:08:55,311 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:08:55,324 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-03-31 05:08:55,326 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:08:55,345 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8b3d6f41e4fb6a37862f1 HTTP/1.1" 200 None 2026-03-31 05:08:55,350 - lava_callback - ERROR - Error processing callback for node 69c8b3d6f41e4fb6a37862f1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:44644 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48800 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43062 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43076 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54018 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52622 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34012 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34020 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37506 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59860 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39066 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39078 - "GET / HTTP/1.0" 200 OK 2026-03-31 05:10:17,759 - lava_callback - INFO - Received callback for node 69c8b429f41e4fb6a3786305 INFO: 172.20.0.1:39082 - "POST /node/69c8b429f41e4fb6a3786305 HTTP/1.0" 202 Accepted 2026-03-31 05:10:17,765 - lava_callback - INFO - Starting async job submit for node 69c8b429f41e4fb6a3786305 2026-03-31 05:10:17,766 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:10:17,786 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8b429f41e4fb6a3786305 HTTP/1.1" 200 None 2026-03-31 05:10:17,786 - lava_callback - DEBUG - Processing node 69c8b429f41e4fb6a3786305: baseline-arm64-clabbe 2026-03-31 05:10:17,787 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8b429f41e4fb6a3786305 2026-03-31 05:10:17,787 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:10:17,800 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8b429f41e4fb6a3786305 HTTP/1.1" 200 None 2026-03-31 05:10:17,802 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:10:17,815 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-03-31 05:10:17,816 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:10:17,837 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8b429f41e4fb6a3786305 HTTP/1.1" 200 None 2026-03-31 05:10:17,842 - lava_callback - ERROR - Error processing callback for node 69c8b429f41e4fb6a3786305: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:55332 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37910 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47648 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47664 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38354 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36870 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51040 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51050 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50966 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56956 - "GET / HTTP/1.0" 200 OK 2026-03-31 05:11:37,812 - lava_callback - INFO - Received callback for node 69c8b475f41e4fb6a378630e INFO: 172.20.0.1:56960 - "POST /node/69c8b475f41e4fb6a378630e HTTP/1.0" 202 Accepted 2026-03-31 05:11:37,818 - lava_callback - INFO - Starting async job submit for node 69c8b475f41e4fb6a378630e 2026-03-31 05:11:37,819 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:11:37,833 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8b475f41e4fb6a378630e HTTP/1.1" 200 None 2026-03-31 05:11:37,834 - lava_callback - DEBUG - Processing node 69c8b475f41e4fb6a378630e: baseline-arm64-clabbe 2026-03-31 05:11:37,834 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8b475f41e4fb6a378630e 2026-03-31 05:11:37,835 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:11:37,848 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8b475f41e4fb6a378630e HTTP/1.1" 200 None 2026-03-31 05:11:37,850 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:11:37,863 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-03-31 05:11:37,865 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:11:37,885 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8b475f41e4fb6a378630e HTTP/1.1" 200 None 2026-03-31 05:11:37,886 - lava_callback - ERROR - Error processing callback for node 69c8b475f41e4fb6a378630e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:53240 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53242 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54152 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36834 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35280 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35286 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41284 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42988 - "GET / HTTP/1.0" 200 OK 2026-03-31 05:12:37,815 - lava_callback - INFO - Received callback for node 69c8b4c0f41e4fb6a378631b INFO: 172.20.0.1:42992 - "POST /node/69c8b4c0f41e4fb6a378631b HTTP/1.0" 202 Accepted 2026-03-31 05:12:37,821 - lava_callback - INFO - Starting async job submit for node 69c8b4c0f41e4fb6a378631b 2026-03-31 05:12:37,822 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:12:37,836 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8b4c0f41e4fb6a378631b HTTP/1.1" 200 None 2026-03-31 05:12:37,837 - lava_callback - DEBUG - Processing node 69c8b4c0f41e4fb6a378631b: baseline-arm64-clabbe 2026-03-31 05:12:37,837 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8b4c0f41e4fb6a378631b 2026-03-31 05:12:37,838 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:12:37,852 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8b4c0f41e4fb6a378631b HTTP/1.1" 200 None 2026-03-31 05:12:37,854 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:12:37,868 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-03-31 05:12:37,869 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:12:37,890 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8b4c0f41e4fb6a378631b HTTP/1.1" 200 None 2026-03-31 05:12:37,898 - lava_callback - ERROR - Error processing callback for node 69c8b4c0f41e4fb6a378631b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:58944 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58948 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58128 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37510 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32794 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:32806 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32934 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58828 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55346 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55348 - "GET / HTTP/1.0" 200 OK 2026-03-31 05:13:55,295 - lava_callback - INFO - Received callback for node 69c8b505f41e4fb6a3786324 INFO: 172.20.0.1:42482 - "POST /node/69c8b505f41e4fb6a3786324 HTTP/1.0" 202 Accepted 2026-03-31 05:13:55,301 - lava_callback - INFO - Starting async job submit for node 69c8b505f41e4fb6a3786324 2026-03-31 05:13:55,302 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:13:55,317 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8b505f41e4fb6a3786324 HTTP/1.1" 200 None 2026-03-31 05:13:55,319 - lava_callback - DEBUG - Processing node 69c8b505f41e4fb6a3786324: baseline-arm64-baylibre 2026-03-31 05:13:55,319 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8b505f41e4fb6a3786324 2026-03-31 05:13:55,320 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:13:55,338 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8b505f41e4fb6a3786324 HTTP/1.1" 200 None 2026-03-31 05:13:55,341 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:13:55,356 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-03-31 05:13:55,357 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:13:55,380 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8b505f41e4fb6a3786324 HTTP/1.1" 200 None 2026-03-31 05:13:55,381 - lava_callback - ERROR - Error processing callback for node 69c8b505f41e4fb6a3786324: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:42498 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34394 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54730 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54736 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35938 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54740 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48338 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48344 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38954 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60246 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51632 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51648 - "GET / HTTP/1.0" 200 OK 2026-03-31 05:15:17,844 - lava_callback - INFO - Received callback for node 69c8b554f41e4fb6a378633a INFO: 172.20.0.1:51650 - "POST /node/69c8b554f41e4fb6a378633a HTTP/1.0" 202 Accepted 2026-03-31 05:15:17,850 - lava_callback - INFO - Starting async job submit for node 69c8b554f41e4fb6a378633a 2026-03-31 05:15:17,851 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:15:17,865 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8b554f41e4fb6a378633a HTTP/1.1" 200 None 2026-03-31 05:15:17,866 - lava_callback - DEBUG - Processing node 69c8b554f41e4fb6a378633a: baseline-arm64-clabbe 2026-03-31 05:15:17,866 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8b554f41e4fb6a378633a 2026-03-31 05:15:17,867 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:15:17,884 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8b554f41e4fb6a378633a HTTP/1.1" 200 None 2026-03-31 05:15:17,885 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:15:17,898 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-03-31 05:15:17,900 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:15:17,925 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8b554f41e4fb6a378633a HTTP/1.1" 200 None 2026-03-31 05:15:17,931 - lava_callback - ERROR - Error processing callback for node 69c8b554f41e4fb6a378633a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:36594 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49972 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47694 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47706 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39808 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58296 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45422 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45430 - "GET / HTTP/1.0" 200 OK 2026-03-31 05:16:17,964 - lava_callback - INFO - Received callback for node 69c8b59ff41e4fb6a3786342 INFO: 172.20.0.1:45436 - "POST /node/69c8b59ff41e4fb6a3786342 HTTP/1.0" 202 Accepted 2026-03-31 05:16:17,971 - lava_callback - INFO - Starting async job submit for node 69c8b59ff41e4fb6a3786342 2026-03-31 05:16:17,972 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:16:17,986 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8b59ff41e4fb6a3786342 HTTP/1.1" 200 None 2026-03-31 05:16:17,987 - lava_callback - DEBUG - Processing node 69c8b59ff41e4fb6a3786342: baseline-arm64-clabbe 2026-03-31 05:16:17,988 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8b59ff41e4fb6a3786342 2026-03-31 05:16:17,988 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:16:18,002 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8b59ff41e4fb6a3786342 HTTP/1.1" 200 None 2026-03-31 05:16:18,003 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:16:18,020 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-03-31 05:16:18,022 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:16:18,049 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8b59ff41e4fb6a3786342 HTTP/1.1" 200 None 2026-03-31 05:16:18,050 - lava_callback - ERROR - Error processing callback for node 69c8b59ff41e4fb6a3786342: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:39076 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53926 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42248 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42258 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43470 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36094 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34630 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34642 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42714 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54788 - "GET / HTTP/1.0" 200 OK 2026-03-31 05:17:38,068 - lava_callback - INFO - Received callback for node 69c8b5e8f41e4fb6a3786351 INFO: 172.20.0.1:54792 - "POST /node/69c8b5e8f41e4fb6a3786351 HTTP/1.0" 202 Accepted 2026-03-31 05:17:38,073 - lava_callback - INFO - Starting async job submit for node 69c8b5e8f41e4fb6a3786351 2026-03-31 05:17:38,074 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:17:38,089 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8b5e8f41e4fb6a3786351 HTTP/1.1" 200 None 2026-03-31 05:17:38,090 - lava_callback - DEBUG - Processing node 69c8b5e8f41e4fb6a3786351: baseline-arm64-clabbe 2026-03-31 05:17:38,090 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8b5e8f41e4fb6a3786351 2026-03-31 05:17:38,091 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:17:38,105 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8b5e8f41e4fb6a3786351 HTTP/1.1" 200 None 2026-03-31 05:17:38,106 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:17:38,120 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-03-31 05:17:38,121 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:17:38,142 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8b5e8f41e4fb6a3786351 HTTP/1.1" 200 None 2026-03-31 05:17:38,148 - lava_callback - ERROR - Error processing callback for node 69c8b5e8f41e4fb6a3786351: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:38266 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38268 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60218 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33540 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37626 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37636 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57222 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53574 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35094 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35108 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58884 - "GET / HTTP/1.0" 200 OK 2026-03-31 05:18:58,112 - lava_callback - INFO - Received callback for node 69c8b632f41e4fb6a3786359 INFO: 172.20.0.1:58890 - "POST /node/69c8b632f41e4fb6a3786359 HTTP/1.0" 202 Accepted 2026-03-31 05:18:58,118 - lava_callback - INFO - Starting async job submit for node 69c8b632f41e4fb6a3786359 2026-03-31 05:18:58,118 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:18:58,135 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8b632f41e4fb6a3786359 HTTP/1.1" 200 None 2026-03-31 05:18:58,136 - lava_callback - DEBUG - Processing node 69c8b632f41e4fb6a3786359: baseline-arm64-clabbe 2026-03-31 05:18:58,136 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8b632f41e4fb6a3786359 2026-03-31 05:18:58,137 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:18:58,151 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8b632f41e4fb6a3786359 HTTP/1.1" 200 None 2026-03-31 05:18:58,152 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:18:58,165 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-03-31 05:18:58,167 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:18:58,188 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8b632f41e4fb6a3786359 HTTP/1.1" 200 None 2026-03-31 05:18:58,190 - lava_callback - ERROR - Error processing callback for node 69c8b632f41e4fb6a3786359: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:58424 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43706 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43718 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58176 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36062 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41710 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41712 - "GET / HTTP/1.0" 200 OK 2026-03-31 05:19:55,761 - lava_callback - INFO - Received callback for node 69c8b66af41e4fb6a3786363 INFO: 172.20.0.1:55918 - "POST /node/69c8b66af41e4fb6a3786363 HTTP/1.0" 202 Accepted 2026-03-31 05:19:55,767 - lava_callback - INFO - Starting async job submit for node 69c8b66af41e4fb6a3786363 2026-03-31 05:19:55,768 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:19:55,782 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8b66af41e4fb6a3786363 HTTP/1.1" 200 None 2026-03-31 05:19:55,783 - lava_callback - DEBUG - Processing node 69c8b66af41e4fb6a3786363: baseline-arm-baylibre 2026-03-31 05:19:55,783 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8b66af41e4fb6a3786363 2026-03-31 05:19:55,784 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:19:55,797 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8b66af41e4fb6a3786363 HTTP/1.1" 200 None 2026-03-31 05:19:55,798 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:19:55,812 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c55c70abf66a8f4d9b9655 HTTP/1.1" 200 None 2026-03-31 05:19:55,813 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:19:55,835 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8b66af41e4fb6a3786363 HTTP/1.1" 200 None 2026-03-31 05:19:55,836 - lava_callback - ERROR - Error processing callback for node 69c8b66af41e4fb6a3786363: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:55930 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56822 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41672 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41676 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54472 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56878 - "GET / HTTP/1.0" 200 OK 2026-03-31 05:20:38,176 - lava_callback - INFO - Received callback for node 69c8b6a0f41e4fb6a3786375 INFO: 172.20.0.1:56890 - "POST /node/69c8b6a0f41e4fb6a3786375 HTTP/1.0" 202 Accepted 2026-03-31 05:20:38,182 - lava_callback - INFO - Starting async job submit for node 69c8b6a0f41e4fb6a3786375 2026-03-31 05:20:38,182 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:20:38,197 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8b6a0f41e4fb6a3786375 HTTP/1.1" 200 None 2026-03-31 05:20:38,198 - lava_callback - DEBUG - Processing node 69c8b6a0f41e4fb6a3786375: baseline-arm64-clabbe 2026-03-31 05:20:38,198 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8b6a0f41e4fb6a3786375 2026-03-31 05:20:38,198 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:20:38,213 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8b6a0f41e4fb6a3786375 HTTP/1.1" 200 None 2026-03-31 05:20:38,214 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:20:38,228 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-03-31 05:20:38,229 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:20:38,251 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8b6a0f41e4fb6a3786375 HTTP/1.1" 200 None 2026-03-31 05:20:38,252 - lava_callback - ERROR - Error processing callback for node 69c8b6a0f41e4fb6a3786375: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:49762 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49774 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36216 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49842 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52580 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52590 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60066 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50428 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46286 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46296 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39300 - "GET / HTTP/1.0" 200 OK 2026-03-31 05:21:58,194 - lava_callback - INFO - Received callback for node 69c8b6ebf41e4fb6a3786383 INFO: 172.20.0.1:39312 - "POST /node/69c8b6ebf41e4fb6a3786383 HTTP/1.0" 202 Accepted 2026-03-31 05:21:58,200 - lava_callback - INFO - Starting async job submit for node 69c8b6ebf41e4fb6a3786383 2026-03-31 05:21:58,201 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:21:58,233 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8b6ebf41e4fb6a3786383 HTTP/1.1" 200 None 2026-03-31 05:21:58,234 - lava_callback - DEBUG - Processing node 69c8b6ebf41e4fb6a3786383: baseline-arm64-clabbe 2026-03-31 05:21:58,234 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8b6ebf41e4fb6a3786383 2026-03-31 05:21:58,235 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:21:58,249 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8b6ebf41e4fb6a3786383 HTTP/1.1" 200 None 2026-03-31 05:21:58,250 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:21:58,281 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-03-31 05:21:58,282 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:21:58,304 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8b6ebf41e4fb6a3786383 HTTP/1.1" 200 None 2026-03-31 05:21:58,306 - lava_callback - ERROR - Error processing callback for node 69c8b6ebf41e4fb6a3786383: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:53610 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44238 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44250 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50288 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34614 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33960 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33968 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38398 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42264 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44058 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44072 - "GET / HTTP/1.0" 200 OK 2026-03-31 05:23:18,309 - lava_callback - INFO - Received callback for node 69c8b736f41e4fb6a3786391 INFO: 172.20.0.1:44074 - "POST /node/69c8b736f41e4fb6a3786391 HTTP/1.0" 202 Accepted 2026-03-31 05:23:18,314 - lava_callback - INFO - Starting async job submit for node 69c8b736f41e4fb6a3786391 2026-03-31 05:23:18,315 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:23:18,331 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8b736f41e4fb6a3786391 HTTP/1.1" 200 None 2026-03-31 05:23:18,332 - lava_callback - DEBUG - Processing node 69c8b736f41e4fb6a3786391: baseline-arm64-clabbe 2026-03-31 05:23:18,332 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8b736f41e4fb6a3786391 2026-03-31 05:23:18,333 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:23:18,347 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8b736f41e4fb6a3786391 HTTP/1.1" 200 None 2026-03-31 05:23:18,348 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:23:18,362 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-03-31 05:23:18,363 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:23:18,392 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8b736f41e4fb6a3786391 HTTP/1.1" 200 None 2026-03-31 05:23:18,393 - lava_callback - ERROR - Error processing callback for node 69c8b736f41e4fb6a3786391: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:54632 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41408 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38920 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38934 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49220 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48168 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40666 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-31 05:24:15,660 - lava_callback - INFO - Received callback for node 69c8b77bf41e4fb6a378639c INFO: 172.20.0.1:40680 - "POST /node/69c8b77bf41e4fb6a378639c HTTP/1.0" 202 Accepted 2026-03-31 05:24:15,666 - lava_callback - INFO - Starting async job submit for node 69c8b77bf41e4fb6a378639c 2026-03-31 05:24:15,667 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:24:15,683 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8b77bf41e4fb6a378639c HTTP/1.1" 200 None 2026-03-31 05:24:15,684 - lava_callback - DEBUG - Processing node 69c8b77bf41e4fb6a378639c: baseline-arm64-baylibre 2026-03-31 05:24:15,684 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8b77bf41e4fb6a378639c 2026-03-31 05:24:15,685 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:24:15,698 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8b77bf41e4fb6a378639c HTTP/1.1" 200 None 2026-03-31 05:24:15,700 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:24:15,712 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7ff4f123b47fd2376d75b HTTP/1.1" 200 None 2026-03-31 05:24:15,714 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:24:15,738 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8b77bf41e4fb6a378639c HTTP/1.1" 200 None 2026-03-31 05:24:15,739 - lava_callback - ERROR - Error processing callback for node 69c8b77bf41e4fb6a378639c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:40690 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39352 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59848 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48636 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48646 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52772 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50598 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60024 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60030 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43666 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44484 - "GET / HTTP/1.0" 200 OK 2026-03-31 05:25:38,368 - lava_callback - INFO - Received callback for node 69c8b7caf41e4fb6a37863ab INFO: 172.20.0.1:44488 - "POST /node/69c8b7caf41e4fb6a37863ab HTTP/1.0" 202 Accepted 2026-03-31 05:25:38,375 - lava_callback - INFO - Starting async job submit for node 69c8b7caf41e4fb6a37863ab 2026-03-31 05:25:38,376 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:25:38,392 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8b7caf41e4fb6a37863ab HTTP/1.1" 200 None 2026-03-31 05:25:38,393 - lava_callback - DEBUG - Processing node 69c8b7caf41e4fb6a37863ab: baseline-arm64-clabbe 2026-03-31 05:25:38,393 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8b7caf41e4fb6a37863ab 2026-03-31 05:25:38,394 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:25:38,411 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8b7caf41e4fb6a37863ab HTTP/1.1" 200 None 2026-03-31 05:25:38,413 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:25:38,428 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7ff4f123b47fd2376d75b HTTP/1.1" 200 None 2026-03-31 05:25:38,429 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:25:38,450 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8b7caf41e4fb6a37863ab HTTP/1.1" 200 None 2026-03-31 05:25:38,451 - lava_callback - ERROR - Error processing callback for node 69c8b7caf41e4fb6a37863ab: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:55460 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55470 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35628 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51630 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40708 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40724 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35334 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58274 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55754 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55764 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46598 - "GET / HTTP/1.0" 200 OK 2026-03-31 05:26:58,449 - lava_callback - INFO - Received callback for node 69c8b813f41e4fb6a37863b8 INFO: 172.20.0.1:46606 - "POST /node/69c8b813f41e4fb6a37863b8 HTTP/1.0" 202 Accepted 2026-03-31 05:26:58,456 - lava_callback - INFO - Starting async job submit for node 69c8b813f41e4fb6a37863b8 2026-03-31 05:26:58,457 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:26:58,479 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8b813f41e4fb6a37863b8 HTTP/1.1" 200 None 2026-03-31 05:26:58,480 - lava_callback - DEBUG - Processing node 69c8b813f41e4fb6a37863b8: baseline-arm64-clabbe 2026-03-31 05:26:58,480 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8b813f41e4fb6a37863b8 2026-03-31 05:26:58,481 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:26:58,502 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8b813f41e4fb6a37863b8 HTTP/1.1" 200 None 2026-03-31 05:26:58,504 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:26:58,525 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-03-31 05:26:58,526 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:26:58,551 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8b813f41e4fb6a37863b8 HTTP/1.1" 200 None 2026-03-31 05:26:58,556 - lava_callback - ERROR - Error processing callback for node 69c8b813f41e4fb6a37863b8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:56350 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45738 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45750 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55454 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59860 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35604 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35606 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58718 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34564 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54456 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54468 - "GET / HTTP/1.0" 200 OK 2026-03-31 05:28:18,566 - lava_callback - INFO - Received callback for node 69c8b861f41e4fb6a37863cc INFO: 172.20.0.1:54480 - "POST /node/69c8b861f41e4fb6a37863cc HTTP/1.0" 202 Accepted 2026-03-31 05:28:18,572 - lava_callback - INFO - Starting async job submit for node 69c8b861f41e4fb6a37863cc 2026-03-31 05:28:18,573 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:28:18,587 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8b861f41e4fb6a37863cc HTTP/1.1" 200 None 2026-03-31 05:28:18,587 - lava_callback - DEBUG - Processing node 69c8b861f41e4fb6a37863cc: baseline-arm64-clabbe 2026-03-31 05:28:18,588 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8b861f41e4fb6a37863cc 2026-03-31 05:28:18,588 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:28:18,602 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8b861f41e4fb6a37863cc HTTP/1.1" 200 None 2026-03-31 05:28:18,604 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:28:18,617 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7ff4f123b47fd2376d75b HTTP/1.1" 200 None 2026-03-31 05:28:18,619 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:28:18,639 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8b861f41e4fb6a37863cc HTTP/1.1" 200 None 2026-03-31 05:28:18,641 - lava_callback - ERROR - Error processing callback for node 69c8b861f41e4fb6a37863cc: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:58024 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47738 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38326 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38334 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40444 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48154 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35112 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35118 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60832 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36760 - "GET / HTTP/1.0" 200 OK 2026-03-31 05:29:38,574 - lava_callback - INFO - Received callback for node 69c8b8aef41e4fb6a37863d7 INFO: 172.20.0.1:36776 - "POST /node/69c8b8aef41e4fb6a37863d7 HTTP/1.0" 202 Accepted 2026-03-31 05:29:38,579 - lava_callback - INFO - Starting async job submit for node 69c8b8aef41e4fb6a37863d7 2026-03-31 05:29:38,580 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:29:38,598 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8b8aef41e4fb6a37863d7 HTTP/1.1" 200 None 2026-03-31 05:29:38,599 - lava_callback - DEBUG - Processing node 69c8b8aef41e4fb6a37863d7: baseline-arm64-clabbe 2026-03-31 05:29:38,599 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8b8aef41e4fb6a37863d7 2026-03-31 05:29:38,600 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:29:38,613 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8b8aef41e4fb6a37863d7 HTTP/1.1" 200 None 2026-03-31 05:29:38,615 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:29:38,629 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-03-31 05:29:38,630 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:29:38,652 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8b8aef41e4fb6a37863d7 HTTP/1.1" 200 None 2026-03-31 05:29:38,653 - lava_callback - ERROR - Error processing callback for node 69c8b8aef41e4fb6a37863d7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:43204 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43208 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41150 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59440 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52852 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52864 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53414 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54078 - "GET / HTTP/1.0" 200 OK 2026-03-31 05:30:38,596 - lava_callback - INFO - Received callback for node 69c8b8f9f41e4fb6a37863ec INFO: 172.20.0.1:54084 - "POST /node/69c8b8f9f41e4fb6a37863ec HTTP/1.0" 202 Accepted 2026-03-31 05:30:38,602 - lava_callback - INFO - Starting async job submit for node 69c8b8f9f41e4fb6a37863ec 2026-03-31 05:30:38,603 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:30:38,617 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8b8f9f41e4fb6a37863ec HTTP/1.1" 200 None 2026-03-31 05:30:38,618 - lava_callback - DEBUG - Processing node 69c8b8f9f41e4fb6a37863ec: baseline-arm64-clabbe 2026-03-31 05:30:38,618 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8b8f9f41e4fb6a37863ec 2026-03-31 05:30:38,619 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:30:38,632 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8b8f9f41e4fb6a37863ec HTTP/1.1" 200 None 2026-03-31 05:30:38,633 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:30:38,646 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7ff4f123b47fd2376d75b HTTP/1.1" 200 None 2026-03-31 05:30:38,648 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:30:38,668 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8b8f9f41e4fb6a37863ec HTTP/1.1" 200 None 2026-03-31 05:30:38,674 - lava_callback - ERROR - Error processing callback for node 69c8b8f9f41e4fb6a37863ec: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:54860 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54864 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47554 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60660 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49208 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49220 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56380 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58262 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40602 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40618 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45304 - "GET / HTTP/1.0" 200 OK 2026-03-31 05:31:58,596 - lava_callback - INFO - Received callback for node 69c8b943f41e4fb6a3786400 INFO: 172.20.0.1:45316 - "POST /node/69c8b943f41e4fb6a3786400 HTTP/1.0" 202 Accepted 2026-03-31 05:31:58,601 - lava_callback - INFO - Starting async job submit for node 69c8b943f41e4fb6a3786400 2026-03-31 05:31:58,602 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:31:58,619 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8b943f41e4fb6a3786400 HTTP/1.1" 200 None 2026-03-31 05:31:58,620 - lava_callback - DEBUG - Processing node 69c8b943f41e4fb6a3786400: baseline-arm64-clabbe 2026-03-31 05:31:58,620 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8b943f41e4fb6a3786400 2026-03-31 05:31:58,621 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:31:58,634 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8b943f41e4fb6a3786400 HTTP/1.1" 200 None 2026-03-31 05:31:58,635 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:31:58,651 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7ff4f123b47fd2376d75b HTTP/1.1" 200 None 2026-03-31 05:31:58,652 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:31:58,678 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8b943f41e4fb6a3786400 HTTP/1.1" 200 None 2026-03-31 05:31:58,679 - lava_callback - ERROR - Error processing callback for node 69c8b943f41e4fb6a3786400: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:45168 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58496 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58498 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47622 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45890 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57546 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57562 - "GET / HTTP/1.0" 200 OK 2026-03-31 05:32:56,219 - lava_callback - INFO - Received callback for node 69c8b97af41e4fb6a378640a INFO: 172.20.0.1:50742 - "POST /node/69c8b97af41e4fb6a378640a HTTP/1.0" 202 Accepted 2026-03-31 05:32:56,225 - lava_callback - INFO - Starting async job submit for node 69c8b97af41e4fb6a378640a 2026-03-31 05:32:56,226 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:32:56,240 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8b97af41e4fb6a378640a HTTP/1.1" 200 None 2026-03-31 05:32:56,240 - lava_callback - DEBUG - Processing node 69c8b97af41e4fb6a378640a: baseline-arm-baylibre 2026-03-31 05:32:56,241 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8b97af41e4fb6a378640a 2026-03-31 05:32:56,241 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:32:56,254 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8b97af41e4fb6a378640a HTTP/1.1" 200 None 2026-03-31 05:32:56,256 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:32:56,269 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c55c70abf66a8f4d9b9655 HTTP/1.1" 200 None 2026-03-31 05:32:56,271 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:32:56,291 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8b97af41e4fb6a378640a HTTP/1.1" 200 None 2026-03-31 05:32:56,292 - lava_callback - ERROR - Error processing callback for node 69c8b97af41e4fb6a378640a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:50748 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47620 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33776 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33788 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41328 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45740 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46182 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46198 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42478 - "GET / HTTP/1.0" 200 OK 2026-03-31 05:33:58,707 - lava_callback - INFO - Received callback for node 69c8b9b1f41e4fb6a378641d INFO: 172.20.0.1:42486 - "POST /node/69c8b9b1f41e4fb6a378641d HTTP/1.0" 202 Accepted 2026-03-31 05:33:58,714 - lava_callback - INFO - Starting async job submit for node 69c8b9b1f41e4fb6a378641d 2026-03-31 05:33:58,716 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:33:58,734 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8b9b1f41e4fb6a378641d HTTP/1.1" 200 None 2026-03-31 05:33:58,735 - lava_callback - DEBUG - Processing node 69c8b9b1f41e4fb6a378641d: baseline-arm64-clabbe 2026-03-31 05:33:58,736 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8b9b1f41e4fb6a378641d 2026-03-31 05:33:58,736 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:33:58,755 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8b9b1f41e4fb6a378641d HTTP/1.1" 200 None 2026-03-31 05:33:58,757 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:33:58,776 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7ff4f123b47fd2376d75b HTTP/1.1" 200 None 2026-03-31 05:33:58,778 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:33:58,802 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8b9b1f41e4fb6a378641d HTTP/1.1" 200 None 2026-03-31 05:33:58,809 - lava_callback - ERROR - Error processing callback for node 69c8b9b1f41e4fb6a378641d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:35018 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41212 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41224 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57888 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49416 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43806 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43814 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54608 - "GET / HTTP/1.0" 200 OK 2026-03-31 05:34:58,747 - lava_callback - INFO - Received callback for node 69c8b9fcf41e4fb6a3786433 INFO: 172.20.0.1:54622 - "POST /node/69c8b9fcf41e4fb6a3786433 HTTP/1.0" 202 Accepted 2026-03-31 05:34:58,754 - lava_callback - INFO - Starting async job submit for node 69c8b9fcf41e4fb6a3786433 2026-03-31 05:34:58,754 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:34:58,770 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8b9fcf41e4fb6a3786433 HTTP/1.1" 200 None 2026-03-31 05:34:58,771 - lava_callback - DEBUG - Processing node 69c8b9fcf41e4fb6a3786433: baseline-arm64-clabbe 2026-03-31 05:34:58,771 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8b9fcf41e4fb6a3786433 2026-03-31 05:34:58,772 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:34:58,786 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8b9fcf41e4fb6a3786433 HTTP/1.1" 200 None 2026-03-31 05:34:58,788 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:34:58,807 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7ff4f123b47fd2376d75b HTTP/1.1" 200 None 2026-03-31 05:34:58,809 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:34:58,841 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8b9fcf41e4fb6a3786433 HTTP/1.1" 200 None 2026-03-31 05:34:58,842 - lava_callback - ERROR - Error processing callback for node 69c8b9fcf41e4fb6a3786433: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:60904 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45146 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45154 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52290 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46796 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37670 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37684 - "GET / HTTP/1.0" 200 OK 2026-03-31 05:35:56,267 - lava_callback - INFO - Received callback for node 69c8ba34f41e4fb6a378643f INFO: 172.20.0.1:39310 - "POST /node/69c8ba34f41e4fb6a378643f HTTP/1.0" 202 Accepted 2026-03-31 05:35:56,275 - lava_callback - INFO - Starting async job submit for node 69c8ba34f41e4fb6a378643f 2026-03-31 05:35:56,276 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:35:56,290 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8ba34f41e4fb6a378643f HTTP/1.1" 200 None 2026-03-31 05:35:56,291 - lava_callback - DEBUG - Processing node 69c8ba34f41e4fb6a378643f: baseline-arm-baylibre 2026-03-31 05:35:56,291 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8ba34f41e4fb6a378643f 2026-03-31 05:35:56,292 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:35:56,309 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8ba34f41e4fb6a378643f HTTP/1.1" 200 None 2026-03-31 05:35:56,311 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:35:56,330 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c55c70abf66a8f4d9b9655 HTTP/1.1" 200 None 2026-03-31 05:35:56,332 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:35:56,357 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8ba34f41e4fb6a378643f HTTP/1.1" 200 None 2026-03-31 05:35:56,359 - lava_callback - ERROR - Error processing callback for node 69c8ba34f41e4fb6a378643f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:39314 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42248 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37762 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37774 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34238 - "GET / HTTP/1.0" 200 OK 2026-03-31 05:36:36,319 - lava_callback - INFO - Received callback for node 69c8ba58f41e4fb6a378644c INFO: 172.20.0.1:38426 - "POST /node/69c8ba58f41e4fb6a378644c HTTP/1.0" 202 Accepted 2026-03-31 05:36:36,325 - lava_callback - INFO - Starting async job submit for node 69c8ba58f41e4fb6a378644c 2026-03-31 05:36:36,326 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:36:36,344 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8ba58f41e4fb6a378644c HTTP/1.1" 200 None 2026-03-31 05:36:36,345 - lava_callback - DEBUG - Processing node 69c8ba58f41e4fb6a378644c: baseline-arm-baylibre 2026-03-31 05:36:36,345 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8ba58f41e4fb6a378644c 2026-03-31 05:36:36,346 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:36:36,360 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8ba58f41e4fb6a378644c HTTP/1.1" 200 None 2026-03-31 05:36:36,361 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:36:36,375 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c55c70abf66a8f4d9b9655 HTTP/1.1" 200 None 2026-03-31 05:36:36,376 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:36:36,397 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8ba58f41e4fb6a378644c HTTP/1.1" 200 None 2026-03-31 05:36:36,398 - lava_callback - ERROR - Error processing callback for node 69c8ba58f41e4fb6a378644c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:38440 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60342 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60348 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46198 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41448 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34728 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34730 - "GET / HTTP/1.0" 200 OK 2026-03-31 05:37:16,584 - lava_callback - INFO - Received callback for node 69c8ba7af41e4fb6a378645a INFO: 172.20.0.1:34744 - "POST /node/69c8ba7af41e4fb6a378645a HTTP/1.0" 202 Accepted 2026-03-31 05:37:16,589 - lava_callback - INFO - Starting async job submit for node 69c8ba7af41e4fb6a378645a 2026-03-31 05:37:16,590 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:37:16,605 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8ba7af41e4fb6a378645a HTTP/1.1" 200 None 2026-03-31 05:37:16,605 - lava_callback - DEBUG - Processing node 69c8ba7af41e4fb6a378645a: baseline-arm-baylibre 2026-03-31 05:37:16,606 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8ba7af41e4fb6a378645a 2026-03-31 05:37:16,606 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:37:16,619 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8ba7af41e4fb6a378645a HTTP/1.1" 200 None 2026-03-31 05:37:16,621 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:37:16,634 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c55c70abf66a8f4d9b9655 HTTP/1.1" 200 None 2026-03-31 05:37:16,636 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:37:16,656 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8ba7af41e4fb6a378645a HTTP/1.1" 200 None 2026-03-31 05:37:16,657 - lava_callback - ERROR - Error processing callback for node 69c8ba7af41e4fb6a378645a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:56136 - "GET / HTTP/1.0" 200 OK 2026-03-31 05:37:36,352 - lava_callback - INFO - Received callback for node 69c8ba9ef41e4fb6a3786464 INFO: 172.20.0.1:53604 - "POST /node/69c8ba9ef41e4fb6a3786464 HTTP/1.0" 202 Accepted 2026-03-31 05:37:36,357 - lava_callback - INFO - Starting async job submit for node 69c8ba9ef41e4fb6a3786464 2026-03-31 05:37:36,358 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:37:36,372 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8ba9ef41e4fb6a3786464 HTTP/1.1" 200 None 2026-03-31 05:37:36,373 - lava_callback - DEBUG - Processing node 69c8ba9ef41e4fb6a3786464: baseline-arm-baylibre 2026-03-31 05:37:36,373 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8ba9ef41e4fb6a3786464 2026-03-31 05:37:36,374 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:37:36,387 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8ba9ef41e4fb6a3786464 HTTP/1.1" 200 None 2026-03-31 05:37:36,389 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:37:36,402 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c55c70abf66a8f4d9b9655 HTTP/1.1" 200 None 2026-03-31 05:37:36,403 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:37:36,438 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8ba9ef41e4fb6a3786464 HTTP/1.1" 200 None 2026-03-31 05:37:36,439 - lava_callback - ERROR - Error processing callback for node 69c8ba9ef41e4fb6a3786464: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:53606 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44208 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44216 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37618 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57794 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55300 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-31 05:38:16,420 - lava_callback - INFO - Received callback for node 69c8bac2f41e4fb6a3786473 INFO: 172.20.0.1:55310 - "POST /node/69c8bac2f41e4fb6a3786473 HTTP/1.0" 202 Accepted 2026-03-31 05:38:16,432 - lava_callback - INFO - Starting async job submit for node 69c8bac2f41e4fb6a3786473 2026-03-31 05:38:16,433 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:38:16,462 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8bac2f41e4fb6a3786473 HTTP/1.1" 200 None 2026-03-31 05:38:16,464 - lava_callback - DEBUG - Processing node 69c8bac2f41e4fb6a3786473: baseline-arm-baylibre 2026-03-31 05:38:16,464 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8bac2f41e4fb6a3786473 2026-03-31 05:38:16,465 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:38:16,482 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8bac2f41e4fb6a3786473 HTTP/1.1" 200 None 2026-03-31 05:38:16,484 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:38:16,498 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c55c70abf66a8f4d9b9655 HTTP/1.1" 200 None 2026-03-31 05:38:16,499 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.20.0.1:55312 - "GET / HTTP/1.0" 200 OK 2026-03-31 05:38:16,523 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8bac2f41e4fb6a3786473 HTTP/1.1" 200 None 2026-03-31 05:38:16,524 - lava_callback - ERROR - Error processing callback for node 69c8bac2f41e4fb6a3786473: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:33990 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45648 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56766 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56768 - "GET / HTTP/1.0" 200 OK 2026-03-31 05:38:56,470 - lava_callback - INFO - Received callback for node 69c8bae5f41e4fb6a378647d INFO: 172.20.0.1:59304 - "POST /node/69c8bae5f41e4fb6a378647d HTTP/1.0" 202 Accepted 2026-03-31 05:38:56,475 - lava_callback - INFO - Starting async job submit for node 69c8bae5f41e4fb6a378647d 2026-03-31 05:38:56,476 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:38:56,490 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8bae5f41e4fb6a378647d HTTP/1.1" 200 None 2026-03-31 05:38:56,491 - lava_callback - DEBUG - Processing node 69c8bae5f41e4fb6a378647d: baseline-arm-baylibre 2026-03-31 05:38:56,492 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8bae5f41e4fb6a378647d 2026-03-31 05:38:56,492 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.20.0.1:59308 - "GET / HTTP/1.0" 200 OK 2026-03-31 05:38:56,506 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8bae5f41e4fb6a378647d HTTP/1.1" 200 None 2026-03-31 05:38:56,507 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:38:56,521 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c55c70abf66a8f4d9b9655 HTTP/1.1" 200 None 2026-03-31 05:38:56,523 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:38:56,553 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8bae5f41e4fb6a378647d HTTP/1.1" 200 None 2026-03-31 05:38:56,554 - lava_callback - ERROR - Error processing callback for node 69c8bae5f41e4fb6a378647d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:42032 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44882 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44892 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45492 - "GET / HTTP/1.0" 200 OK 2026-03-31 05:39:36,423 - lava_callback - INFO - Received callback for node 69c8bb0af41e4fb6a3786489 INFO: 172.20.0.1:54860 - "POST /node/69c8bb0af41e4fb6a3786489 HTTP/1.0" 202 Accepted 2026-03-31 05:39:36,428 - lava_callback - INFO - Starting async job submit for node 69c8bb0af41e4fb6a3786489 2026-03-31 05:39:36,429 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:39:36,443 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8bb0af41e4fb6a3786489 HTTP/1.1" 200 None 2026-03-31 05:39:36,444 - lava_callback - DEBUG - Processing node 69c8bb0af41e4fb6a3786489: baseline-arm-baylibre 2026-03-31 05:39:36,444 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8bb0af41e4fb6a3786489 2026-03-31 05:39:36,445 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:39:36,458 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8bb0af41e4fb6a3786489 HTTP/1.1" 200 None 2026-03-31 05:39:36,460 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:39:36,482 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c55c70abf66a8f4d9b9655 HTTP/1.1" 200 None 2026-03-31 05:39:36,483 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:39:36,508 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8bb0af41e4fb6a3786489 HTTP/1.1" 200 None 2026-03-31 05:39:36,510 - lava_callback - ERROR - Error processing callback for node 69c8bb0af41e4fb6a3786489: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:54864 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35788 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35802 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45528 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40678 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38790 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38792 - "GET / HTTP/1.0" 200 OK 2026-03-31 05:40:16,521 - lava_callback - INFO - Received callback for node 69c8bb2ef41e4fb6a3786499 INFO: 172.20.0.1:38804 - "POST /node/69c8bb2ef41e4fb6a3786499 HTTP/1.0" 202 Accepted 2026-03-31 05:40:16,526 - lava_callback - INFO - Starting async job submit for node 69c8bb2ef41e4fb6a3786499 2026-03-31 05:40:16,527 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:40:16,547 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8bb2ef41e4fb6a3786499 HTTP/1.1" 200 None 2026-03-31 05:40:16,548 - lava_callback - DEBUG - Processing node 69c8bb2ef41e4fb6a3786499: baseline-arm-baylibre 2026-03-31 05:40:16,548 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8bb2ef41e4fb6a3786499 2026-03-31 05:40:16,548 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:40:16,567 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8bb2ef41e4fb6a3786499 HTTP/1.1" 200 None 2026-03-31 05:40:16,568 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:40:16,581 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c55c70abf66a8f4d9b9655 HTTP/1.1" 200 None 2026-03-31 05:40:16,583 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:40:16,603 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8bb2ef41e4fb6a3786499 HTTP/1.1" 200 None 2026-03-31 05:40:16,610 - lava_callback - ERROR - Error processing callback for node 69c8bb2ef41e4fb6a3786499: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:40728 - "GET / HTTP/1.0" 200 OK 2026-03-31 05:40:36,480 - lava_callback - INFO - Received callback for node 69c8bb51f41e4fb6a37864a1 INFO: 172.20.0.1:46676 - "POST /node/69c8bb51f41e4fb6a37864a1 HTTP/1.0" 202 Accepted 2026-03-31 05:40:36,485 - lava_callback - INFO - Starting async job submit for node 69c8bb51f41e4fb6a37864a1 2026-03-31 05:40:36,486 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:40:36,502 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8bb51f41e4fb6a37864a1 HTTP/1.1" 200 None 2026-03-31 05:40:36,503 - lava_callback - DEBUG - Processing node 69c8bb51f41e4fb6a37864a1: baseline-arm-baylibre 2026-03-31 05:40:36,503 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8bb51f41e4fb6a37864a1 2026-03-31 05:40:36,503 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.20.0.1:46682 - "GET / HTTP/1.0" 200 OK 2026-03-31 05:40:36,518 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8bb51f41e4fb6a37864a1 HTTP/1.1" 200 None 2026-03-31 05:40:36,520 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:40:36,544 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c55c70abf66a8f4d9b9655 HTTP/1.1" 200 None 2026-03-31 05:40:36,545 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:40:36,565 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8bb51f41e4fb6a37864a1 HTTP/1.1" 200 None 2026-03-31 05:40:36,573 - lava_callback - ERROR - Error processing callback for node 69c8bb51f41e4fb6a37864a1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:44888 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44894 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51900 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55072 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49100 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-31 05:41:16,504 - lava_callback - INFO - Received callback for node 69c8bb77f41e4fb6a37864af INFO: 172.20.0.1:49114 - "POST /node/69c8bb77f41e4fb6a37864af HTTP/1.0" 202 Accepted 2026-03-31 05:41:16,509 - lava_callback - INFO - Starting async job submit for node 69c8bb77f41e4fb6a37864af 2026-03-31 05:41:16,510 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:41:16,524 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8bb77f41e4fb6a37864af HTTP/1.1" 200 None 2026-03-31 05:41:16,525 - lava_callback - DEBUG - Processing node 69c8bb77f41e4fb6a37864af: baseline-arm-baylibre 2026-03-31 05:41:16,525 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8bb77f41e4fb6a37864af 2026-03-31 05:41:16,526 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.20.0.1:49130 - "GET / HTTP/1.0" 200 OK 2026-03-31 05:41:16,540 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8bb77f41e4fb6a37864af HTTP/1.1" 200 None 2026-03-31 05:41:16,542 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:41:16,555 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c55c70abf66a8f4d9b9655 HTTP/1.1" 200 None 2026-03-31 05:41:16,557 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:41:16,587 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8bb77f41e4fb6a37864af HTTP/1.1" 200 None 2026-03-31 05:41:16,596 - lava_callback - ERROR - Error processing callback for node 69c8bb77f41e4fb6a37864af: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:33118 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52306 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56938 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56948 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41286 - "GET / HTTP/1.0" 200 OK 2026-03-31 05:41:56,587 - lava_callback - INFO - Received callback for node 69c8bb9af41e4fb6a37864ba INFO: 172.20.0.1:41296 - "POST /node/69c8bb9af41e4fb6a37864ba HTTP/1.0" 202 Accepted 2026-03-31 05:41:56,593 - lava_callback - INFO - Starting async job submit for node 69c8bb9af41e4fb6a37864ba 2026-03-31 05:41:56,594 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:41:56,608 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8bb9af41e4fb6a37864ba HTTP/1.1" 200 None 2026-03-31 05:41:56,609 - lava_callback - DEBUG - Processing node 69c8bb9af41e4fb6a37864ba: baseline-arm-baylibre 2026-03-31 05:41:56,609 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8bb9af41e4fb6a37864ba 2026-03-31 05:41:56,609 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:41:56,623 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8bb9af41e4fb6a37864ba HTTP/1.1" 200 None 2026-03-31 05:41:56,624 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:41:56,637 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c55c70abf66a8f4d9b9655 HTTP/1.1" 200 None 2026-03-31 05:41:56,639 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:41:56,663 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8bb9af41e4fb6a37864ba HTTP/1.1" 200 None 2026-03-31 05:41:56,665 - lava_callback - ERROR - Error processing callback for node 69c8bb9af41e4fb6a37864ba: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:56428 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47770 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47784 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33486 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50938 - "GET / HTTP/1.0" 200 OK 2026-03-31 05:42:39,024 - lava_callback - INFO - Received callback for node 69c8bbc4f41e4fb6a37864c9 INFO: 172.20.0.1:50950 - "POST /node/69c8bbc4f41e4fb6a37864c9 HTTP/1.0" 202 Accepted 2026-03-31 05:42:39,030 - lava_callback - INFO - Starting async job submit for node 69c8bbc4f41e4fb6a37864c9 2026-03-31 05:42:39,031 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:42:39,059 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8bbc4f41e4fb6a37864c9 HTTP/1.1" 200 None 2026-03-31 05:42:39,060 - lava_callback - DEBUG - Processing node 69c8bbc4f41e4fb6a37864c9: baseline-arm-clabbe 2026-03-31 05:42:39,060 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8bbc4f41e4fb6a37864c9 2026-03-31 05:42:39,060 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:42:39,074 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8bbc4f41e4fb6a37864c9 HTTP/1.1" 200 None 2026-03-31 05:42:39,076 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:42:39,089 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c55c70abf66a8f4d9b9655 HTTP/1.1" 200 None 2026-03-31 05:42:39,091 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:42:39,125 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8bbc4f41e4fb6a37864c9 HTTP/1.1" 200 None 2026-03-31 05:42:39,127 - lava_callback - ERROR - Error processing callback for node 69c8bbc4f41e4fb6a37864c9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:51160 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51168 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39846 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50212 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38272 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38286 - "GET / HTTP/1.0" 200 OK 2026-03-31 05:43:16,645 - lava_callback - INFO - Received callback for node 69c8bbe5f41e4fb6a37864d3 INFO: 172.20.0.1:38298 - "POST /node/69c8bbe5f41e4fb6a37864d3 HTTP/1.0" 202 Accepted 2026-03-31 05:43:16,651 - lava_callback - INFO - Starting async job submit for node 69c8bbe5f41e4fb6a37864d3 2026-03-31 05:43:16,652 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:43:16,666 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8bbe5f41e4fb6a37864d3 HTTP/1.1" 200 None 2026-03-31 05:43:16,667 - lava_callback - DEBUG - Processing node 69c8bbe5f41e4fb6a37864d3: baseline-arm-baylibre 2026-03-31 05:43:16,667 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8bbe5f41e4fb6a37864d3 2026-03-31 05:43:16,668 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:43:16,681 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8bbe5f41e4fb6a37864d3 HTTP/1.1" 200 None 2026-03-31 05:43:16,682 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:43:16,695 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c55c70abf66a8f4d9b9655 HTTP/1.1" 200 None 2026-03-31 05:43:16,697 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:43:16,722 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8bbe5f41e4fb6a37864d3 HTTP/1.1" 200 None 2026-03-31 05:43:16,723 - lava_callback - ERROR - Error processing callback for node 69c8bbe5f41e4fb6a37864d3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:56334 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59364 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53254 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53258 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38474 - "GET / HTTP/1.0" 200 OK 2026-03-31 05:43:56,686 - lava_callback - INFO - Received callback for node 69c8bc0bf41e4fb6a37864df INFO: 172.20.0.1:38476 - "POST /node/69c8bc0bf41e4fb6a37864df HTTP/1.0" 202 Accepted 2026-03-31 05:43:56,692 - lava_callback - INFO - Starting async job submit for node 69c8bc0bf41e4fb6a37864df 2026-03-31 05:43:56,693 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:43:56,709 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8bc0bf41e4fb6a37864df HTTP/1.1" 200 None 2026-03-31 05:43:56,710 - lava_callback - DEBUG - Processing node 69c8bc0bf41e4fb6a37864df: baseline-arm-baylibre 2026-03-31 05:43:56,710 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8bc0bf41e4fb6a37864df 2026-03-31 05:43:56,711 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:43:56,727 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8bc0bf41e4fb6a37864df HTTP/1.1" 200 None 2026-03-31 05:43:56,728 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:43:56,741 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c55c70abf66a8f4d9b9655 HTTP/1.1" 200 None 2026-03-31 05:43:56,743 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:43:56,763 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8bc0bf41e4fb6a37864df HTTP/1.1" 200 None 2026-03-31 05:43:56,768 - lava_callback - ERROR - Error processing callback for node 69c8bc0bf41e4fb6a37864df: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:60892 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45508 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45520 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54052 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40412 - "GET / HTTP/1.0" 200 OK 2026-03-31 05:44:39,115 - lava_callback - INFO - Received callback for node 69c8bc35f41e4fb6a37864ee INFO: 172.20.0.1:40422 - "POST /node/69c8bc35f41e4fb6a37864ee HTTP/1.0" 202 Accepted 2026-03-31 05:44:39,121 - lava_callback - INFO - Starting async job submit for node 69c8bc35f41e4fb6a37864ee 2026-03-31 05:44:39,122 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:44:39,143 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8bc35f41e4fb6a37864ee HTTP/1.1" 200 None 2026-03-31 05:44:39,145 - lava_callback - DEBUG - Processing node 69c8bc35f41e4fb6a37864ee: baseline-arm-clabbe 2026-03-31 05:44:39,145 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8bc35f41e4fb6a37864ee 2026-03-31 05:44:39,146 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:44:39,167 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8bc35f41e4fb6a37864ee HTTP/1.1" 200 None 2026-03-31 05:44:39,169 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:44:39,183 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c55c70abf66a8f4d9b9655 HTTP/1.1" 200 None 2026-03-31 05:44:39,185 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:44:39,209 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8bc35f41e4fb6a37864ee HTTP/1.1" 200 None 2026-03-31 05:44:39,210 - lava_callback - ERROR - Error processing callback for node 69c8bc35f41e4fb6a37864ee: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:48778 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48782 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43110 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53472 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36954 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36970 - "GET / HTTP/1.0" 200 OK 2026-03-31 05:45:19,078 - lava_callback - INFO - Received callback for node 69c8bc5bf41e4fb6a37864fe INFO: 172.20.0.1:36976 - "POST /node/69c8bc5bf41e4fb6a37864fe HTTP/1.0" 202 Accepted 2026-03-31 05:45:19,083 - lava_callback - INFO - Starting async job submit for node 69c8bc5bf41e4fb6a37864fe 2026-03-31 05:45:19,084 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:45:19,099 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8bc5bf41e4fb6a37864fe HTTP/1.1" 200 None 2026-03-31 05:45:19,100 - lava_callback - DEBUG - Processing node 69c8bc5bf41e4fb6a37864fe: baseline-arm-clabbe 2026-03-31 05:45:19,100 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8bc5bf41e4fb6a37864fe 2026-03-31 05:45:19,101 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:45:19,119 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8bc5bf41e4fb6a37864fe HTTP/1.1" 200 None 2026-03-31 05:45:19,121 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:45:19,136 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c55c70abf66a8f4d9b9655 HTTP/1.1" 200 None 2026-03-31 05:45:19,138 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:45:19,165 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8bc5bf41e4fb6a37864fe HTTP/1.1" 200 None 2026-03-31 05:45:19,166 - lava_callback - ERROR - Error processing callback for node 69c8bc5bf41e4fb6a37864fe: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:39636 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44102 - "GET / HTTP/1.0" 200 OK 2026-03-31 05:45:39,147 - lava_callback - INFO - Received callback for node 69c8bc80f41e4fb6a3786509 INFO: 172.20.0.1:44116 - "POST /node/69c8bc80f41e4fb6a3786509 HTTP/1.0" 202 Accepted 2026-03-31 05:45:39,153 - lava_callback - INFO - Starting async job submit for node 69c8bc80f41e4fb6a3786509 2026-03-31 05:45:39,154 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:45:39,169 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8bc80f41e4fb6a3786509 HTTP/1.1" 200 None 2026-03-31 05:45:39,170 - lava_callback - DEBUG - Processing node 69c8bc80f41e4fb6a3786509: baseline-arm-clabbe 2026-03-31 05:45:39,170 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8bc80f41e4fb6a3786509 2026-03-31 05:45:39,171 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:45:39,186 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8bc80f41e4fb6a3786509 HTTP/1.1" 200 None 2026-03-31 05:45:39,187 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:45:39,202 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c55c70abf66a8f4d9b9655 HTTP/1.1" 200 None 2026-03-31 05:45:39,204 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:45:39,231 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8bc80f41e4fb6a3786509 HTTP/1.1" 200 None 2026-03-31 05:45:39,233 - lava_callback - ERROR - Error processing callback for node 69c8bc80f41e4fb6a3786509: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:60338 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60342 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49002 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37494 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41724 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41736 - "GET / HTTP/1.0" 200 OK 2026-03-31 05:46:19,252 - lava_callback - INFO - Received callback for node 69c8bca5f41e4fb6a3786517 INFO: 172.20.0.1:41742 - "POST /node/69c8bca5f41e4fb6a3786517 HTTP/1.0" 202 Accepted 2026-03-31 05:46:19,258 - lava_callback - INFO - Starting async job submit for node 69c8bca5f41e4fb6a3786517 2026-03-31 05:46:19,258 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:46:19,273 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8bca5f41e4fb6a3786517 HTTP/1.1" 200 None 2026-03-31 05:46:19,274 - lava_callback - DEBUG - Processing node 69c8bca5f41e4fb6a3786517: baseline-arm-clabbe 2026-03-31 05:46:19,275 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8bca5f41e4fb6a3786517 2026-03-31 05:46:19,275 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:46:19,288 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8bca5f41e4fb6a3786517 HTTP/1.1" 200 None 2026-03-31 05:46:19,290 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:46:19,309 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c55c70abf66a8f4d9b9655 HTTP/1.1" 200 None 2026-03-31 05:46:19,311 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:46:19,340 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8bca5f41e4fb6a3786517 HTTP/1.1" 200 None 2026-03-31 05:46:19,343 - lava_callback - ERROR - Error processing callback for node 69c8bca5f41e4fb6a3786517: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:50164 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34364 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41434 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41436 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33688 - "GET / HTTP/1.0" 200 OK 2026-03-31 05:46:56,968 - lava_callback - INFO - Received callback for node 69c8bcc6f41e4fb6a3786526 INFO: 172.20.0.1:33698 - "POST /node/69c8bcc6f41e4fb6a3786526 HTTP/1.0" 202 Accepted 2026-03-31 05:46:56,973 - lava_callback - INFO - Starting async job submit for node 69c8bcc6f41e4fb6a3786526 2026-03-31 05:46:56,974 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:46:56,988 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8bcc6f41e4fb6a3786526 HTTP/1.1" 200 None 2026-03-31 05:46:56,989 - lava_callback - DEBUG - Processing node 69c8bcc6f41e4fb6a3786526: baseline-arm-baylibre 2026-03-31 05:46:56,989 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8bcc6f41e4fb6a3786526 2026-03-31 05:46:56,990 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:46:57,003 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8bcc6f41e4fb6a3786526 HTTP/1.1" 200 None 2026-03-31 05:46:57,005 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:46:57,021 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c55c70abf66a8f4d9b9655 HTTP/1.1" 200 None 2026-03-31 05:46:57,023 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:46:57,043 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8bcc6f41e4fb6a3786526 HTTP/1.1" 200 None 2026-03-31 05:46:57,044 - lava_callback - ERROR - Error processing callback for node 69c8bcc6f41e4fb6a3786526: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:39312 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46472 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46486 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58820 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53430 - "GET / HTTP/1.0" 200 OK 2026-03-31 05:47:39,208 - lava_callback - INFO - Received callback for node 69c8bceff41e4fb6a3786539 INFO: 172.20.0.1:53432 - "POST /node/69c8bceff41e4fb6a3786539 HTTP/1.0" 202 Accepted 2026-03-31 05:47:39,213 - lava_callback - INFO - Starting async job submit for node 69c8bceff41e4fb6a3786539 2026-03-31 05:47:39,214 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:47:39,228 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8bceff41e4fb6a3786539 HTTP/1.1" 200 None 2026-03-31 05:47:39,229 - lava_callback - DEBUG - Processing node 69c8bceff41e4fb6a3786539: baseline-arm-clabbe 2026-03-31 05:47:39,229 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8bceff41e4fb6a3786539 2026-03-31 05:47:39,230 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:47:39,244 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8bceff41e4fb6a3786539 HTTP/1.1" 200 None 2026-03-31 05:47:39,245 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:47:39,258 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c55c70abf66a8f4d9b9655 HTTP/1.1" 200 None 2026-03-31 05:47:39,259 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:47:39,284 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8bceff41e4fb6a3786539 HTTP/1.1" 200 None 2026-03-31 05:47:39,286 - lava_callback - ERROR - Error processing callback for node 69c8bceff41e4fb6a3786539: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:39898 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39902 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54914 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34700 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41768 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41784 - "GET / HTTP/1.0" 200 OK 2026-03-31 05:48:19,200 - lava_callback - INFO - Received callback for node 69c8bd16f41e4fb6a3786542 INFO: 172.20.0.1:41790 - "POST /node/69c8bd16f41e4fb6a3786542 HTTP/1.0" 202 Accepted 2026-03-31 05:48:19,205 - lava_callback - INFO - Starting async job submit for node 69c8bd16f41e4fb6a3786542 2026-03-31 05:48:19,206 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:48:19,220 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8bd16f41e4fb6a3786542 HTTP/1.1" 200 None 2026-03-31 05:48:19,220 - lava_callback - DEBUG - Processing node 69c8bd16f41e4fb6a3786542: baseline-arm-clabbe 2026-03-31 05:48:19,221 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8bd16f41e4fb6a3786542 2026-03-31 05:48:19,221 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:48:19,235 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8bd16f41e4fb6a3786542 HTTP/1.1" 200 None 2026-03-31 05:48:19,236 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:48:19,249 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c55c70abf66a8f4d9b9655 HTTP/1.1" 200 None 2026-03-31 05:48:19,251 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:48:19,271 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8bd16f41e4fb6a3786542 HTTP/1.1" 200 None 2026-03-31 05:48:19,272 - lava_callback - ERROR - Error processing callback for node 69c8bd16f41e4fb6a3786542: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:34490 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53960 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49618 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49622 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33822 - "GET / HTTP/1.0" 200 OK 2026-03-31 05:48:59,238 - lava_callback - INFO - Received callback for node 69c8bd3bf41e4fb6a378654f INFO: 172.20.0.1:33830 - "POST /node/69c8bd3bf41e4fb6a378654f HTTP/1.0" 202 Accepted 2026-03-31 05:48:59,244 - lava_callback - INFO - Starting async job submit for node 69c8bd3bf41e4fb6a378654f 2026-03-31 05:48:59,245 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:48:59,259 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8bd3bf41e4fb6a378654f HTTP/1.1" 200 None 2026-03-31 05:48:59,260 - lava_callback - DEBUG - Processing node 69c8bd3bf41e4fb6a378654f: baseline-arm-clabbe 2026-03-31 05:48:59,260 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8bd3bf41e4fb6a378654f 2026-03-31 05:48:59,260 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:48:59,273 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8bd3bf41e4fb6a378654f HTTP/1.1" 200 None 2026-03-31 05:48:59,274 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:48:59,298 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c55c70abf66a8f4d9b9655 HTTP/1.1" 200 None 2026-03-31 05:48:59,299 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:48:59,322 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8bd3bf41e4fb6a378654f HTTP/1.1" 200 None 2026-03-31 05:48:59,324 - lava_callback - ERROR - Error processing callback for node 69c8bd3bf41e4fb6a378654f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:53918 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41254 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41258 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54986 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42590 - "GET / HTTP/1.0" 200 OK 2026-03-31 05:49:37,002 - lava_callback - INFO - Received callback for node 69c8bd5ff41e4fb6a378655e INFO: 172.20.0.1:42606 - "POST /node/69c8bd5ff41e4fb6a378655e HTTP/1.0" 202 Accepted 2026-03-31 05:49:37,008 - lava_callback - INFO - Starting async job submit for node 69c8bd5ff41e4fb6a378655e 2026-03-31 05:49:37,009 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:49:37,023 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8bd5ff41e4fb6a378655e HTTP/1.1" 200 None 2026-03-31 05:49:37,024 - lava_callback - DEBUG - Processing node 69c8bd5ff41e4fb6a378655e: baseline-arm-baylibre 2026-03-31 05:49:37,024 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8bd5ff41e4fb6a378655e 2026-03-31 05:49:37,025 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:49:37,038 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8bd5ff41e4fb6a378655e HTTP/1.1" 200 None 2026-03-31 05:49:37,040 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:49:37,053 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c55c70abf66a8f4d9b9655 HTTP/1.1" 200 None 2026-03-31 05:49:37,055 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:49:37,075 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8bd5ff41e4fb6a378655e HTTP/1.1" 200 None 2026-03-31 05:49:37,083 - lava_callback - ERROR - Error processing callback for node 69c8bd5ff41e4fb6a378655e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:59264 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59274 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52500 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34318 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33222 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33232 - "GET / HTTP/1.0" 200 OK 2026-03-31 05:50:19,292 - lava_callback - INFO - Received callback for node 69c8bd86f41e4fb6a378656e INFO: 172.20.0.1:33238 - "POST /node/69c8bd86f41e4fb6a378656e HTTP/1.0" 202 Accepted 2026-03-31 05:50:19,297 - lava_callback - INFO - Starting async job submit for node 69c8bd86f41e4fb6a378656e 2026-03-31 05:50:19,298 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:50:19,312 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8bd86f41e4fb6a378656e HTTP/1.1" 200 None 2026-03-31 05:50:19,313 - lava_callback - DEBUG - Processing node 69c8bd86f41e4fb6a378656e: baseline-arm-clabbe 2026-03-31 05:50:19,313 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8bd86f41e4fb6a378656e 2026-03-31 05:50:19,314 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:50:19,327 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8bd86f41e4fb6a378656e HTTP/1.1" 200 None 2026-03-31 05:50:19,329 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:50:19,342 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c55c70abf66a8f4d9b9655 HTTP/1.1" 200 None 2026-03-31 05:50:19,344 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:50:19,373 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8bd86f41e4fb6a378656e HTTP/1.1" 200 None 2026-03-31 05:50:19,374 - lava_callback - ERROR - Error processing callback for node 69c8bd86f41e4fb6a378656e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:33336 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36208 - "GET / HTTP/1.0" 200 OK 2026-03-31 05:50:36,977 - lava_callback - INFO - Received callback for node 69c8bda9f41e4fb6a3786576 INFO: 172.20.0.1:36224 - "POST /node/69c8bda9f41e4fb6a3786576 HTTP/1.0" 202 Accepted 2026-03-31 05:50:36,983 - lava_callback - INFO - Starting async job submit for node 69c8bda9f41e4fb6a3786576 2026-03-31 05:50:36,984 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:50:37,009 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8bda9f41e4fb6a3786576 HTTP/1.1" 200 None 2026-03-31 05:50:37,010 - lava_callback - DEBUG - Processing node 69c8bda9f41e4fb6a3786576: baseline-arm-baylibre 2026-03-31 05:50:37,010 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8bda9f41e4fb6a3786576 2026-03-31 05:50:37,011 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:50:37,026 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8bda9f41e4fb6a3786576 HTTP/1.1" 200 None 2026-03-31 05:50:37,028 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:50:37,045 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c55c70abf66a8f4d9b9655 HTTP/1.1" 200 None 2026-03-31 05:50:37,047 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:50:37,074 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8bda9f41e4fb6a3786576 HTTP/1.1" 200 None 2026-03-31 05:50:37,076 - lava_callback - ERROR - Error processing callback for node 69c8bda9f41e4fb6a3786576: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:37066 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37082 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60676 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56028 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42010 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42016 - "GET / HTTP/1.0" 200 OK 2026-03-31 05:51:19,370 - lava_callback - INFO - Received callback for node 69c8bdcef41e4fb6a3786583 INFO: 172.20.0.1:42030 - "POST /node/69c8bdcef41e4fb6a3786583 HTTP/1.0" 202 Accepted 2026-03-31 05:51:19,376 - lava_callback - INFO - Starting async job submit for node 69c8bdcef41e4fb6a3786583 2026-03-31 05:51:19,377 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:51:19,407 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8bdcef41e4fb6a3786583 HTTP/1.1" 200 None 2026-03-31 05:51:19,408 - lava_callback - DEBUG - Processing node 69c8bdcef41e4fb6a3786583: baseline-arm-clabbe 2026-03-31 05:51:19,408 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8bdcef41e4fb6a3786583 2026-03-31 05:51:19,408 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:51:19,432 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8bdcef41e4fb6a3786583 HTTP/1.1" 200 None 2026-03-31 05:51:19,433 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:51:19,450 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c55c70abf66a8f4d9b9655 HTTP/1.1" 200 None 2026-03-31 05:51:19,452 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:51:19,472 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8bdcef41e4fb6a3786583 HTTP/1.1" 200 None 2026-03-31 05:51:19,474 - lava_callback - ERROR - Error processing callback for node 69c8bdcef41e4fb6a3786583: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:50672 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41090 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43026 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43042 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47846 - "GET / HTTP/1.0" 200 OK 2026-03-31 05:51:59,344 - lava_callback - INFO - Received callback for node 69c8bdf2f41e4fb6a378658f INFO: 172.20.0.1:47858 - "POST /node/69c8bdf2f41e4fb6a378658f HTTP/1.0" 202 Accepted 2026-03-31 05:51:59,350 - lava_callback - INFO - Starting async job submit for node 69c8bdf2f41e4fb6a378658f 2026-03-31 05:51:59,350 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:51:59,364 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8bdf2f41e4fb6a378658f HTTP/1.1" 200 None 2026-03-31 05:51:59,365 - lava_callback - DEBUG - Processing node 69c8bdf2f41e4fb6a378658f: baseline-arm-clabbe 2026-03-31 05:51:59,365 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8bdf2f41e4fb6a378658f 2026-03-31 05:51:59,366 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:51:59,378 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8bdf2f41e4fb6a378658f HTTP/1.1" 200 None 2026-03-31 05:51:59,380 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:51:59,392 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c55c70abf66a8f4d9b9655 HTTP/1.1" 200 None 2026-03-31 05:51:59,394 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:51:59,419 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8bdf2f41e4fb6a378658f HTTP/1.1" 200 None 2026-03-31 05:51:59,421 - lava_callback - ERROR - Error processing callback for node 69c8bdf2f41e4fb6a378658f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:55408 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60744 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60756 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50580 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42442 - "GET / HTTP/1.0" 200 OK 2026-03-31 05:52:39,374 - lava_callback - INFO - Received callback for node 69c8be16f41e4fb6a378659c INFO: 172.20.0.1:42450 - "POST /node/69c8be16f41e4fb6a378659c HTTP/1.0" 202 Accepted 2026-03-31 05:52:39,380 - lava_callback - INFO - Starting async job submit for node 69c8be16f41e4fb6a378659c 2026-03-31 05:52:39,380 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:52:39,394 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8be16f41e4fb6a378659c HTTP/1.1" 200 None 2026-03-31 05:52:39,395 - lava_callback - DEBUG - Processing node 69c8be16f41e4fb6a378659c: baseline-arm-clabbe 2026-03-31 05:52:39,395 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8be16f41e4fb6a378659c 2026-03-31 05:52:39,396 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:52:39,413 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8be16f41e4fb6a378659c HTTP/1.1" 200 None 2026-03-31 05:52:39,414 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:52:39,428 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c55c70abf66a8f4d9b9655 HTTP/1.1" 200 None 2026-03-31 05:52:39,429 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:52:39,449 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8be16f41e4fb6a378659c HTTP/1.1" 200 None 2026-03-31 05:52:39,455 - lava_callback - ERROR - Error processing callback for node 69c8be16f41e4fb6a378659c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:45114 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45116 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53592 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45892 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50518 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50520 - "GET / HTTP/1.0" 200 OK 2026-03-31 05:53:19,356 - lava_callback - INFO - Received callback for node 69c8be3bf41e4fb6a37865aa INFO: 172.20.0.1:50534 - "POST /node/69c8be3bf41e4fb6a37865aa HTTP/1.0" 202 Accepted 2026-03-31 05:53:19,362 - lava_callback - INFO - Starting async job submit for node 69c8be3bf41e4fb6a37865aa 2026-03-31 05:53:19,362 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:53:19,377 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8be3bf41e4fb6a37865aa HTTP/1.1" 200 None 2026-03-31 05:53:19,377 - lava_callback - DEBUG - Processing node 69c8be3bf41e4fb6a37865aa: baseline-arm-clabbe 2026-03-31 05:53:19,378 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8be3bf41e4fb6a37865aa 2026-03-31 05:53:19,378 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:53:19,391 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8be3bf41e4fb6a37865aa HTTP/1.1" 200 None 2026-03-31 05:53:19,392 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:53:19,405 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c55c70abf66a8f4d9b9655 HTTP/1.1" 200 None 2026-03-31 05:53:19,407 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:53:19,437 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8be3bf41e4fb6a37865aa HTTP/1.1" 200 None 2026-03-31 05:53:19,438 - lava_callback - ERROR - Error processing callback for node 69c8be3bf41e4fb6a37865aa: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:48956 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45364 - "GET / HTTP/1.0" 200 OK 2026-03-31 05:53:39,424 - lava_callback - INFO - Received callback for node 69c8be5ff41e4fb6a37865b5 INFO: 172.20.0.1:45374 - "POST /node/69c8be5ff41e4fb6a37865b5 HTTP/1.0" 202 Accepted 2026-03-31 05:53:39,430 - lava_callback - INFO - Starting async job submit for node 69c8be5ff41e4fb6a37865b5 2026-03-31 05:53:39,430 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:53:39,445 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8be5ff41e4fb6a37865b5 HTTP/1.1" 200 None 2026-03-31 05:53:39,446 - lava_callback - DEBUG - Processing node 69c8be5ff41e4fb6a37865b5: baseline-arm-clabbe 2026-03-31 05:53:39,446 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8be5ff41e4fb6a37865b5 2026-03-31 05:53:39,447 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:53:39,463 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8be5ff41e4fb6a37865b5 HTTP/1.1" 200 None 2026-03-31 05:53:39,464 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:53:39,479 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c55c70abf66a8f4d9b9655 HTTP/1.1" 200 None 2026-03-31 05:53:39,481 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:53:39,508 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8be5ff41e4fb6a37865b5 HTTP/1.1" 200 None 2026-03-31 05:53:39,510 - lava_callback - ERROR - Error processing callback for node 69c8be5ff41e4fb6a37865b5: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:35796 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35808 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57346 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39688 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41366 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41378 - "GET / HTTP/1.0" 200 OK 2026-03-31 05:54:19,377 - lava_callback - INFO - Received callback for node 69c8be84f41e4fb6a37865c2 INFO: 172.20.0.1:41388 - "POST /node/69c8be84f41e4fb6a37865c2 HTTP/1.0" 202 Accepted 2026-03-31 05:54:19,383 - lava_callback - INFO - Starting async job submit for node 69c8be84f41e4fb6a37865c2 2026-03-31 05:54:19,384 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:54:19,398 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8be84f41e4fb6a37865c2 HTTP/1.1" 200 None 2026-03-31 05:54:19,398 - lava_callback - DEBUG - Processing node 69c8be84f41e4fb6a37865c2: baseline-arm-clabbe 2026-03-31 05:54:19,399 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8be84f41e4fb6a37865c2 2026-03-31 05:54:19,399 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:54:19,412 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8be84f41e4fb6a37865c2 HTTP/1.1" 200 None 2026-03-31 05:54:19,414 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:54:19,427 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c55c70abf66a8f4d9b9655 HTTP/1.1" 200 None 2026-03-31 05:54:19,429 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 05:54:19,449 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8be84f41e4fb6a37865c2 HTTP/1.1" 200 None 2026-03-31 05:54:19,451 - lava_callback - ERROR - Error processing callback for node 69c8be84f41e4fb6a37865c2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:59134 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51582 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39066 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39080 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55254 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56740 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43406 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43416 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41678 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53300 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54098 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54106 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57214 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37146 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47650 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47660 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42238 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38592 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57028 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57034 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32816 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38416 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45860 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45864 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40770 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47898 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54084 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54096 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38024 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36922 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42142 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42144 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42366 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57380 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37624 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37628 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51120 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53010 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57430 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57444 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56552 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59182 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42140 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42154 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52660 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44468 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40192 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40198 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43826 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60540 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55470 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55476 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39124 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36154 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37762 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37772 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36486 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48726 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40472 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40474 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37448 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55130 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37616 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37618 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45522 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38404 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39524 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39534 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54408 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40566 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45228 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45234 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40890 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52406 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38480 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38496 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54848 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55742 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39070 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39086 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36690 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43252 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36196 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36198 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45650 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44074 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50750 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50754 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35936 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39150 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55078 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55084 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33256 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47704 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51020 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51026 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42946 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35230 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41926 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41938 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47330 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42050 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39624 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39630 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42444 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39760 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38378 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38384 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35732 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51846 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51328 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51344 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52860 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40508 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44428 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44432 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35948 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52666 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56662 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56664 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39992 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58002 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58272 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58284 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53614 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57902 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57404 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57408 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40632 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56528 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59180 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59188 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52804 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37936 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55100 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55112 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47792 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59000 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52142 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52148 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53702 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35732 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41840 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41852 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55860 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41566 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60920 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60930 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54466 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38558 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57352 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57366 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50240 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33054 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55356 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55370 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40766 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41020 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51778 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51780 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58098 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38224 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36020 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36030 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60436 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32846 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43432 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43442 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55380 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52504 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52952 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52956 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57368 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43274 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43416 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43418 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59598 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58754 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44260 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44264 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45074 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51010 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39450 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39460 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47064 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43576 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33048 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33054 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56914 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56418 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44982 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44988 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45568 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46710 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54464 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54470 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50782 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40166 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52528 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52542 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41036 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38598 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42184 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42190 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57466 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45032 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57764 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57770 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37550 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54630 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43476 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43488 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58408 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56386 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37338 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37352 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51834 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57606 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44104 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44106 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58592 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42022 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51368 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51384 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45344 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42044 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44774 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44782 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35054 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45290 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36646 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36656 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51534 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42204 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40326 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40336 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60082 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51254 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37810 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37826 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37830 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38558 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60774 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60788 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42472 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49700 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34382 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34390 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45322 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54350 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41642 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41654 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34480 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60642 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51280 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51282 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47162 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43762 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39976 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39992 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47744 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45436 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59640 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59646 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33540 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35610 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60008 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60024 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39384 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34338 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36396 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36412 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50560 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58294 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51042 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51058 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55920 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43674 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37466 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37480 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34288 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47978 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49370 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49372 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36164 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43538 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38156 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38172 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45198 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35206 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49688 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49698 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51404 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43730 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57806 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57808 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38468 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35052 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35612 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35622 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47962 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58616 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45916 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45930 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47900 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37760 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49006 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49016 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35076 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44508 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37566 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37580 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40826 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37610 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33736 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33752 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53848 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40052 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53398 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53408 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60770 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52526 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59136 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59144 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55736 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39974 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43836 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43848 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51120 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54994 - "GET / HTTP/1.0" 200 OK 2026-03-31 06:35:39,656 - lava_callback - INFO - Received callback for node 69c8c833f41e4fb6a3786618 INFO: 172.20.0.1:54998 - "POST /node/69c8c833f41e4fb6a3786618 HTTP/1.0" 202 Accepted 2026-03-31 06:35:39,661 - lava_callback - INFO - Starting async job submit for node 69c8c833f41e4fb6a3786618 2026-03-31 06:35:39,662 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 06:35:39,676 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8c833f41e4fb6a3786618 HTTP/1.1" 200 None 2026-03-31 06:35:39,677 - lava_callback - DEBUG - Processing node 69c8c833f41e4fb6a3786618: baseline-arm-baylibre 2026-03-31 06:35:39,678 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8c833f41e4fb6a3786618 2026-03-31 06:35:39,678 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 06:35:39,691 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8c833f41e4fb6a3786618 HTTP/1.1" 200 None 2026-03-31 06:35:39,692 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 06:35:39,714 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-03-31 06:35:39,716 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 06:35:39,740 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8c833f41e4fb6a3786618 HTTP/1.1" 200 None 2026-03-31 06:35:39,742 - lava_callback - ERROR - Error processing callback for node 69c8c833f41e4fb6a3786618: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:58562 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58574 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50686 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57734 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45650 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45654 - "GET / HTTP/1.0" 200 OK 2026-03-31 06:36:19,517 - lava_callback - INFO - Received callback for node 69c8c858f41e4fb6a3786627 INFO: 172.20.0.1:45660 - "POST /node/69c8c858f41e4fb6a3786627 HTTP/1.0" 202 Accepted 2026-03-31 06:36:19,523 - lava_callback - INFO - Starting async job submit for node 69c8c858f41e4fb6a3786627 2026-03-31 06:36:19,524 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 06:36:19,544 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8c858f41e4fb6a3786627 HTTP/1.1" 200 None 2026-03-31 06:36:19,546 - lava_callback - DEBUG - Processing node 69c8c858f41e4fb6a3786627: baseline-arm-baylibre 2026-03-31 06:36:19,546 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8c858f41e4fb6a3786627 2026-03-31 06:36:19,547 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 06:36:19,561 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8c858f41e4fb6a3786627 HTTP/1.1" 200 None 2026-03-31 06:36:19,562 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 06:36:19,580 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-03-31 06:36:19,581 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 06:36:19,606 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8c858f41e4fb6a3786627 HTTP/1.1" 200 None 2026-03-31 06:36:19,608 - lava_callback - ERROR - Error processing callback for node 69c8c858f41e4fb6a3786627: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:58968 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43540 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43836 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43844 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35686 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34202 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35086 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35094 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54630 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45328 - "GET / HTTP/1.0" 200 OK 2026-03-31 06:37:39,656 - lava_callback - INFO - Received callback for node 69c8c89ff41e4fb6a3786642 INFO: 172.20.0.1:45344 - "POST /node/69c8c89ff41e4fb6a3786642 HTTP/1.0" 202 Accepted 2026-03-31 06:37:39,661 - lava_callback - INFO - Starting async job submit for node 69c8c89ff41e4fb6a3786642 2026-03-31 06:37:39,662 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 06:37:39,676 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8c89ff41e4fb6a3786642 HTTP/1.1" 200 None 2026-03-31 06:37:39,677 - lava_callback - DEBUG - Processing node 69c8c89ff41e4fb6a3786642: baseline-arm-baylibre 2026-03-31 06:37:39,677 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8c89ff41e4fb6a3786642 2026-03-31 06:37:39,678 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 06:37:39,691 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8c89ff41e4fb6a3786642 HTTP/1.1" 200 None 2026-03-31 06:37:39,693 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 06:37:39,706 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-03-31 06:37:39,707 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 06:37:39,733 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8c89ff41e4fb6a3786642 HTTP/1.1" 200 None 2026-03-31 06:37:39,741 - lava_callback - ERROR - Error processing callback for node 69c8c89ff41e4fb6a3786642: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:44914 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44916 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33214 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49616 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60076 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60092 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46630 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37942 - "GET / HTTP/1.0" 200 OK 2026-03-31 06:38:39,640 - lava_callback - INFO - Received callback for node 69c8c8e7f41e4fb6a378665f INFO: 172.20.0.1:37954 - "POST /node/69c8c8e7f41e4fb6a378665f HTTP/1.0" 202 Accepted 2026-03-31 06:38:39,649 - lava_callback - INFO - Starting async job submit for node 69c8c8e7f41e4fb6a378665f 2026-03-31 06:38:39,650 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 06:38:39,667 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8c8e7f41e4fb6a378665f HTTP/1.1" 200 None 2026-03-31 06:38:39,668 - lava_callback - DEBUG - Processing node 69c8c8e7f41e4fb6a378665f: baseline-arm-baylibre 2026-03-31 06:38:39,668 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8c8e7f41e4fb6a378665f 2026-03-31 06:38:39,669 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 06:38:39,694 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8c8e7f41e4fb6a378665f HTTP/1.1" 200 None 2026-03-31 06:38:39,695 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 06:38:39,709 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-03-31 06:38:39,710 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 06:38:39,734 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8c8e7f41e4fb6a378665f HTTP/1.1" 200 None 2026-03-31 06:38:39,736 - lava_callback - ERROR - Error processing callback for node 69c8c8e7f41e4fb6a378665f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:50524 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50536 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38406 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57732 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60286 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60298 - "GET / HTTP/1.0" 200 OK 2026-03-31 06:39:19,946 - lava_callback - INFO - Received callback for node 69c8c90bf41e4fb6a378666d INFO: 172.20.0.1:51618 - "POST /node/69c8c90bf41e4fb6a378666d HTTP/1.0" 202 Accepted 2026-03-31 06:39:19,952 - lava_callback - INFO - Starting async job submit for node 69c8c90bf41e4fb6a378666d 2026-03-31 06:39:19,953 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 06:39:19,968 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8c90bf41e4fb6a378666d HTTP/1.1" 200 None 2026-03-31 06:39:19,969 - lava_callback - DEBUG - Processing node 69c8c90bf41e4fb6a378666d: baseline-arm-baylibre 2026-03-31 06:39:19,970 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8c90bf41e4fb6a378666d 2026-03-31 06:39:19,970 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 06:39:20,016 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8c90bf41e4fb6a378666d HTTP/1.1" 200 None 2026-03-31 06:39:20,022 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 06:39:20,050 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-03-31 06:39:20,052 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 06:39:20,084 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8c90bf41e4fb6a378666d HTTP/1.1" 200 None 2026-03-31 06:39:20,085 - lava_callback - ERROR - Error processing callback for node 69c8c90bf41e4fb6a378666d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:51630 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52686 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37196 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37204 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56736 - "GET / HTTP/1.0" 200 OK 2026-03-31 06:40:01,329 - lava_callback - INFO - Received callback for node 69c8c931f41e4fb6a378667a INFO: 172.20.0.1:48980 - "POST /node/69c8c931f41e4fb6a378667a HTTP/1.0" 202 Accepted 2026-03-31 06:40:01,335 - lava_callback - INFO - Starting async job submit for node 69c8c931f41e4fb6a378667a 2026-03-31 06:40:01,336 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 06:40:01,351 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8c931f41e4fb6a378667a HTTP/1.1" 200 None 2026-03-31 06:40:01,352 - lava_callback - DEBUG - Processing node 69c8c931f41e4fb6a378667a: baseline-arm-clabbe 2026-03-31 06:40:01,352 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8c931f41e4fb6a378667a 2026-03-31 06:40:01,353 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 06:40:01,367 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8c931f41e4fb6a378667a HTTP/1.1" 200 None 2026-03-31 06:40:01,368 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 06:40:01,382 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-03-31 06:40:01,384 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 06:40:01,404 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8c931f41e4fb6a378667a HTTP/1.1" 200 None 2026-03-31 06:40:01,405 - lava_callback - ERROR - Error processing callback for node 69c8c931f41e4fb6a378667a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:48984 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45754 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45766 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57856 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59366 - "GET / HTTP/1.0" 200 OK 2026-03-31 06:40:39,809 - lava_callback - INFO - Received callback for node 69c8c952f41e4fb6a37866b2 INFO: 172.20.0.1:39864 - "POST /node/69c8c952f41e4fb6a37866b2 HTTP/1.0" 202 Accepted 2026-03-31 06:40:39,814 - lava_callback - INFO - Starting async job submit for node 69c8c952f41e4fb6a37866b2 2026-03-31 06:40:39,815 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 06:40:39,830 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8c952f41e4fb6a37866b2 HTTP/1.1" 200 None 2026-03-31 06:40:39,831 - lava_callback - DEBUG - Processing node 69c8c952f41e4fb6a37866b2: baseline-arm-baylibre 2026-03-31 06:40:39,831 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8c952f41e4fb6a37866b2 2026-03-31 06:40:39,831 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 06:40:39,854 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8c952f41e4fb6a37866b2 HTTP/1.1" 200 None 2026-03-31 06:40:39,856 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 06:40:39,873 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-03-31 06:40:39,874 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 06:40:39,895 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8c952f41e4fb6a37866b2 HTTP/1.1" 200 None 2026-03-31 06:40:39,900 - lava_callback - ERROR - Error processing callback for node 69c8c952f41e4fb6a37866b2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:39880 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39894 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43180 - "GET / HTTP/1.0" 200 OK 2026-03-31 06:40:59,818 - lava_callback - INFO - Received callback for node 69c8c976f41e4fb6a37866c7 INFO: 172.20.0.1:34522 - "POST /node/69c8c976f41e4fb6a37866c7 HTTP/1.0" 202 Accepted 2026-03-31 06:40:59,824 - lava_callback - INFO - Starting async job submit for node 69c8c976f41e4fb6a37866c7 2026-03-31 06:40:59,824 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 06:40:59,839 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8c976f41e4fb6a37866c7 HTTP/1.1" 200 None 2026-03-31 06:40:59,840 - lava_callback - DEBUG - Processing node 69c8c976f41e4fb6a37866c7: baseline-arm-baylibre 2026-03-31 06:40:59,840 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8c976f41e4fb6a37866c7 2026-03-31 06:40:59,840 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 06:40:59,854 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8c976f41e4fb6a37866c7 HTTP/1.1" 200 None 2026-03-31 06:40:59,855 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 06:40:59,892 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-03-31 06:40:59,893 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 06:40:59,927 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8c976f41e4fb6a37866c7 HTTP/1.1" 200 None 2026-03-31 06:40:59,932 - lava_callback - ERROR - Error processing callback for node 69c8c976f41e4fb6a37866c7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:34528 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47888 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47890 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40240 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56166 - "GET / HTTP/1.0" 200 OK 2026-03-31 06:41:41,313 - lava_callback - INFO - Received callback for node 69c8c99cf41e4fb6a37866d4 INFO: 172.20.0.1:56000 - "POST /node/69c8c99cf41e4fb6a37866d4 HTTP/1.0" 202 Accepted 2026-03-31 06:41:41,319 - lava_callback - INFO - Starting async job submit for node 69c8c99cf41e4fb6a37866d4 2026-03-31 06:41:41,320 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 06:41:41,340 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8c99cf41e4fb6a37866d4 HTTP/1.1" 200 None 2026-03-31 06:41:41,341 - lava_callback - DEBUG - Processing node 69c8c99cf41e4fb6a37866d4: baseline-arm-clabbe 2026-03-31 06:41:41,342 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8c99cf41e4fb6a37866d4 2026-03-31 06:41:41,342 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 06:41:41,357 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8c99cf41e4fb6a37866d4 HTTP/1.1" 200 None 2026-03-31 06:41:41,359 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 06:41:41,374 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-03-31 06:41:41,376 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 06:41:41,399 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8c99cf41e4fb6a37866d4 HTTP/1.1" 200 None 2026-03-31 06:41:41,401 - lava_callback - ERROR - Error processing callback for node 69c8c99cf41e4fb6a37866d4: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:56004 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56012 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41578 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48208 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59586 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59592 - "GET / HTTP/1.0" 200 OK 2026-03-31 06:42:19,885 - lava_callback - INFO - Received callback for node 69c8c9bef41e4fb6a37866e2 INFO: 172.20.0.1:42006 - "POST /node/69c8c9bef41e4fb6a37866e2 HTTP/1.0" 202 Accepted 2026-03-31 06:42:19,891 - lava_callback - INFO - Starting async job submit for node 69c8c9bef41e4fb6a37866e2 2026-03-31 06:42:19,892 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 06:42:19,906 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8c9bef41e4fb6a37866e2 HTTP/1.1" 200 None 2026-03-31 06:42:19,907 - lava_callback - DEBUG - Processing node 69c8c9bef41e4fb6a37866e2: baseline-arm-baylibre 2026-03-31 06:42:19,907 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8c9bef41e4fb6a37866e2 2026-03-31 06:42:19,908 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 06:42:19,921 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8c9bef41e4fb6a37866e2 HTTP/1.1" 200 None 2026-03-31 06:42:19,922 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 06:42:19,940 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-03-31 06:42:19,941 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 06:42:19,963 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8c9bef41e4fb6a37866e2 HTTP/1.1" 200 None 2026-03-31 06:42:19,970 - lava_callback - ERROR - Error processing callback for node 69c8c9bef41e4fb6a37866e2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:42014 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53890 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40738 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40740 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43472 - "GET / HTTP/1.0" 200 OK 2026-03-31 06:43:05,285 - lava_callback - INFO - Received callback for node 69cb1f5b8c06c2799bf57ac8 INFO: 172.20.0.1:59762 - "POST /node/69cb1f5b8c06c2799bf57ac8 HTTP/1.0" 202 Accepted 2026-03-31 06:43:05,294 - lava_callback - INFO - Starting async job submit for node 69cb1f5b8c06c2799bf57ac8 2026-03-31 06:43:05,319 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 06:43:05,332 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1f5b8c06c2799bf57ac8 HTTP/1.1" 200 None 2026-03-31 06:43:05,333 - lava_callback - DEBUG - Processing node 69cb1f5b8c06c2799bf57ac8: baseline-arm64-broonie 2026-03-31 06:43:05,626 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 06:43:05,783 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 06:43:05,785 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cb1f5b8c06c2799bf57ac8/log.txt.gz 2026-03-31 06:43:05,959 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 06:43:06,181 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 06:43:06,183 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cb1f5b8c06c2799bf57ac8/lava-logs.yaml 2026-03-31 06:43:06,190 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 06:43:06,332 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 06:43:06,334 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cb1f5b8c06c2799bf57ac8/lava_callback.json.gz 2026-03-31 06:43:06,357 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 06:43:06,379 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1f5b8c06c2799bf57ac8 HTTP/1.1" 200 None 2026-03-31 06:43:06,381 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 06:43:06,397 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12e08c06c2799bf5502b HTTP/1.1" 200 None 2026-03-31 06:43:06,399 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 06:43:06,424 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb1f5b8c06c2799bf57ac8 HTTP/1.1" 200 None 2026-03-31 06:43:06,434 - lava_callback - INFO - Completed processing callback for node 69cb1f5b8c06c2799bf57ac8 INFO: 172.20.0.1:59778 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40424 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40426 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35824 - "GET / HTTP/1.0" 200 OK 2026-03-31 06:43:31,941 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 06:43:31,958 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:51096 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42308 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42320 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39324 - "GET / HTTP/1.0" 200 OK 2026-03-31 06:44:00,030 - lava_callback - INFO - Received callback for node 69c8ca26f41e4fb6a378670b INFO: 172.20.0.1:46288 - "POST /node/69c8ca26f41e4fb6a378670b HTTP/1.0" 202 Accepted 2026-03-31 06:44:00,036 - lava_callback - INFO - Starting async job submit for node 69c8ca26f41e4fb6a378670b 2026-03-31 06:44:00,037 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 06:44:00,051 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8ca26f41e4fb6a378670b HTTP/1.1" 200 None 2026-03-31 06:44:00,052 - lava_callback - DEBUG - Processing node 69c8ca26f41e4fb6a378670b: baseline-arm-baylibre 2026-03-31 06:44:00,052 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8ca26f41e4fb6a378670b 2026-03-31 06:44:00,053 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 06:44:00,067 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8ca26f41e4fb6a378670b HTTP/1.1" 200 None 2026-03-31 06:44:00,069 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 06:44:00,083 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f72f41e4fb6a3782f3e HTTP/1.1" 200 None 2026-03-31 06:44:00,084 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 06:44:00,105 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8ca26f41e4fb6a378670b HTTP/1.1" 200 None 2026-03-31 06:44:00,112 - lava_callback - ERROR - Error processing callback for node 69c8ca26f41e4fb6a378670b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:46300 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54430 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54438 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49024 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38164 - "GET / HTTP/1.0" 200 OK 2026-03-31 06:44:39,979 - lava_callback - INFO - Received callback for node 69c8ca4af41e4fb6a378671b INFO: 172.20.0.1:48170 - "POST /node/69c8ca4af41e4fb6a378671b HTTP/1.0" 202 Accepted 2026-03-31 06:44:39,985 - lava_callback - INFO - Starting async job submit for node 69c8ca4af41e4fb6a378671b 2026-03-31 06:44:39,986 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 06:44:40,002 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8ca4af41e4fb6a378671b HTTP/1.1" 200 None 2026-03-31 06:44:40,003 - lava_callback - DEBUG - Processing node 69c8ca4af41e4fb6a378671b: baseline-arm-baylibre 2026-03-31 06:44:40,003 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8ca4af41e4fb6a378671b 2026-03-31 06:44:40,004 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 06:44:40,035 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8ca4af41e4fb6a378671b HTTP/1.1" 200 None 2026-03-31 06:44:40,036 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 06:44:40,049 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f72f41e4fb6a3782f3e HTTP/1.1" 200 None 2026-03-31 06:44:40,051 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 06:44:40,072 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8ca4af41e4fb6a378671b HTTP/1.1" 200 None 2026-03-31 06:44:40,074 - lava_callback - ERROR - Error processing callback for node 69c8ca4af41e4fb6a378671b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:48180 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48194 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37998 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43052 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48590 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48594 - "GET / HTTP/1.0" 200 OK 2026-03-31 06:45:20,044 - lava_callback - INFO - Received callback for node 69c8ca6ef41e4fb6a3786724 INFO: 172.20.0.1:41680 - "POST /node/69c8ca6ef41e4fb6a3786724 HTTP/1.0" 202 Accepted 2026-03-31 06:45:20,050 - lava_callback - INFO - Starting async job submit for node 69c8ca6ef41e4fb6a3786724 2026-03-31 06:45:20,050 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 06:45:20,065 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8ca6ef41e4fb6a3786724 HTTP/1.1" 200 None 2026-03-31 06:45:20,066 - lava_callback - DEBUG - Processing node 69c8ca6ef41e4fb6a3786724: baseline-arm-baylibre 2026-03-31 06:45:20,066 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8ca6ef41e4fb6a3786724 2026-03-31 06:45:20,066 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 06:45:20,080 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8ca6ef41e4fb6a3786724 HTTP/1.1" 200 None 2026-03-31 06:45:20,081 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 06:45:20,101 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f72f41e4fb6a3782f3e HTTP/1.1" 200 None 2026-03-31 06:45:20,103 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 06:45:20,123 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8ca6ef41e4fb6a3786724 HTTP/1.1" 200 None 2026-03-31 06:45:20,129 - lava_callback - ERROR - Error processing callback for node 69c8ca6ef41e4fb6a3786724: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:41692 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39236 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39186 - "GET /node/68fad089a011bb5928ee7513%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:39198 - "GET /node/693e3473407f32424eb5a6a2%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:39210 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39224 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47474 - "GET / HTTP/1.0" 200 OK 2026-03-31 06:46:01,581 - lava_callback - INFO - Received callback for node 69c8ca93f41e4fb6a3786734 INFO: 172.20.0.1:52142 - "POST /node/69c8ca93f41e4fb6a3786734 HTTP/1.0" 202 Accepted 2026-03-31 06:46:01,587 - lava_callback - INFO - Starting async job submit for node 69c8ca93f41e4fb6a3786734 2026-03-31 06:46:01,587 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 06:46:01,604 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8ca93f41e4fb6a3786734 HTTP/1.1" 200 None 2026-03-31 06:46:01,605 - lava_callback - DEBUG - Processing node 69c8ca93f41e4fb6a3786734: baseline-arm-clabbe 2026-03-31 06:46:01,605 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8ca93f41e4fb6a3786734 2026-03-31 06:46:01,606 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 06:46:01,618 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8ca93f41e4fb6a3786734 HTTP/1.1" 200 None 2026-03-31 06:46:01,620 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 06:46:01,633 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-03-31 06:46:01,634 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 06:46:01,655 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8ca93f41e4fb6a3786734 HTTP/1.1" 200 None 2026-03-31 06:46:01,661 - lava_callback - ERROR - Error processing callback for node 69c8ca93f41e4fb6a3786734: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:52146 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37894 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37910 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37128 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56648 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53160 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53172 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53174 - "GET /node/694dd6668177197a294effce%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:38630 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40288 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47804 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47812 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51086 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59698 - "GET / HTTP/1.0" 200 OK 2026-03-31 06:47:41,592 - lava_callback - INFO - Received callback for node 69c8cafdf41e4fb6a3786760 INFO: 172.20.0.1:58662 - "POST /node/69c8cafdf41e4fb6a3786760 HTTP/1.0" 202 Accepted 2026-03-31 06:47:41,597 - lava_callback - INFO - Starting async job submit for node 69c8cafdf41e4fb6a3786760 2026-03-31 06:47:41,598 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 06:47:41,611 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8cafdf41e4fb6a3786760 HTTP/1.1" 200 None 2026-03-31 06:47:41,612 - lava_callback - DEBUG - Processing node 69c8cafdf41e4fb6a3786760: baseline-arm-clabbe 2026-03-31 06:47:41,612 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8cafdf41e4fb6a3786760 2026-03-31 06:47:41,613 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 06:47:41,626 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8cafdf41e4fb6a3786760 HTTP/1.1" 200 None 2026-03-31 06:47:41,627 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 06:47:41,640 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f72f41e4fb6a3782f3e HTTP/1.1" 200 None 2026-03-31 06:47:41,642 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 06:47:41,685 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8cafdf41e4fb6a3786760 HTTP/1.1" 200 None 2026-03-31 06:47:41,686 - lava_callback - ERROR - Error processing callback for node 69c8cafdf41e4fb6a3786760: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:58672 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58676 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57830 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42032 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43672 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43674 - "GET / HTTP/1.0" 200 OK 2026-03-31 06:48:21,565 - lava_callback - INFO - Received callback for node 69c8cb22f41e4fb6a3786769 INFO: 172.20.0.1:34326 - "POST /node/69c8cb22f41e4fb6a3786769 HTTP/1.0" 202 Accepted 2026-03-31 06:48:21,571 - lava_callback - INFO - Starting async job submit for node 69c8cb22f41e4fb6a3786769 2026-03-31 06:48:21,572 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 06:48:21,593 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8cb22f41e4fb6a3786769 HTTP/1.1" 200 None 2026-03-31 06:48:21,594 - lava_callback - DEBUG - Processing node 69c8cb22f41e4fb6a3786769: baseline-arm-clabbe 2026-03-31 06:48:21,594 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8cb22f41e4fb6a3786769 2026-03-31 06:48:21,595 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 06:48:21,608 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8cb22f41e4fb6a3786769 HTTP/1.1" 200 None 2026-03-31 06:48:21,609 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 06:48:21,622 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f72f41e4fb6a3782f3e HTTP/1.1" 200 None 2026-03-31 06:48:21,624 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 06:48:21,644 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8cb22f41e4fb6a3786769 HTTP/1.1" 200 None 2026-03-31 06:48:21,650 - lava_callback - ERROR - Error processing callback for node 69c8cb22f41e4fb6a3786769: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:34328 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48952 - "GET / HTTP/1.0" 200 OK 2026-03-31 06:48:41,605 - lava_callback - INFO - Received callback for node 69c8cb46f41e4fb6a3786776 INFO: 172.20.0.1:37078 - "POST /node/69c8cb46f41e4fb6a3786776 HTTP/1.0" 202 Accepted 2026-03-31 06:48:41,611 - lava_callback - INFO - Starting async job submit for node 69c8cb46f41e4fb6a3786776 2026-03-31 06:48:41,612 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 06:48:41,626 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8cb46f41e4fb6a3786776 HTTP/1.1" 200 None 2026-03-31 06:48:41,627 - lava_callback - DEBUG - Processing node 69c8cb46f41e4fb6a3786776: baseline-arm-clabbe 2026-03-31 06:48:41,627 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8cb46f41e4fb6a3786776 2026-03-31 06:48:41,628 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 06:48:41,641 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8cb46f41e4fb6a3786776 HTTP/1.1" 200 None 2026-03-31 06:48:41,643 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 06:48:41,657 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f72f41e4fb6a3782f3e HTTP/1.1" 200 None 2026-03-31 06:48:41,658 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 06:48:41,685 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8cb46f41e4fb6a3786776 HTTP/1.1" 200 None 2026-03-31 06:48:41,686 - lava_callback - ERROR - Error processing callback for node 69c8cb46f41e4fb6a3786776: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:37092 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37100 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59360 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45832 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57438 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57452 - "GET / HTTP/1.0" 200 OK 2026-03-31 06:49:21,608 - lava_callback - INFO - Received callback for node 69c8cb6af41e4fb6a3786785 INFO: 172.20.0.1:36684 - "POST /node/69c8cb6af41e4fb6a3786785 HTTP/1.0" 202 Accepted 2026-03-31 06:49:21,614 - lava_callback - INFO - Starting async job submit for node 69c8cb6af41e4fb6a3786785 2026-03-31 06:49:21,614 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 06:49:21,629 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8cb6af41e4fb6a3786785 HTTP/1.1" 200 None 2026-03-31 06:49:21,630 - lava_callback - DEBUG - Processing node 69c8cb6af41e4fb6a3786785: baseline-arm-clabbe 2026-03-31 06:49:21,630 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8cb6af41e4fb6a3786785 2026-03-31 06:49:21,631 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 06:49:21,643 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8cb6af41e4fb6a3786785 HTTP/1.1" 200 None 2026-03-31 06:49:21,645 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 06:49:21,659 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f72f41e4fb6a3782f3e HTTP/1.1" 200 None 2026-03-31 06:49:21,661 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 06:49:21,681 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8cb6af41e4fb6a3786785 HTTP/1.1" 200 None 2026-03-31 06:49:21,686 - lava_callback - ERROR - Error processing callback for node 69c8cb6af41e4fb6a3786785: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:36700 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33266 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37322 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37332 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43470 - "GET / HTTP/1.0" 200 OK 2026-03-31 06:50:01,633 - lava_callback - INFO - Received callback for node 69c8cb8ef41e4fb6a3786793 INFO: 172.20.0.1:42286 - "POST /node/69c8cb8ef41e4fb6a3786793 HTTP/1.0" 202 Accepted 2026-03-31 06:50:01,639 - lava_callback - INFO - Starting async job submit for node 69c8cb8ef41e4fb6a3786793 2026-03-31 06:50:01,640 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 06:50:01,671 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8cb8ef41e4fb6a3786793 HTTP/1.1" 200 None 2026-03-31 06:50:01,672 - lava_callback - DEBUG - Processing node 69c8cb8ef41e4fb6a3786793: baseline-arm-clabbe 2026-03-31 06:50:01,673 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8cb8ef41e4fb6a3786793 2026-03-31 06:50:01,673 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 06:50:01,687 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8cb8ef41e4fb6a3786793 HTTP/1.1" 200 None 2026-03-31 06:50:01,688 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 06:50:01,701 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f72f41e4fb6a3782f3e HTTP/1.1" 200 None 2026-03-31 06:50:01,703 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 06:50:01,724 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8cb8ef41e4fb6a3786793 HTTP/1.1" 200 None 2026-03-31 06:50:01,733 - lava_callback - ERROR - Error processing callback for node 69c8cb8ef41e4fb6a3786793: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:42296 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58738 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58752 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47002 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59666 - "GET / HTTP/1.0" 200 OK 2026-03-31 06:50:41,628 - lava_callback - INFO - Received callback for node 69c8cbb3f41e4fb6a37867a4 INFO: 172.20.0.1:34060 - "POST /node/69c8cbb3f41e4fb6a37867a4 HTTP/1.0" 202 Accepted 2026-03-31 06:50:41,634 - lava_callback - INFO - Starting async job submit for node 69c8cbb3f41e4fb6a37867a4 2026-03-31 06:50:41,634 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 06:50:41,649 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8cbb3f41e4fb6a37867a4 HTTP/1.1" 200 None 2026-03-31 06:50:41,649 - lava_callback - DEBUG - Processing node 69c8cbb3f41e4fb6a37867a4: baseline-arm-clabbe 2026-03-31 06:50:41,650 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8cbb3f41e4fb6a37867a4 2026-03-31 06:50:41,650 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 06:50:41,665 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8cbb3f41e4fb6a37867a4 HTTP/1.1" 200 None 2026-03-31 06:50:41,666 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 06:50:41,852 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-03-31 06:50:41,854 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 06:50:41,874 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8cbb3f41e4fb6a37867a4 HTTP/1.1" 200 None 2026-03-31 06:50:41,876 - lava_callback - ERROR - Error processing callback for node 69c8cbb3f41e4fb6a37867a4: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:34066 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34072 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60280 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34714 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35902 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35914 - "GET / HTTP/1.0" 200 OK 2026-03-31 06:51:21,731 - lava_callback - INFO - Received callback for node 69c8cbd7f41e4fb6a37867af INFO: 172.20.0.1:52016 - "POST /node/69c8cbd7f41e4fb6a37867af HTTP/1.0" 202 Accepted 2026-03-31 06:51:21,738 - lava_callback - INFO - Starting async job submit for node 69c8cbd7f41e4fb6a37867af 2026-03-31 06:51:21,739 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 06:51:21,754 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8cbd7f41e4fb6a37867af HTTP/1.1" 200 None 2026-03-31 06:51:21,756 - lava_callback - DEBUG - Processing node 69c8cbd7f41e4fb6a37867af: baseline-arm-clabbe 2026-03-31 06:51:21,756 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8cbd7f41e4fb6a37867af 2026-03-31 06:51:21,757 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 06:51:21,771 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8cbd7f41e4fb6a37867af HTTP/1.1" 200 None 2026-03-31 06:51:21,772 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 06:51:21,786 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-03-31 06:51:21,788 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 06:51:21,814 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8cbd7f41e4fb6a37867af HTTP/1.1" 200 None 2026-03-31 06:51:21,816 - lava_callback - ERROR - Error processing callback for node 69c8cbd7f41e4fb6a37867af: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:52020 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35014 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60488 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60494 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53362 - "GET / HTTP/1.0" 200 OK 2026-03-31 06:52:01,702 - lava_callback - INFO - Received callback for node 69c8cbfcf41e4fb6a37867bd INFO: 172.20.0.1:47084 - "POST /node/69c8cbfcf41e4fb6a37867bd HTTP/1.0" 202 Accepted 2026-03-31 06:52:01,708 - lava_callback - INFO - Starting async job submit for node 69c8cbfcf41e4fb6a37867bd 2026-03-31 06:52:01,709 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 06:52:01,724 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8cbfcf41e4fb6a37867bd HTTP/1.1" 200 None 2026-03-31 06:52:01,725 - lava_callback - DEBUG - Processing node 69c8cbfcf41e4fb6a37867bd: baseline-arm-clabbe 2026-03-31 06:52:01,725 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8cbfcf41e4fb6a37867bd 2026-03-31 06:52:01,725 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 06:52:01,738 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8cbfcf41e4fb6a37867bd HTTP/1.1" 200 None 2026-03-31 06:52:01,740 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 06:52:01,754 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-03-31 06:52:01,755 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 06:52:01,777 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8cbfcf41e4fb6a37867bd HTTP/1.1" 200 None 2026-03-31 06:52:01,782 - lava_callback - ERROR - Error processing callback for node 69c8cbfcf41e4fb6a37867bd: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:47094 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34226 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34242 - "GET / HTTP/1.0" 200 OK 2026-03-31 06:52:21,750 - lava_callback - INFO - Received callback for node 69c8cc20f41e4fb6a37867cb INFO: 172.20.0.1:53846 - "POST /node/69c8cc20f41e4fb6a37867cb HTTP/1.0" 202 Accepted 2026-03-31 06:52:21,755 - lava_callback - INFO - Starting async job submit for node 69c8cc20f41e4fb6a37867cb 2026-03-31 06:52:21,756 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 06:52:21,775 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8cc20f41e4fb6a37867cb HTTP/1.1" 200 None 2026-03-31 06:52:21,776 - lava_callback - DEBUG - Processing node 69c8cc20f41e4fb6a37867cb: baseline-arm-clabbe 2026-03-31 06:52:21,776 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8cc20f41e4fb6a37867cb 2026-03-31 06:52:21,777 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 06:52:21,799 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8cc20f41e4fb6a37867cb HTTP/1.1" 200 None 2026-03-31 06:52:21,801 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 06:52:21,821 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-03-31 06:52:21,823 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 06:52:21,850 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8cc20f41e4fb6a37867cb HTTP/1.1" 200 None 2026-03-31 06:52:21,851 - lava_callback - ERROR - Error processing callback for node 69c8cc20f41e4fb6a37867cb: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:53850 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34186 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53114 - "GET /node/69892f8585773790cd34b9d4%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:53120 - "GET /node/699034ef985e4efca431c58d%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:53132 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53136 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57928 - "GET / HTTP/1.0" 200 OK 2026-03-31 06:53:01,762 - lava_callback - INFO - Received callback for node 69c8cc43f41e4fb6a37867d7 INFO: 172.20.0.1:38096 - "POST /node/69c8cc43f41e4fb6a37867d7 HTTP/1.0" 202 Accepted 2026-03-31 06:53:01,770 - lava_callback - INFO - Starting async job submit for node 69c8cc43f41e4fb6a37867d7 2026-03-31 06:53:01,771 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 06:53:01,787 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8cc43f41e4fb6a37867d7 HTTP/1.1" 200 None 2026-03-31 06:53:01,788 - lava_callback - DEBUG - Processing node 69c8cc43f41e4fb6a37867d7: baseline-arm-clabbe 2026-03-31 06:53:01,788 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8cc43f41e4fb6a37867d7 2026-03-31 06:53:01,789 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 06:53:01,806 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8cc43f41e4fb6a37867d7 HTTP/1.1" 200 None 2026-03-31 06:53:01,808 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 06:53:01,822 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-03-31 06:53:01,823 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 06:53:01,858 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8cc43f41e4fb6a37867d7 HTTP/1.1" 200 None 2026-03-31 06:53:01,861 - lava_callback - ERROR - Error processing callback for node 69c8cc43f41e4fb6a37867d7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:38112 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44242 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44246 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45626 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44592 - "GET / HTTP/1.0" 200 OK 2026-03-31 06:53:41,806 - lava_callback - INFO - Received callback for node 69c8cc67f41e4fb6a37867e8 INFO: 172.20.0.1:33980 - "POST /node/69c8cc67f41e4fb6a37867e8 HTTP/1.0" 202 Accepted 2026-03-31 06:53:41,812 - lava_callback - INFO - Starting async job submit for node 69c8cc67f41e4fb6a37867e8 2026-03-31 06:53:41,813 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 06:53:41,834 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8cc67f41e4fb6a37867e8 HTTP/1.1" 200 None 2026-03-31 06:53:41,835 - lava_callback - DEBUG - Processing node 69c8cc67f41e4fb6a37867e8: baseline-arm-clabbe 2026-03-31 06:53:41,835 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8cc67f41e4fb6a37867e8 2026-03-31 06:53:41,836 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 06:53:41,850 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8cc67f41e4fb6a37867e8 HTTP/1.1" 200 None 2026-03-31 06:53:41,851 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 06:53:41,864 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-03-31 06:53:41,866 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 06:53:41,891 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8cc67f41e4fb6a37867e8 HTTP/1.1" 200 None 2026-03-31 06:53:41,892 - lava_callback - ERROR - Error processing callback for node 69c8cc67f41e4fb6a37867e8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:33992 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34002 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53196 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36410 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46050 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46056 - "GET / HTTP/1.0" 200 OK 2026-03-31 06:54:21,798 - lava_callback - INFO - Received callback for node 69c8cc8cf41e4fb6a37867f0 INFO: 172.20.0.1:60306 - "POST /node/69c8cc8cf41e4fb6a37867f0 HTTP/1.0" 202 Accepted 2026-03-31 06:54:21,804 - lava_callback - INFO - Starting async job submit for node 69c8cc8cf41e4fb6a37867f0 2026-03-31 06:54:21,805 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 06:54:21,819 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8cc8cf41e4fb6a37867f0 HTTP/1.1" 200 None 2026-03-31 06:54:21,820 - lava_callback - DEBUG - Processing node 69c8cc8cf41e4fb6a37867f0: baseline-arm-clabbe 2026-03-31 06:54:21,820 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8cc8cf41e4fb6a37867f0 2026-03-31 06:54:21,821 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 06:54:21,834 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8cc8cf41e4fb6a37867f0 HTTP/1.1" 200 None 2026-03-31 06:54:21,836 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 06:54:21,849 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-03-31 06:54:21,850 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 06:54:21,876 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8cc8cf41e4fb6a37867f0 HTTP/1.1" 200 None 2026-03-31 06:54:21,877 - lava_callback - ERROR - Error processing callback for node 69c8cc8cf41e4fb6a37867f0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:60318 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33970 - "GET / HTTP/1.0" 200 OK 2026-03-31 06:54:40,680 - lava_callback - INFO - Received callback for node 69c8ccabf41e4fb6a37867fb INFO: 172.20.0.1:46748 - "POST /node/69c8ccabf41e4fb6a37867fb HTTP/1.0" 202 Accepted 2026-03-31 06:54:40,686 - lava_callback - INFO - Starting async job submit for node 69c8ccabf41e4fb6a37867fb 2026-03-31 06:54:40,686 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 06:54:40,701 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8ccabf41e4fb6a37867fb HTTP/1.1" 200 None 2026-03-31 06:54:40,702 - lava_callback - DEBUG - Processing node 69c8ccabf41e4fb6a37867fb: baseline-arm-baylibre 2026-03-31 06:54:40,702 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8ccabf41e4fb6a37867fb 2026-03-31 06:54:40,703 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 06:54:40,716 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8ccabf41e4fb6a37867fb HTTP/1.1" 200 None 2026-03-31 06:54:40,717 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 06:54:40,730 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3293e18a3717aa65d01ad HTTP/1.1" 200 None 2026-03-31 06:54:40,732 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 06:54:40,751 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8ccabf41e4fb6a37867fb HTTP/1.1" 200 None 2026-03-31 06:54:40,753 - lava_callback - ERROR - Error processing callback for node 69c8ccabf41e4fb6a37867fb: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:46754 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46758 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58736 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39130 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37662 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37666 - "GET / HTTP/1.0" 200 OK 2026-03-31 06:55:20,575 - lava_callback - INFO - Received callback for node 69c8cccff41e4fb6a3786805 INFO: 172.20.0.1:36972 - "POST /node/69c8cccff41e4fb6a3786805 HTTP/1.0" 202 Accepted 2026-03-31 06:55:20,580 - lava_callback - INFO - Starting async job submit for node 69c8cccff41e4fb6a3786805 2026-03-31 06:55:20,581 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 06:55:20,600 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8cccff41e4fb6a3786805 HTTP/1.1" 200 None 2026-03-31 06:55:20,601 - lava_callback - DEBUG - Processing node 69c8cccff41e4fb6a3786805: baseline-arm-baylibre 2026-03-31 06:55:20,601 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8cccff41e4fb6a3786805 2026-03-31 06:55:20,602 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 06:55:20,614 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8cccff41e4fb6a3786805 HTTP/1.1" 200 None 2026-03-31 06:55:20,616 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 06:55:20,630 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3293e18a3717aa65d01ad HTTP/1.1" 200 None 2026-03-31 06:55:20,631 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 06:55:20,652 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8cccff41e4fb6a3786805 HTTP/1.1" 200 None 2026-03-31 06:55:20,658 - lava_callback - ERROR - Error processing callback for node 69c8cccff41e4fb6a3786805: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:36988 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44146 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50934 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50946 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44634 - "GET / HTTP/1.0" 200 OK 2026-03-31 06:56:00,674 - lava_callback - INFO - Received callback for node 69c8ccf4f41e4fb6a378680e INFO: 172.20.0.1:57548 - "POST /node/69c8ccf4f41e4fb6a378680e HTTP/1.0" 202 Accepted 2026-03-31 06:56:00,679 - lava_callback - INFO - Starting async job submit for node 69c8ccf4f41e4fb6a378680e 2026-03-31 06:56:00,680 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 06:56:00,694 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8ccf4f41e4fb6a378680e HTTP/1.1" 200 None 2026-03-31 06:56:00,695 - lava_callback - DEBUG - Processing node 69c8ccf4f41e4fb6a378680e: baseline-arm-baylibre 2026-03-31 06:56:00,695 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8ccf4f41e4fb6a378680e 2026-03-31 06:56:00,696 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 06:56:00,710 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8ccf4f41e4fb6a378680e HTTP/1.1" 200 None 2026-03-31 06:56:00,711 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 06:56:00,724 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3293e18a3717aa65d01ad HTTP/1.1" 200 None 2026-03-31 06:56:00,725 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 06:56:00,746 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8ccf4f41e4fb6a378680e HTTP/1.1" 200 None 2026-03-31 06:56:00,747 - lava_callback - ERROR - Error processing callback for node 69c8ccf4f41e4fb6a378680e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:57562 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60716 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60724 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60272 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45684 - "GET / HTTP/1.0" 200 OK 2026-03-31 06:56:40,679 - lava_callback - INFO - Received callback for node 69c8cd17f41e4fb6a3786816 INFO: 172.20.0.1:60246 - "POST /node/69c8cd17f41e4fb6a3786816 HTTP/1.0" 202 Accepted 2026-03-31 06:56:40,685 - lava_callback - INFO - Starting async job submit for node 69c8cd17f41e4fb6a3786816 2026-03-31 06:56:40,686 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 06:56:40,700 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8cd17f41e4fb6a3786816 HTTP/1.1" 200 None 2026-03-31 06:56:40,701 - lava_callback - DEBUG - Processing node 69c8cd17f41e4fb6a3786816: baseline-arm-baylibre 2026-03-31 06:56:40,701 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8cd17f41e4fb6a3786816 2026-03-31 06:56:40,702 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 06:56:40,715 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8cd17f41e4fb6a3786816 HTTP/1.1" 200 None 2026-03-31 06:56:40,717 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 06:56:40,730 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3293e18a3717aa65d01ad HTTP/1.1" 200 None 2026-03-31 06:56:40,731 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 06:56:40,755 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8cd17f41e4fb6a3786816 HTTP/1.1" 200 None 2026-03-31 06:56:40,756 - lava_callback - ERROR - Error processing callback for node 69c8cd17f41e4fb6a3786816: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:60262 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60274 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50580 - "GET / HTTP/1.0" 200 OK 2026-03-31 06:57:00,687 - lava_callback - INFO - Received callback for node 69c8cd3af41e4fb6a3786824 INFO: 172.20.0.1:53232 - "POST /node/69c8cd3af41e4fb6a3786824 HTTP/1.0" 202 Accepted 2026-03-31 06:57:00,693 - lava_callback - INFO - Starting async job submit for node 69c8cd3af41e4fb6a3786824 2026-03-31 06:57:00,693 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 06:57:00,708 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8cd3af41e4fb6a3786824 HTTP/1.1" 200 None 2026-03-31 06:57:00,708 - lava_callback - DEBUG - Processing node 69c8cd3af41e4fb6a3786824: baseline-arm-baylibre 2026-03-31 06:57:00,709 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8cd3af41e4fb6a3786824 2026-03-31 06:57:00,709 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 06:57:00,723 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8cd3af41e4fb6a3786824 HTTP/1.1" 200 None 2026-03-31 06:57:00,724 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 06:57:00,749 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3293e18a3717aa65d01ad HTTP/1.1" 200 None 2026-03-31 06:57:00,751 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 06:57:00,781 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8cd3af41e4fb6a3786824 HTTP/1.1" 200 None 2026-03-31 06:57:00,783 - lava_callback - ERROR - Error processing callback for node 69c8cd3af41e4fb6a3786824: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:53236 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46604 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46618 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58070 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49012 - "GET / HTTP/1.0" 200 OK 2026-03-31 06:57:41,983 - lava_callback - INFO - Received callback for node 69c8cd5ff41e4fb6a378682d INFO: 172.20.0.1:42996 - "POST /node/69c8cd5ff41e4fb6a378682d HTTP/1.0" 202 Accepted 2026-03-31 06:57:41,988 - lava_callback - INFO - Starting async job submit for node 69c8cd5ff41e4fb6a378682d 2026-03-31 06:57:41,989 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 06:57:42,005 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8cd5ff41e4fb6a378682d HTTP/1.1" 200 None 2026-03-31 06:57:42,006 - lava_callback - DEBUG - Processing node 69c8cd5ff41e4fb6a378682d: baseline-arm-clabbe 2026-03-31 06:57:42,006 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8cd5ff41e4fb6a378682d 2026-03-31 06:57:42,007 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 06:57:42,022 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8cd5ff41e4fb6a378682d HTTP/1.1" 200 None 2026-03-31 06:57:42,023 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 06:57:42,041 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3293e18a3717aa65d01ad HTTP/1.1" 200 None 2026-03-31 06:57:42,042 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 06:57:42,063 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8cd5ff41e4fb6a378682d HTTP/1.1" 200 None 2026-03-31 06:57:42,072 - lava_callback - ERROR - Error processing callback for node 69c8cd5ff41e4fb6a378682d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:43000 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43010 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42214 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47974 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45336 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45338 - "GET / HTTP/1.0" 200 OK 2026-03-31 06:58:21,965 - lava_callback - INFO - Received callback for node 69c8cd83f41e4fb6a3786838 INFO: 172.20.0.1:49288 - "POST /node/69c8cd83f41e4fb6a3786838 HTTP/1.0" 202 Accepted 2026-03-31 06:58:21,970 - lava_callback - INFO - Starting async job submit for node 69c8cd83f41e4fb6a3786838 2026-03-31 06:58:21,971 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 06:58:21,987 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8cd83f41e4fb6a3786838 HTTP/1.1" 200 None 2026-03-31 06:58:21,988 - lava_callback - DEBUG - Processing node 69c8cd83f41e4fb6a3786838: baseline-arm-clabbe 2026-03-31 06:58:21,988 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8cd83f41e4fb6a3786838 2026-03-31 06:58:21,988 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 06:58:22,002 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8cd83f41e4fb6a3786838 HTTP/1.1" 200 None 2026-03-31 06:58:22,003 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 06:58:22,016 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3293e18a3717aa65d01ad HTTP/1.1" 200 None 2026-03-31 06:58:22,018 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 06:58:22,038 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8cd83f41e4fb6a3786838 HTTP/1.1" 200 None 2026-03-31 06:58:22,044 - lava_callback - ERROR - Error processing callback for node 69c8cd83f41e4fb6a3786838: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:49290 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34884 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47872 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47886 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48310 - "GET / HTTP/1.0" 200 OK 2026-03-31 06:59:02,005 - lava_callback - INFO - Received callback for node 69c8cda7f41e4fb6a3786843 INFO: 172.20.0.1:57416 - "POST /node/69c8cda7f41e4fb6a3786843 HTTP/1.0" 202 Accepted 2026-03-31 06:59:02,011 - lava_callback - INFO - Starting async job submit for node 69c8cda7f41e4fb6a3786843 2026-03-31 06:59:02,012 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 06:59:02,031 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8cda7f41e4fb6a3786843 HTTP/1.1" 200 None 2026-03-31 06:59:02,032 - lava_callback - DEBUG - Processing node 69c8cda7f41e4fb6a3786843: baseline-arm-clabbe 2026-03-31 06:59:02,032 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8cda7f41e4fb6a3786843 2026-03-31 06:59:02,033 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 06:59:02,046 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8cda7f41e4fb6a3786843 HTTP/1.1" 200 None 2026-03-31 06:59:02,047 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 06:59:02,062 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3293e18a3717aa65d01ad HTTP/1.1" 200 None 2026-03-31 06:59:02,063 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 06:59:02,088 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8cda7f41e4fb6a3786843 HTTP/1.1" 200 None 2026-03-31 06:59:02,090 - lava_callback - ERROR - Error processing callback for node 69c8cda7f41e4fb6a3786843: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:57428 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44066 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44074 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58352 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60656 - "GET / HTTP/1.0" 200 OK 2026-03-31 06:59:42,038 - lava_callback - INFO - Received callback for node 69c8cdccf41e4fb6a378684d INFO: 172.20.0.1:45132 - "POST /node/69c8cdccf41e4fb6a378684d HTTP/1.0" 202 Accepted 2026-03-31 06:59:42,044 - lava_callback - INFO - Starting async job submit for node 69c8cdccf41e4fb6a378684d 2026-03-31 06:59:42,044 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 06:59:42,059 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8cdccf41e4fb6a378684d HTTP/1.1" 200 None 2026-03-31 06:59:42,060 - lava_callback - DEBUG - Processing node 69c8cdccf41e4fb6a378684d: baseline-arm-clabbe 2026-03-31 06:59:42,060 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8cdccf41e4fb6a378684d 2026-03-31 06:59:42,061 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 06:59:42,075 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8cdccf41e4fb6a378684d HTTP/1.1" 200 None 2026-03-31 06:59:42,076 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 06:59:42,102 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3293e18a3717aa65d01ad HTTP/1.1" 200 None 2026-03-31 06:59:42,104 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 06:59:42,125 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8cdccf41e4fb6a378684d HTTP/1.1" 200 None 2026-03-31 06:59:42,130 - lava_callback - ERROR - Error processing callback for node 69c8cdccf41e4fb6a378684d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:45144 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45148 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35882 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35518 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36262 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36274 - "GET / HTTP/1.0" 200 OK 2026-03-31 07:00:22,246 - lava_callback - INFO - Received callback for node 69c8cdf0f41e4fb6a378685a INFO: 172.20.0.1:51572 - "POST /node/69c8cdf0f41e4fb6a378685a HTTP/1.0" 202 Accepted 2026-03-31 07:00:22,252 - lava_callback - INFO - Starting async job submit for node 69c8cdf0f41e4fb6a378685a 2026-03-31 07:00:22,252 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:00:22,266 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8cdf0f41e4fb6a378685a HTTP/1.1" 200 None 2026-03-31 07:00:22,267 - lava_callback - DEBUG - Processing node 69c8cdf0f41e4fb6a378685a: baseline-arm-clabbe 2026-03-31 07:00:22,267 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8cdf0f41e4fb6a378685a 2026-03-31 07:00:22,268 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:00:22,282 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8cdf0f41e4fb6a378685a HTTP/1.1" 200 None 2026-03-31 07:00:22,283 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:00:22,310 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3293e18a3717aa65d01ad HTTP/1.1" 200 None 2026-03-31 07:00:22,311 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:00:22,332 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8cdf0f41e4fb6a378685a HTTP/1.1" 200 None 2026-03-31 07:00:22,337 - lava_callback - ERROR - Error processing callback for node 69c8cdf0f41e4fb6a378685a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:51580 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58876 - "GET / HTTP/1.0" 200 OK 2026-03-31 07:00:42,113 - lava_callback - INFO - Received callback for node 69c8ce15f41e4fb6a3786863 INFO: 172.20.0.1:41986 - "POST /node/69c8ce15f41e4fb6a3786863 HTTP/1.0" 202 Accepted 2026-03-31 07:00:42,119 - lava_callback - INFO - Starting async job submit for node 69c8ce15f41e4fb6a3786863 2026-03-31 07:00:42,120 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:00:42,134 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8ce15f41e4fb6a3786863 HTTP/1.1" 200 None 2026-03-31 07:00:42,135 - lava_callback - DEBUG - Processing node 69c8ce15f41e4fb6a3786863: baseline-arm-clabbe 2026-03-31 07:00:42,136 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8ce15f41e4fb6a3786863 2026-03-31 07:00:42,136 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:00:42,150 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8ce15f41e4fb6a3786863 HTTP/1.1" 200 None 2026-03-31 07:00:42,151 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:00:42,164 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3293e18a3717aa65d01ad HTTP/1.1" 200 None 2026-03-31 07:00:42,165 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:00:42,188 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8ce15f41e4fb6a3786863 HTTP/1.1" 200 None 2026-03-31 07:00:42,189 - lava_callback - ERROR - Error processing callback for node 69c8ce15f41e4fb6a3786863: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:41998 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42000 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49482 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60860 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42948 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42960 - "GET / HTTP/1.0" 200 OK 2026-03-31 07:01:22,170 - lava_callback - INFO - Received callback for node 69c8ce39f41e4fb6a378686d INFO: 172.20.0.1:57686 - "POST /node/69c8ce39f41e4fb6a378686d HTTP/1.0" 202 Accepted 2026-03-31 07:01:22,175 - lava_callback - INFO - Starting async job submit for node 69c8ce39f41e4fb6a378686d 2026-03-31 07:01:22,176 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:01:22,190 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8ce39f41e4fb6a378686d HTTP/1.1" 200 None 2026-03-31 07:01:22,190 - lava_callback - DEBUG - Processing node 69c8ce39f41e4fb6a378686d: baseline-arm-clabbe 2026-03-31 07:01:22,191 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8ce39f41e4fb6a378686d 2026-03-31 07:01:22,191 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:01:22,218 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8ce39f41e4fb6a378686d HTTP/1.1" 200 None 2026-03-31 07:01:22,219 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:01:22,232 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3293e18a3717aa65d01ad HTTP/1.1" 200 None 2026-03-31 07:01:22,234 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:01:22,273 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8ce39f41e4fb6a378686d HTTP/1.1" 200 None 2026-03-31 07:01:22,274 - lava_callback - ERROR - Error processing callback for node 69c8ce39f41e4fb6a378686d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:57690 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53802 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58858 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58874 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45594 - "GET / HTTP/1.0" 200 OK 2026-03-31 07:02:02,171 - lava_callback - INFO - Received callback for node 69c8ce5ef41e4fb6a3786877 INFO: 172.20.0.1:37918 - "POST /node/69c8ce5ef41e4fb6a3786877 HTTP/1.0" 202 Accepted 2026-03-31 07:02:02,176 - lava_callback - INFO - Starting async job submit for node 69c8ce5ef41e4fb6a3786877 2026-03-31 07:02:02,177 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:02:02,192 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8ce5ef41e4fb6a3786877 HTTP/1.1" 200 None 2026-03-31 07:02:02,192 - lava_callback - DEBUG - Processing node 69c8ce5ef41e4fb6a3786877: baseline-arm-clabbe 2026-03-31 07:02:02,193 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8ce5ef41e4fb6a3786877 2026-03-31 07:02:02,193 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:02:02,215 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8ce5ef41e4fb6a3786877 HTTP/1.1" 200 None 2026-03-31 07:02:02,216 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:02:02,229 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3293e18a3717aa65d01ad HTTP/1.1" 200 None 2026-03-31 07:02:02,231 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:02:02,250 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8ce5ef41e4fb6a3786877 HTTP/1.1" 200 None 2026-03-31 07:02:02,255 - lava_callback - ERROR - Error processing callback for node 69c8ce5ef41e4fb6a3786877: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:37920 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37940 - "GET /node/698d50d2820eff0a56c16137%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:37956 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37972 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54008 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41226 - "GET / HTTP/1.0" 200 OK 2026-03-31 07:02:42,209 - lava_callback - INFO - Received callback for node 69c8ce82f41e4fb6a3786887 INFO: 172.20.0.1:37818 - "POST /node/69c8ce82f41e4fb6a3786887 HTTP/1.0" 202 Accepted 2026-03-31 07:02:42,215 - lava_callback - INFO - Starting async job submit for node 69c8ce82f41e4fb6a3786887 2026-03-31 07:02:42,216 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:02:42,233 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8ce82f41e4fb6a3786887 HTTP/1.1" 200 None 2026-03-31 07:02:42,233 - lava_callback - DEBUG - Processing node 69c8ce82f41e4fb6a3786887: baseline-arm-clabbe 2026-03-31 07:02:42,234 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8ce82f41e4fb6a3786887 2026-03-31 07:02:42,234 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:02:42,252 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8ce82f41e4fb6a3786887 HTTP/1.1" 200 None 2026-03-31 07:02:42,254 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:02:42,273 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3293e18a3717aa65d01ad HTTP/1.1" 200 None 2026-03-31 07:02:42,275 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:02:42,310 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8ce82f41e4fb6a3786887 HTTP/1.1" 200 None 2026-03-31 07:02:42,311 - lava_callback - ERROR - Error processing callback for node 69c8ce82f41e4fb6a3786887: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:37828 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37842 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56722 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50820 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47074 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47078 - "GET / HTTP/1.0" 200 OK 2026-03-31 07:03:22,241 - lava_callback - INFO - Received callback for node 69c8cea8f41e4fb6a3786890 INFO: 172.20.0.1:57642 - "POST /node/69c8cea8f41e4fb6a3786890 HTTP/1.0" 202 Accepted 2026-03-31 07:03:22,247 - lava_callback - INFO - Starting async job submit for node 69c8cea8f41e4fb6a3786890 2026-03-31 07:03:22,248 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:03:22,262 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8cea8f41e4fb6a3786890 HTTP/1.1" 200 None 2026-03-31 07:03:22,263 - lava_callback - DEBUG - Processing node 69c8cea8f41e4fb6a3786890: baseline-arm-clabbe 2026-03-31 07:03:22,263 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8cea8f41e4fb6a3786890 2026-03-31 07:03:22,264 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:03:22,277 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8cea8f41e4fb6a3786890 HTTP/1.1" 200 None 2026-03-31 07:03:22,278 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:03:22,292 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3293e18a3717aa65d01ad HTTP/1.1" 200 None 2026-03-31 07:03:22,293 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:03:22,315 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8cea8f41e4fb6a3786890 HTTP/1.1" 200 None 2026-03-31 07:03:22,316 - lava_callback - ERROR - Error processing callback for node 69c8cea8f41e4fb6a3786890: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:57658 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46412 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36714 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36716 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58980 - "GET / HTTP/1.0" 200 OK 2026-03-31 07:04:02,336 - lava_callback - INFO - Received callback for node 69c8cecdf41e4fb6a378689b INFO: 172.20.0.1:33250 - "POST /node/69c8cecdf41e4fb6a378689b HTTP/1.0" 202 Accepted 2026-03-31 07:04:02,343 - lava_callback - INFO - Starting async job submit for node 69c8cecdf41e4fb6a378689b 2026-03-31 07:04:02,344 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:04:02,371 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8cecdf41e4fb6a378689b HTTP/1.1" 200 None 2026-03-31 07:04:02,372 - lava_callback - DEBUG - Processing node 69c8cecdf41e4fb6a378689b: baseline-arm-clabbe 2026-03-31 07:04:02,372 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8cecdf41e4fb6a378689b 2026-03-31 07:04:02,373 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:04:02,387 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8cecdf41e4fb6a378689b HTTP/1.1" 200 None 2026-03-31 07:04:02,389 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:04:02,405 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3293e18a3717aa65d01ad HTTP/1.1" 200 None 2026-03-31 07:04:02,407 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:04:02,443 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8cecdf41e4fb6a378689b HTTP/1.1" 200 None 2026-03-31 07:04:02,444 - lava_callback - ERROR - Error processing callback for node 69c8cecdf41e4fb6a378689b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:33258 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38778 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38792 - "GET / HTTP/1.0" 200 OK 2026-03-31 07:04:22,272 - lava_callback - INFO - Received callback for node 69c8cef1f41e4fb6a37868a5 INFO: 172.20.0.1:58350 - "POST /node/69c8cef1f41e4fb6a37868a5 HTTP/1.0" 202 Accepted 2026-03-31 07:04:22,280 - lava_callback - INFO - Starting async job submit for node 69c8cef1f41e4fb6a37868a5 2026-03-31 07:04:22,282 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:04:22,299 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8cef1f41e4fb6a37868a5 HTTP/1.1" 200 None 2026-03-31 07:04:22,300 - lava_callback - DEBUG - Processing node 69c8cef1f41e4fb6a37868a5: baseline-arm-clabbe 2026-03-31 07:04:22,300 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8cef1f41e4fb6a37868a5 2026-03-31 07:04:22,301 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:04:22,324 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8cef1f41e4fb6a37868a5 HTTP/1.1" 200 None 2026-03-31 07:04:22,326 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:04:22,341 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3293e18a3717aa65d01ad HTTP/1.1" 200 None 2026-03-31 07:04:22,342 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:04:22,369 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8cef1f41e4fb6a37868a5 HTTP/1.1" 200 None 2026-03-31 07:04:22,371 - lava_callback - ERROR - Error processing callback for node 69c8cef1f41e4fb6a37868a5: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:58366 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36930 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47316 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47332 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44936 - "GET / HTTP/1.0" 200 OK 2026-03-31 07:05:02,358 - lava_callback - INFO - Received callback for node 69c8cf15f41e4fb6a37868b0 INFO: 172.20.0.1:39050 - "POST /node/69c8cf15f41e4fb6a37868b0 HTTP/1.0" 202 Accepted 2026-03-31 07:05:02,364 - lava_callback - INFO - Starting async job submit for node 69c8cf15f41e4fb6a37868b0 2026-03-31 07:05:02,365 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:05:02,380 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8cf15f41e4fb6a37868b0 HTTP/1.1" 200 None 2026-03-31 07:05:02,380 - lava_callback - DEBUG - Processing node 69c8cf15f41e4fb6a37868b0: baseline-arm-clabbe 2026-03-31 07:05:02,381 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8cf15f41e4fb6a37868b0 2026-03-31 07:05:02,381 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:05:02,395 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8cf15f41e4fb6a37868b0 HTTP/1.1" 200 None 2026-03-31 07:05:02,396 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:05:02,425 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3293e18a3717aa65d01ad HTTP/1.1" 200 None 2026-03-31 07:05:02,426 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:05:02,448 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8cf15f41e4fb6a37868b0 HTTP/1.1" 200 None 2026-03-31 07:05:02,450 - lava_callback - ERROR - Error processing callback for node 69c8cf15f41e4fb6a37868b0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:39058 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39074 - "GET /node/693b070cfee8383e92b70a6b%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:58334 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58346 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42974 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59486 - "GET / HTTP/1.0" 200 OK 2026-03-31 07:05:42,364 - lava_callback - INFO - Received callback for node 69c8cf39f41e4fb6a37868bf INFO: 172.20.0.1:37700 - "POST /node/69c8cf39f41e4fb6a37868bf HTTP/1.0" 202 Accepted 2026-03-31 07:05:42,371 - lava_callback - INFO - Starting async job submit for node 69c8cf39f41e4fb6a37868bf 2026-03-31 07:05:42,372 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:05:42,386 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8cf39f41e4fb6a37868bf HTTP/1.1" 200 None 2026-03-31 07:05:42,387 - lava_callback - DEBUG - Processing node 69c8cf39f41e4fb6a37868bf: baseline-arm-clabbe 2026-03-31 07:05:42,388 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8cf39f41e4fb6a37868bf 2026-03-31 07:05:42,388 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:05:42,404 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8cf39f41e4fb6a37868bf HTTP/1.1" 200 None 2026-03-31 07:05:42,405 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:05:42,434 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3293e18a3717aa65d01ad HTTP/1.1" 200 None 2026-03-31 07:05:42,436 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:05:42,457 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8cf39f41e4fb6a37868bf HTTP/1.1" 200 None 2026-03-31 07:05:42,463 - lava_callback - ERROR - Error processing callback for node 69c8cf39f41e4fb6a37868bf: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:37702 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37704 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58820 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60230 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48708 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48718 - "GET / HTTP/1.0" 200 OK 2026-03-31 07:06:22,407 - lava_callback - INFO - Received callback for node 69c8cf5ef41e4fb6a37868c9 INFO: 172.20.0.1:33684 - "POST /node/69c8cf5ef41e4fb6a37868c9 HTTP/1.0" 202 Accepted 2026-03-31 07:06:22,415 - lava_callback - INFO - Starting async job submit for node 69c8cf5ef41e4fb6a37868c9 2026-03-31 07:06:22,416 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:06:22,437 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8cf5ef41e4fb6a37868c9 HTTP/1.1" 200 None 2026-03-31 07:06:22,438 - lava_callback - DEBUG - Processing node 69c8cf5ef41e4fb6a37868c9: baseline-arm-clabbe 2026-03-31 07:06:22,438 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8cf5ef41e4fb6a37868c9 2026-03-31 07:06:22,438 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:06:22,452 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8cf5ef41e4fb6a37868c9 HTTP/1.1" 200 None 2026-03-31 07:06:22,454 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:06:22,467 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3293e18a3717aa65d01ad HTTP/1.1" 200 None 2026-03-31 07:06:22,468 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:06:22,490 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8cf5ef41e4fb6a37868c9 HTTP/1.1" 200 None 2026-03-31 07:06:22,498 - lava_callback - ERROR - Error processing callback for node 69c8cf5ef41e4fb6a37868c9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:33698 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54120 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47448 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47462 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35086 - "GET / HTTP/1.0" 200 OK 2026-03-31 07:07:01,283 - lava_callback - INFO - Received callback for node 69c8cf87f41e4fb6a37868d6 INFO: 172.20.0.1:38890 - "POST /node/69c8cf87f41e4fb6a37868d6 HTTP/1.0" 202 Accepted 2026-03-31 07:07:01,289 - lava_callback - INFO - Starting async job submit for node 69c8cf87f41e4fb6a37868d6 2026-03-31 07:07:01,290 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:07:01,305 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8cf87f41e4fb6a37868d6 HTTP/1.1" 200 None 2026-03-31 07:07:01,306 - lava_callback - DEBUG - Processing node 69c8cf87f41e4fb6a37868d6: baseline-arm64-baylibre 2026-03-31 07:07:01,306 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8cf87f41e4fb6a37868d6 2026-03-31 07:07:01,307 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:07:01,320 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8cf87f41e4fb6a37868d6 HTTP/1.1" 200 None 2026-03-31 07:07:01,322 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:07:01,335 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-03-31 07:07:01,336 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:07:01,359 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8cf87f41e4fb6a37868d6 HTTP/1.1" 200 None 2026-03-31 07:07:01,360 - lava_callback - ERROR - Error processing callback for node 69c8cf87f41e4fb6a37868d6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:38904 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33108 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33116 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40946 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60074 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60672 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60678 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45494 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55108 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55346 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55362 - "GET / HTTP/1.0" 200 OK 2026-03-31 07:08:21,345 - lava_callback - INFO - Received callback for node 69c8cfd2f41e4fb6a37868e3 INFO: 172.20.0.1:57182 - "POST /node/69c8cfd2f41e4fb6a37868e3 HTTP/1.0" 202 Accepted 2026-03-31 07:08:21,352 - lava_callback - INFO - Starting async job submit for node 69c8cfd2f41e4fb6a37868e3 2026-03-31 07:08:21,353 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:08:21,370 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8cfd2f41e4fb6a37868e3 HTTP/1.1" 200 None 2026-03-31 07:08:21,371 - lava_callback - DEBUG - Processing node 69c8cfd2f41e4fb6a37868e3: baseline-arm64-baylibre 2026-03-31 07:08:21,371 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8cfd2f41e4fb6a37868e3 2026-03-31 07:08:21,372 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:08:21,385 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8cfd2f41e4fb6a37868e3 HTTP/1.1" 200 None 2026-03-31 07:08:21,386 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:08:21,400 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-03-31 07:08:21,402 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:08:21,427 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8cfd2f41e4fb6a37868e3 HTTP/1.1" 200 None 2026-03-31 07:08:21,433 - lava_callback - ERROR - Error processing callback for node 69c8cfd2f41e4fb6a37868e3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:57198 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52404 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36366 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36376 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36388 - "GET /node/6988cb0f9fc35f969e1ef168%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:36396 - "GET /node/6992811d98a8bdb31c395a4c%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:43296 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60014 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47386 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47396 - "GET / HTTP/1.0" 200 OK 2026-03-31 07:09:21,343 - lava_callback - INFO - Received callback for node 69c8d01df41e4fb6a37868ee INFO: 172.20.0.1:47738 - "POST /node/69c8d01df41e4fb6a37868ee HTTP/1.0" 202 Accepted 2026-03-31 07:09:21,350 - lava_callback - INFO - Starting async job submit for node 69c8d01df41e4fb6a37868ee 2026-03-31 07:09:21,351 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:09:21,371 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8d01df41e4fb6a37868ee HTTP/1.1" 200 None 2026-03-31 07:09:21,372 - lava_callback - DEBUG - Processing node 69c8d01df41e4fb6a37868ee: baseline-arm64-baylibre 2026-03-31 07:09:21,373 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8d01df41e4fb6a37868ee 2026-03-31 07:09:21,374 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:09:21,387 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8d01df41e4fb6a37868ee HTTP/1.1" 200 None 2026-03-31 07:09:21,388 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:09:21,403 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-03-31 07:09:21,404 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:09:21,426 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8d01df41e4fb6a37868ee HTTP/1.1" 200 None 2026-03-31 07:09:21,447 - lava_callback - ERROR - Error processing callback for node 69c8d01df41e4fb6a37868ee: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:47748 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47606 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55240 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55250 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43190 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54818 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55296 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55312 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60042 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60056 - "GET /node/6947779e4917c2d89a1ed570%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:60068 - "GET /node/6985d90871989fb14149a88a%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:53052 - "GET / HTTP/1.0" 200 OK 2026-03-31 07:10:41,422 - lava_callback - INFO - Received callback for node 69c8d069f41e4fb6a37868fe INFO: 172.20.0.1:46778 - "POST /node/69c8d069f41e4fb6a37868fe HTTP/1.0" 202 Accepted 2026-03-31 07:10:41,428 - lava_callback - INFO - Starting async job submit for node 69c8d069f41e4fb6a37868fe 2026-03-31 07:10:41,429 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:10:41,443 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8d069f41e4fb6a37868fe HTTP/1.1" 200 None 2026-03-31 07:10:41,444 - lava_callback - DEBUG - Processing node 69c8d069f41e4fb6a37868fe: baseline-arm64-baylibre 2026-03-31 07:10:41,444 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8d069f41e4fb6a37868fe 2026-03-31 07:10:41,445 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:10:41,459 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8d069f41e4fb6a37868fe HTTP/1.1" 200 None 2026-03-31 07:10:41,460 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:10:41,511 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-03-31 07:10:41,513 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:10:41,533 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8d069f41e4fb6a37868fe HTTP/1.1" 200 None 2026-03-31 07:10:41,538 - lava_callback - ERROR - Error processing callback for node 69c8d069f41e4fb6a37868fe: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:46794 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46802 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50810 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51840 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37824 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37828 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57294 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47038 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49988 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49992 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37454 - "GET / HTTP/1.0" 200 OK 2026-03-31 07:12:01,543 - lava_callback - INFO - Received callback for node 69c8d0b1f41e4fb6a378690a INFO: 172.20.0.1:43748 - "POST /node/69c8d0b1f41e4fb6a378690a HTTP/1.0" 202 Accepted 2026-03-31 07:12:01,549 - lava_callback - INFO - Starting async job submit for node 69c8d0b1f41e4fb6a378690a 2026-03-31 07:12:01,550 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:12:01,565 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8d0b1f41e4fb6a378690a HTTP/1.1" 200 None 2026-03-31 07:12:01,565 - lava_callback - DEBUG - Processing node 69c8d0b1f41e4fb6a378690a: baseline-arm64-baylibre 2026-03-31 07:12:01,566 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8d0b1f41e4fb6a378690a 2026-03-31 07:12:01,566 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:12:01,580 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8d0b1f41e4fb6a378690a HTTP/1.1" 200 None 2026-03-31 07:12:01,582 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:12:01,595 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-03-31 07:12:01,596 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:12:01,617 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8d0b1f41e4fb6a378690a HTTP/1.1" 200 None 2026-03-31 07:12:01,623 - lava_callback - ERROR - Error processing callback for node 69c8d0b1f41e4fb6a378690a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:43752 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42998 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43002 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46468 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59918 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50234 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50242 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54126 - "GET / HTTP/1.0" 200 OK 2026-03-31 07:13:01,549 - lava_callback - INFO - Received callback for node 69c8d0fcf41e4fb6a3786919 INFO: 172.20.0.1:58132 - "POST /node/69c8d0fcf41e4fb6a3786919 HTTP/1.0" 202 Accepted 2026-03-31 07:13:01,555 - lava_callback - INFO - Starting async job submit for node 69c8d0fcf41e4fb6a3786919 2026-03-31 07:13:01,556 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:13:01,571 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8d0fcf41e4fb6a3786919 HTTP/1.1" 200 None 2026-03-31 07:13:01,572 - lava_callback - DEBUG - Processing node 69c8d0fcf41e4fb6a3786919: baseline-arm64-baylibre 2026-03-31 07:13:01,573 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8d0fcf41e4fb6a3786919 2026-03-31 07:13:01,573 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:13:01,594 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8d0fcf41e4fb6a3786919 HTTP/1.1" 200 None 2026-03-31 07:13:01,596 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:13:01,610 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-03-31 07:13:01,612 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:13:01,633 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8d0fcf41e4fb6a3786919 HTTP/1.1" 200 None 2026-03-31 07:13:01,643 - lava_callback - ERROR - Error processing callback for node 69c8d0fcf41e4fb6a3786919: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:58148 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32778 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:32786 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35604 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51214 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59998 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60012 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59882 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51896 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37230 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37236 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46992 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38072 - "GET / HTTP/1.0" 200 OK 2026-03-31 07:14:42,786 - lava_callback - INFO - Received callback for node 69c8d14df41e4fb6a3786923 INFO: 172.20.0.1:57568 - "POST /node/69c8d14df41e4fb6a3786923 HTTP/1.0" 202 Accepted 2026-03-31 07:14:42,791 - lava_callback - INFO - Starting async job submit for node 69c8d14df41e4fb6a3786923 2026-03-31 07:14:42,792 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:14:42,807 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8d14df41e4fb6a3786923 HTTP/1.1" 200 None 2026-03-31 07:14:42,808 - lava_callback - DEBUG - Processing node 69c8d14df41e4fb6a3786923: baseline-arm64-clabbe 2026-03-31 07:14:42,809 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8d14df41e4fb6a3786923 2026-03-31 07:14:42,809 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:14:42,824 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8d14df41e4fb6a3786923 HTTP/1.1" 200 None 2026-03-31 07:14:42,826 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:14:42,840 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-03-31 07:14:42,842 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:14:42,864 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8d14df41e4fb6a3786923 HTTP/1.1" 200 None 2026-03-31 07:14:42,865 - lava_callback - ERROR - Error processing callback for node 69c8d14df41e4fb6a3786923: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:57578 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57580 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34450 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43318 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48008 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48012 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41742 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55490 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36482 - "GET /node/6910a8b08b15debe9dd52c16%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:36492 - "GET /node/691904709c0fefc407585d70%5Cn HTTP/1.0" 405 Method Not Allowed 2026-03-31 07:15:42,773 - lava_callback - INFO - Received callback for node 69c8d199f41e4fb6a3786934 INFO: 172.20.0.1:36506 - "POST /node/69c8d199f41e4fb6a3786934 HTTP/1.0" 202 Accepted 2026-03-31 07:15:42,779 - lava_callback - INFO - Starting async job submit for node 69c8d199f41e4fb6a3786934 2026-03-31 07:15:42,780 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:15:42,820 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8d199f41e4fb6a3786934 HTTP/1.1" 200 None 2026-03-31 07:15:42,821 - lava_callback - DEBUG - Processing node 69c8d199f41e4fb6a3786934: baseline-arm64-clabbe 2026-03-31 07:15:42,821 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8d199f41e4fb6a3786934 2026-03-31 07:15:42,822 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:15:42,835 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8d199f41e4fb6a3786934 HTTP/1.1" 200 None 2026-03-31 07:15:42,836 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:15:42,853 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-03-31 07:15:42,855 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:15:42,879 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8d199f41e4fb6a3786934 HTTP/1.1" 200 None 2026-03-31 07:15:42,883 - lava_callback - ERROR - Error processing callback for node 69c8d199f41e4fb6a3786934: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:36510 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36512 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39706 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45350 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51364 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51366 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54060 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35064 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35068 - "GET /node/6988bc7a9fc35f969e1ece51%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:35070 - "GET /node/693af480fee8383e92b6aed0%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:54980 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54996 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40818 - "GET / HTTP/1.0" 200 OK 2026-03-31 07:17:02,826 - lava_callback - INFO - Received callback for node 69c8d1e6f41e4fb6a378693e INFO: 172.20.0.1:60212 - "POST /node/69c8d1e6f41e4fb6a378693e HTTP/1.0" 202 Accepted 2026-03-31 07:17:02,832 - lava_callback - INFO - Starting async job submit for node 69c8d1e6f41e4fb6a378693e 2026-03-31 07:17:02,833 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:17:02,846 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8d1e6f41e4fb6a378693e HTTP/1.1" 200 None 2026-03-31 07:17:02,847 - lava_callback - DEBUG - Processing node 69c8d1e6f41e4fb6a378693e: baseline-arm64-clabbe 2026-03-31 07:17:02,847 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8d1e6f41e4fb6a378693e 2026-03-31 07:17:02,848 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:17:02,861 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8d1e6f41e4fb6a378693e HTTP/1.1" 200 None 2026-03-31 07:17:02,863 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:17:02,884 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-03-31 07:17:02,886 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:17:02,906 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8d1e6f41e4fb6a378693e HTTP/1.1" 200 None 2026-03-31 07:17:02,908 - lava_callback - ERROR - Error processing callback for node 69c8d1e6f41e4fb6a378693e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:60214 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60218 - "GET /node/698a177222b4cc0f227ae1db%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:52756 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52770 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39314 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45462 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40060 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40064 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45296 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40666 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51408 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51418 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45038 - "GET /node/6983f228969566dcb0d2a7ec%5Cn HTTP/1.0" 405 Method Not Allowed 2026-03-31 07:18:21,998 - lava_callback - INFO - Received callback for node 69c8d22cf41e4fb6a378694e INFO: 172.20.0.1:45040 - "POST /node/69c8d22cf41e4fb6a378694e HTTP/1.0" 202 Accepted 2026-03-31 07:18:22,005 - lava_callback - INFO - Starting async job submit for node 69c8d22cf41e4fb6a378694e 2026-03-31 07:18:22,006 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:18:22,029 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8d22cf41e4fb6a378694e HTTP/1.1" 200 None 2026-03-31 07:18:22,031 - lava_callback - DEBUG - Processing node 69c8d22cf41e4fb6a378694e: baseline-arm64-baylibre 2026-03-31 07:18:22,031 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8d22cf41e4fb6a378694e 2026-03-31 07:18:22,032 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:18:22,051 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8d22cf41e4fb6a378694e HTTP/1.1" 200 None 2026-03-31 07:18:22,053 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:18:22,072 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-03-31 07:18:22,074 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:18:22,096 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8d22cf41e4fb6a378694e HTTP/1.1" 200 None 2026-03-31 07:18:22,103 - lava_callback - ERROR - Error processing callback for node 69c8d22cf41e4fb6a378694e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:45056 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60958 - "GET /node/694ac4b16c96ed51d316f105%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:60966 - "GET /node/69935092523507f3806a69d1%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:60982 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56220 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56228 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52926 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41076 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34330 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34336 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34352 - "GET /node/6988c04f9fc35f969e1ed7b0%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:34404 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50328 - "GET / HTTP/1.0" 200 OK 2026-03-31 07:19:42,878 - lava_callback - INFO - Received callback for node 69c8d27ef41e4fb6a3786961 INFO: 172.20.0.1:58960 - "POST /node/69c8d27ef41e4fb6a3786961 HTTP/1.0" 202 Accepted 2026-03-31 07:19:42,883 - lava_callback - INFO - Starting async job submit for node 69c8d27ef41e4fb6a3786961 2026-03-31 07:19:42,884 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:19:42,898 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8d27ef41e4fb6a3786961 HTTP/1.1" 200 None 2026-03-31 07:19:42,899 - lava_callback - DEBUG - Processing node 69c8d27ef41e4fb6a3786961: baseline-arm64-clabbe 2026-03-31 07:19:42,899 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8d27ef41e4fb6a3786961 2026-03-31 07:19:42,900 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:19:42,913 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8d27ef41e4fb6a3786961 HTTP/1.1" 200 None 2026-03-31 07:19:42,915 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:19:42,928 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-03-31 07:19:42,930 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:19:42,949 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8d27ef41e4fb6a3786961 HTTP/1.1" 200 None 2026-03-31 07:19:42,950 - lava_callback - ERROR - Error processing callback for node 69c8d27ef41e4fb6a3786961: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:58976 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58992 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45920 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55086 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47054 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47056 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39274 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37576 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55660 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55670 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32818 - "GET / HTTP/1.0" 200 OK 2026-03-31 07:21:02,867 - lava_callback - INFO - Received callback for node 69c8d2ccf41e4fb6a378696b INFO: 172.20.0.1:52326 - "POST /node/69c8d2ccf41e4fb6a378696b HTTP/1.0" 202 Accepted 2026-03-31 07:21:02,872 - lava_callback - INFO - Starting async job submit for node 69c8d2ccf41e4fb6a378696b 2026-03-31 07:21:02,873 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:21:02,889 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8d2ccf41e4fb6a378696b HTTP/1.1" 200 None 2026-03-31 07:21:02,890 - lava_callback - DEBUG - Processing node 69c8d2ccf41e4fb6a378696b: baseline-arm64-clabbe 2026-03-31 07:21:02,890 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8d2ccf41e4fb6a378696b 2026-03-31 07:21:02,890 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:21:02,906 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8d2ccf41e4fb6a378696b HTTP/1.1" 200 None 2026-03-31 07:21:02,907 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:21:02,920 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-03-31 07:21:02,922 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:21:02,954 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8d2ccf41e4fb6a378696b HTTP/1.1" 200 None 2026-03-31 07:21:02,959 - lava_callback - ERROR - Error processing callback for node 69c8d2ccf41e4fb6a378696b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:52332 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33636 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33650 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53998 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42956 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45290 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45300 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33620 - "GET / HTTP/1.0" 200 OK 2026-03-31 07:22:02,937 - lava_callback - INFO - Received callback for node 69c8d318f41e4fb6a378697e INFO: 172.20.0.1:43356 - "POST /node/69c8d318f41e4fb6a378697e HTTP/1.0" 202 Accepted 2026-03-31 07:22:02,943 - lava_callback - INFO - Starting async job submit for node 69c8d318f41e4fb6a378697e 2026-03-31 07:22:02,944 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:22:02,958 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8d318f41e4fb6a378697e HTTP/1.1" 200 None 2026-03-31 07:22:02,959 - lava_callback - DEBUG - Processing node 69c8d318f41e4fb6a378697e: baseline-arm64-clabbe 2026-03-31 07:22:02,959 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8d318f41e4fb6a378697e 2026-03-31 07:22:02,960 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:22:02,973 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8d318f41e4fb6a378697e HTTP/1.1" 200 None 2026-03-31 07:22:02,975 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:22:02,989 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-03-31 07:22:02,990 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:22:03,012 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8d318f41e4fb6a378697e HTTP/1.1" 200 None 2026-03-31 07:22:03,018 - lava_callback - ERROR - Error processing callback for node 69c8d318f41e4fb6a378697e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:43364 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58464 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58476 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39468 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36838 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40816 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40818 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33552 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41244 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35588 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35598 - "GET / HTTP/1.0" 200 OK 2026-03-31 07:23:22,982 - lava_callback - INFO - Received callback for node 69c8d364f41e4fb6a3786986 INFO: 172.20.0.1:41178 - "POST /node/69c8d364f41e4fb6a3786986 HTTP/1.0" 202 Accepted 2026-03-31 07:23:22,988 - lava_callback - INFO - Starting async job submit for node 69c8d364f41e4fb6a3786986 2026-03-31 07:23:22,989 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:23:23,003 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8d364f41e4fb6a3786986 HTTP/1.1" 200 None 2026-03-31 07:23:23,003 - lava_callback - DEBUG - Processing node 69c8d364f41e4fb6a3786986: baseline-arm64-clabbe 2026-03-31 07:23:23,004 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8d364f41e4fb6a3786986 2026-03-31 07:23:23,004 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:23:23,031 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8d364f41e4fb6a3786986 HTTP/1.1" 200 None 2026-03-31 07:23:23,032 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:23:23,045 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-03-31 07:23:23,046 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:23:23,073 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8d364f41e4fb6a3786986 HTTP/1.1" 200 None 2026-03-31 07:23:23,075 - lava_callback - ERROR - Error processing callback for node 69c8d364f41e4fb6a3786986: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:41194 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44934 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49972 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49982 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53954 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45046 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56538 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56554 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57642 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45982 - "GET / HTTP/1.0" 200 OK 2026-03-31 07:24:43,186 - lava_callback - INFO - Received callback for node 69c8d3b0f41e4fb6a3786994 INFO: 172.20.0.1:48382 - "POST /node/69c8d3b0f41e4fb6a3786994 HTTP/1.0" 202 Accepted 2026-03-31 07:24:43,191 - lava_callback - INFO - Starting async job submit for node 69c8d3b0f41e4fb6a3786994 2026-03-31 07:24:43,192 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:24:43,212 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8d3b0f41e4fb6a3786994 HTTP/1.1" 200 None 2026-03-31 07:24:43,212 - lava_callback - DEBUG - Processing node 69c8d3b0f41e4fb6a3786994: baseline-arm64-clabbe 2026-03-31 07:24:43,213 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8d3b0f41e4fb6a3786994 2026-03-31 07:24:43,213 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:24:43,226 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8d3b0f41e4fb6a3786994 HTTP/1.1" 200 None 2026-03-31 07:24:43,228 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:24:43,242 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-03-31 07:24:43,243 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:24:43,264 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8d3b0f41e4fb6a3786994 HTTP/1.1" 200 None 2026-03-31 07:24:43,270 - lava_callback - ERROR - Error processing callback for node 69c8d3b0f41e4fb6a3786994: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:48390 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48398 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45984 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54970 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38184 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38196 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33160 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33802 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36308 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36314 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58702 - "GET / HTTP/1.0" 200 OK 2026-03-31 07:26:03,246 - lava_callback - INFO - Received callback for node 69c8d3fbf41e4fb6a378699b INFO: 172.20.0.1:44160 - "POST /node/69c8d3fbf41e4fb6a378699b HTTP/1.0" 202 Accepted 2026-03-31 07:26:03,251 - lava_callback - INFO - Starting async job submit for node 69c8d3fbf41e4fb6a378699b 2026-03-31 07:26:03,252 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:26:03,274 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8d3fbf41e4fb6a378699b HTTP/1.1" 200 None 2026-03-31 07:26:03,275 - lava_callback - DEBUG - Processing node 69c8d3fbf41e4fb6a378699b: baseline-arm64-clabbe 2026-03-31 07:26:03,275 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8d3fbf41e4fb6a378699b 2026-03-31 07:26:03,276 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:26:03,289 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8d3fbf41e4fb6a378699b HTTP/1.1" 200 None 2026-03-31 07:26:03,290 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:26:03,304 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-03-31 07:26:03,306 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:26:03,335 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8d3fbf41e4fb6a378699b HTTP/1.1" 200 None 2026-03-31 07:26:03,337 - lava_callback - ERROR - Error processing callback for node 69c8d3fbf41e4fb6a378699b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:44170 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56478 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56492 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53908 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41356 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60686 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60694 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34718 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48968 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52712 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52714 - "GET / HTTP/1.0" 200 OK 2026-03-31 07:27:23,250 - lava_callback - INFO - Received callback for node 69c8d447f41e4fb6a37869a7 INFO: 172.20.0.1:35148 - "POST /node/69c8d447f41e4fb6a37869a7 HTTP/1.0" 202 Accepted 2026-03-31 07:27:23,255 - lava_callback - INFO - Starting async job submit for node 69c8d447f41e4fb6a37869a7 2026-03-31 07:27:23,256 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:27:23,279 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8d447f41e4fb6a37869a7 HTTP/1.1" 200 None 2026-03-31 07:27:23,280 - lava_callback - DEBUG - Processing node 69c8d447f41e4fb6a37869a7: baseline-arm64-clabbe 2026-03-31 07:27:23,280 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8d447f41e4fb6a37869a7 2026-03-31 07:27:23,280 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:27:23,294 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8d447f41e4fb6a37869a7 HTTP/1.1" 200 None 2026-03-31 07:27:23,295 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:27:23,309 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-03-31 07:27:23,310 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:27:23,331 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8d447f41e4fb6a37869a7 HTTP/1.1" 200 None 2026-03-31 07:27:23,333 - lava_callback - ERROR - Error processing callback for node 69c8d447f41e4fb6a37869a7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:35154 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34992 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47248 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47250 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51140 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33100 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46174 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46178 - "GET / HTTP/1.0" 200 OK 2026-03-31 07:28:23,277 - lava_callback - INFO - Received callback for node 69c8d494f41e4fb6a37869ad INFO: 172.20.0.1:49484 - "POST /node/69c8d494f41e4fb6a37869ad HTTP/1.0" 202 Accepted 2026-03-31 07:28:23,282 - lava_callback - INFO - Starting async job submit for node 69c8d494f41e4fb6a37869ad 2026-03-31 07:28:23,283 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:28:23,309 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8d494f41e4fb6a37869ad HTTP/1.1" 200 None 2026-03-31 07:28:23,310 - lava_callback - DEBUG - Processing node 69c8d494f41e4fb6a37869ad: baseline-arm64-clabbe 2026-03-31 07:28:23,310 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8d494f41e4fb6a37869ad 2026-03-31 07:28:23,311 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:28:23,325 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8d494f41e4fb6a37869ad HTTP/1.1" 200 None 2026-03-31 07:28:23,326 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:28:23,340 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-03-31 07:28:23,341 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:28:23,372 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8d494f41e4fb6a37869ad HTTP/1.1" 200 None 2026-03-31 07:28:23,378 - lava_callback - ERROR - Error processing callback for node 69c8d494f41e4fb6a37869ad: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:49498 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38002 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54860 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54862 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34456 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36898 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35556 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35558 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50234 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39240 - "GET / HTTP/1.0" 200 OK 2026-03-31 07:29:43,351 - lava_callback - INFO - Received callback for node 69c8d4e1f41e4fb6a37869bd INFO: 172.20.0.1:52402 - "POST /node/69c8d4e1f41e4fb6a37869bd HTTP/1.0" 202 Accepted 2026-03-31 07:29:43,357 - lava_callback - INFO - Starting async job submit for node 69c8d4e1f41e4fb6a37869bd 2026-03-31 07:29:43,358 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:29:43,372 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8d4e1f41e4fb6a37869bd HTTP/1.1" 200 None 2026-03-31 07:29:43,373 - lava_callback - DEBUG - Processing node 69c8d4e1f41e4fb6a37869bd: baseline-arm64-clabbe 2026-03-31 07:29:43,373 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8d4e1f41e4fb6a37869bd 2026-03-31 07:29:43,374 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:29:43,386 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8d4e1f41e4fb6a37869bd HTTP/1.1" 200 None 2026-03-31 07:29:43,388 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:29:43,403 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-03-31 07:29:43,404 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:29:43,424 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8d4e1f41e4fb6a37869bd HTTP/1.1" 200 None 2026-03-31 07:29:43,431 - lava_callback - ERROR - Error processing callback for node 69c8d4e1f41e4fb6a37869bd: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:52418 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52432 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51502 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35230 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60504 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60508 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48636 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45096 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49950 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49956 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55864 - "GET / HTTP/1.0" 200 OK 2026-03-31 07:31:02,775 - lava_callback - INFO - Received callback for node 69c8d525f41e4fb6a37869c9 INFO: 172.20.0.1:50004 - "POST /node/69c8d525f41e4fb6a37869c9 HTTP/1.0" 202 Accepted 2026-03-31 07:31:02,781 - lava_callback - INFO - Starting async job submit for node 69c8d525f41e4fb6a37869c9 2026-03-31 07:31:02,782 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:31:02,797 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8d525f41e4fb6a37869c9 HTTP/1.1" 200 None 2026-03-31 07:31:02,798 - lava_callback - DEBUG - Processing node 69c8d525f41e4fb6a37869c9: baseline-arm64-baylibre 2026-03-31 07:31:02,798 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8d525f41e4fb6a37869c9 2026-03-31 07:31:02,799 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:31:02,813 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8d525f41e4fb6a37869c9 HTTP/1.1" 200 None 2026-03-31 07:31:02,815 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:31:02,837 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3295318a3717aa65d01c3 HTTP/1.1" 200 None 2026-03-31 07:31:02,839 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:31:02,870 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8d525f41e4fb6a37869c9 HTTP/1.1" 200 None 2026-03-31 07:31:02,872 - lava_callback - ERROR - Error processing callback for node 69c8d525f41e4fb6a37869c9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:50016 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37612 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37622 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37594 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56076 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45682 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45686 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53594 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54258 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59122 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59128 - "GET / HTTP/1.0" 200 OK 2026-03-31 07:32:22,562 - lava_callback - INFO - Received callback for node 69c8d571f41e4fb6a37869d9 INFO: 172.20.0.1:43354 - "POST /node/69c8d571f41e4fb6a37869d9 HTTP/1.0" 202 Accepted 2026-03-31 07:32:22,567 - lava_callback - INFO - Starting async job submit for node 69c8d571f41e4fb6a37869d9 2026-03-31 07:32:22,568 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:32:22,604 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8d571f41e4fb6a37869d9 HTTP/1.1" 200 None 2026-03-31 07:32:22,605 - lava_callback - DEBUG - Processing node 69c8d571f41e4fb6a37869d9: baseline-arm64-baylibre 2026-03-31 07:32:22,605 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8d571f41e4fb6a37869d9 2026-03-31 07:32:22,606 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:32:22,625 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8d571f41e4fb6a37869d9 HTTP/1.1" 200 None 2026-03-31 07:32:22,627 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:32:22,643 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3295318a3717aa65d01c3 HTTP/1.1" 200 None 2026-03-31 07:32:22,645 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:32:22,674 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8d571f41e4fb6a37869d9 HTTP/1.1" 200 None 2026-03-31 07:32:22,676 - lava_callback - ERROR - Error processing callback for node 69c8d571f41e4fb6a37869d9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:43364 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40944 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54056 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54062 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36322 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53188 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52790 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52806 - "GET / HTTP/1.0" 200 OK 2026-03-31 07:33:22,555 - lava_callback - INFO - Received callback for node 69c8d5bcf41e4fb6a37869e3 INFO: 172.20.0.1:33160 - "POST /node/69c8d5bcf41e4fb6a37869e3 HTTP/1.0" 202 Accepted 2026-03-31 07:33:22,560 - lava_callback - INFO - Starting async job submit for node 69c8d5bcf41e4fb6a37869e3 2026-03-31 07:33:22,561 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:33:22,576 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8d5bcf41e4fb6a37869e3 HTTP/1.1" 200 None 2026-03-31 07:33:22,577 - lava_callback - DEBUG - Processing node 69c8d5bcf41e4fb6a37869e3: baseline-arm64-baylibre 2026-03-31 07:33:22,577 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8d5bcf41e4fb6a37869e3 2026-03-31 07:33:22,578 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:33:22,591 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8d5bcf41e4fb6a37869e3 HTTP/1.1" 200 None 2026-03-31 07:33:22,592 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:33:22,605 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3295318a3717aa65d01c3 HTTP/1.1" 200 None 2026-03-31 07:33:22,606 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:33:22,626 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8d5bcf41e4fb6a37869e3 HTTP/1.1" 200 None 2026-03-31 07:33:22,632 - lava_callback - ERROR - Error processing callback for node 69c8d5bcf41e4fb6a37869e3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:33166 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51820 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55226 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55234 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33804 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47894 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42718 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42726 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59664 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60052 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39776 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39792 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45784 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57942 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55360 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55370 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53042 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50490 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57524 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57534 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33218 - "GET / HTTP/1.0" 200 OK 2026-03-31 07:36:02,728 - lava_callback - INFO - Received callback for node 69c8d651f41e4fb6a37869f7 INFO: 172.20.0.1:59184 - "POST /node/69c8d651f41e4fb6a37869f7 HTTP/1.0" 202 Accepted 2026-03-31 07:36:02,733 - lava_callback - INFO - Starting async job submit for node 69c8d651f41e4fb6a37869f7 2026-03-31 07:36:02,734 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:36:02,749 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8d651f41e4fb6a37869f7 HTTP/1.1" 200 None 2026-03-31 07:36:02,750 - lava_callback - DEBUG - Processing node 69c8d651f41e4fb6a37869f7: baseline-arm64-baylibre 2026-03-31 07:36:02,750 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8d651f41e4fb6a37869f7 2026-03-31 07:36:02,751 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:36:02,764 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8d651f41e4fb6a37869f7 HTTP/1.1" 200 None 2026-03-31 07:36:02,766 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:36:02,778 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3295318a3717aa65d01c3 HTTP/1.1" 200 None 2026-03-31 07:36:02,780 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:36:02,800 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8d651f41e4fb6a37869f7 HTTP/1.1" 200 None 2026-03-31 07:36:02,802 - lava_callback - ERROR - Error processing callback for node 69c8d651f41e4fb6a37869f7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:59194 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57882 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57890 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40568 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55716 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42296 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42302 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47894 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49910 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35206 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35210 - "GET / HTTP/1.0" 200 OK 2026-03-31 07:37:22,922 - lava_callback - INFO - Received callback for node 69c8d69ff41e4fb6a3786a05 INFO: 172.20.0.1:34634 - "POST /node/69c8d69ff41e4fb6a3786a05 HTTP/1.0" 202 Accepted 2026-03-31 07:37:22,928 - lava_callback - INFO - Starting async job submit for node 69c8d69ff41e4fb6a3786a05 2026-03-31 07:37:22,929 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:37:22,944 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8d69ff41e4fb6a3786a05 HTTP/1.1" 200 None 2026-03-31 07:37:22,945 - lava_callback - DEBUG - Processing node 69c8d69ff41e4fb6a3786a05: baseline-arm64-baylibre 2026-03-31 07:37:22,945 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8d69ff41e4fb6a3786a05 2026-03-31 07:37:22,945 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:37:22,959 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8d69ff41e4fb6a3786a05 HTTP/1.1" 200 None 2026-03-31 07:37:22,960 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:37:22,973 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3295318a3717aa65d01c3 HTTP/1.1" 200 None 2026-03-31 07:37:22,975 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:37:22,995 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8d69ff41e4fb6a3786a05 HTTP/1.1" 200 None 2026-03-31 07:37:22,997 - lava_callback - ERROR - Error processing callback for node 69c8d69ff41e4fb6a3786a05: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:34640 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51138 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38502 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38506 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53494 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57068 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58836 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58838 - "GET / HTTP/1.0" 200 OK 2026-03-31 07:38:22,868 - lava_callback - INFO - Received callback for node 69c8d6e9f41e4fb6a3786a0f INFO: 172.20.0.1:36350 - "POST /node/69c8d6e9f41e4fb6a3786a0f HTTP/1.0" 202 Accepted 2026-03-31 07:38:22,873 - lava_callback - INFO - Starting async job submit for node 69c8d6e9f41e4fb6a3786a0f 2026-03-31 07:38:22,874 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:38:22,889 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8d6e9f41e4fb6a3786a0f HTTP/1.1" 200 None 2026-03-31 07:38:22,890 - lava_callback - DEBUG - Processing node 69c8d6e9f41e4fb6a3786a0f: baseline-arm64-baylibre 2026-03-31 07:38:22,890 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8d6e9f41e4fb6a3786a0f 2026-03-31 07:38:22,890 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:38:22,905 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8d6e9f41e4fb6a3786a0f HTTP/1.1" 200 None 2026-03-31 07:38:22,906 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:38:22,924 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3295318a3717aa65d01c3 HTTP/1.1" 200 None 2026-03-31 07:38:22,926 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:38:23,128 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8d6e9f41e4fb6a3786a0f HTTP/1.1" 200 None 2026-03-31 07:38:23,130 - lava_callback - ERROR - Error processing callback for node 69c8d6e9f41e4fb6a3786a0f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:36366 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54066 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45392 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45406 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51504 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33978 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49888 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49900 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49998 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40538 - "GET / HTTP/1.0" 200 OK 2026-03-31 07:39:42,996 - lava_callback - INFO - Received callback for node 69c8d731f41e4fb6a3786a2d INFO: 172.20.0.1:53690 - "POST /node/69c8d731f41e4fb6a3786a2d HTTP/1.0" 202 Accepted 2026-03-31 07:39:43,002 - lava_callback - INFO - Starting async job submit for node 69c8d731f41e4fb6a3786a2d 2026-03-31 07:39:43,003 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:39:43,017 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8d731f41e4fb6a3786a2d HTTP/1.1" 200 None 2026-03-31 07:39:43,018 - lava_callback - DEBUG - Processing node 69c8d731f41e4fb6a3786a2d: baseline-arm64-baylibre 2026-03-31 07:39:43,018 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8d731f41e4fb6a3786a2d 2026-03-31 07:39:43,019 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:39:43,032 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8d731f41e4fb6a3786a2d HTTP/1.1" 200 None 2026-03-31 07:39:43,033 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:39:43,046 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3295318a3717aa65d01c3 HTTP/1.1" 200 None 2026-03-31 07:39:43,048 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:39:43,076 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8d731f41e4fb6a3786a2d HTTP/1.1" 200 None 2026-03-31 07:39:43,078 - lava_callback - ERROR - Error processing callback for node 69c8d731f41e4fb6a3786a2d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:53694 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53696 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54080 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39422 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42198 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42210 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55794 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42646 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43698 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43704 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56130 - "GET / HTTP/1.0" 200 OK 2026-03-31 07:41:03,996 - lava_callback - INFO - Received callback for node 69c8d787f41e4fb6a3786a4a INFO: 172.20.0.1:57460 - "POST /node/69c8d787f41e4fb6a3786a4a HTTP/1.0" 202 Accepted 2026-03-31 07:41:04,004 - lava_callback - INFO - Starting async job submit for node 69c8d787f41e4fb6a3786a4a 2026-03-31 07:41:04,005 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:41:04,020 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8d787f41e4fb6a3786a4a HTTP/1.1" 200 None 2026-03-31 07:41:04,021 - lava_callback - DEBUG - Processing node 69c8d787f41e4fb6a3786a4a: baseline-arm64-clabbe 2026-03-31 07:41:04,021 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8d787f41e4fb6a3786a4a 2026-03-31 07:41:04,022 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:41:04,044 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8d787f41e4fb6a3786a4a HTTP/1.1" 200 None 2026-03-31 07:41:04,046 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:41:04,064 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3295318a3717aa65d01c3 HTTP/1.1" 200 None 2026-03-31 07:41:04,066 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:41:04,088 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8d787f41e4fb6a3786a4a HTTP/1.1" 200 None 2026-03-31 07:41:04,089 - lava_callback - ERROR - Error processing callback for node 69c8d787f41e4fb6a3786a4a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:57474 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54156 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54170 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59356 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47642 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42244 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42254 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54288 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33812 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41170 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41184 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50342 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44848 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34134 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34150 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33036 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37158 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37008 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37016 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42722 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47892 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36644 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36648 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47122 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58048 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57316 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57332 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42884 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43388 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38852 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38860 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46756 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35576 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60680 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60694 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60222 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36730 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46192 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46204 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37208 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52702 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55362 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55376 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49698 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54536 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55716 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55720 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48350 - "GET / HTTP/1.0" 200 OK 2026-03-31 07:47:03,448 - lava_callback - INFO - Received callback for node 69c8d8f2f41e4fb6a3786a86 INFO: 172.20.0.1:50048 - "POST /node/69c8d8f2f41e4fb6a3786a86 HTTP/1.0" 202 Accepted 2026-03-31 07:47:03,454 - lava_callback - INFO - Starting async job submit for node 69c8d8f2f41e4fb6a3786a86 2026-03-31 07:47:03,455 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:47:03,469 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8d8f2f41e4fb6a3786a86 HTTP/1.1" 200 None 2026-03-31 07:47:03,469 - lava_callback - DEBUG - Processing node 69c8d8f2f41e4fb6a3786a86: baseline-arm64-baylibre 2026-03-31 07:47:03,470 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8d8f2f41e4fb6a3786a86 2026-03-31 07:47:03,470 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:47:03,483 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8d8f2f41e4fb6a3786a86 HTTP/1.1" 200 None 2026-03-31 07:47:03,485 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:47:03,498 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f7ef41e4fb6a3782f4d HTTP/1.1" 200 None 2026-03-31 07:47:03,499 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:47:03,520 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8d8f2f41e4fb6a3786a86 HTTP/1.1" 200 None 2026-03-31 07:47:03,521 - lava_callback - ERROR - Error processing callback for node 69c8d8f2f41e4fb6a3786a86: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:50060 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57928 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57944 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47676 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47028 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52748 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52762 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38368 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46302 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35642 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35658 - "GET / HTTP/1.0" 200 OK 2026-03-31 07:48:23,447 - lava_callback - INFO - Received callback for node 69c8d93bf41e4fb6a3786a8d INFO: 172.20.0.1:56938 - "POST /node/69c8d93bf41e4fb6a3786a8d HTTP/1.0" 202 Accepted 2026-03-31 07:48:23,452 - lava_callback - INFO - Starting async job submit for node 69c8d93bf41e4fb6a3786a8d 2026-03-31 07:48:23,453 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:48:23,468 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8d93bf41e4fb6a3786a8d HTTP/1.1" 200 None 2026-03-31 07:48:23,469 - lava_callback - DEBUG - Processing node 69c8d93bf41e4fb6a3786a8d: baseline-arm64-baylibre 2026-03-31 07:48:23,469 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8d93bf41e4fb6a3786a8d 2026-03-31 07:48:23,470 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:48:23,491 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8d93bf41e4fb6a3786a8d HTTP/1.1" 200 None 2026-03-31 07:48:23,492 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:48:23,506 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3295318a3717aa65d01c3 HTTP/1.1" 200 None 2026-03-31 07:48:23,507 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:48:23,533 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8d93bf41e4fb6a3786a8d HTTP/1.1" 200 None 2026-03-31 07:48:23,534 - lava_callback - ERROR - Error processing callback for node 69c8d93bf41e4fb6a3786a8d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:56944 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59264 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57488 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57504 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56384 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57838 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41860 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41874 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48290 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47552 - "GET / HTTP/1.0" 200 OK 2026-03-31 07:49:44,211 - lava_callback - INFO - Received callback for node 69c8d98df41e4fb6a3786a9a INFO: 172.20.0.1:56006 - "POST /node/69c8d98df41e4fb6a3786a9a HTTP/1.0" 202 Accepted 2026-03-31 07:49:44,217 - lava_callback - INFO - Starting async job submit for node 69c8d98df41e4fb6a3786a9a 2026-03-31 07:49:44,218 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:49:44,232 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8d98df41e4fb6a3786a9a HTTP/1.1" 200 None 2026-03-31 07:49:44,233 - lava_callback - DEBUG - Processing node 69c8d98df41e4fb6a3786a9a: baseline-arm64-clabbe 2026-03-31 07:49:44,233 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8d98df41e4fb6a3786a9a 2026-03-31 07:49:44,233 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:49:44,247 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8d98df41e4fb6a3786a9a HTTP/1.1" 200 None 2026-03-31 07:49:44,248 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:49:44,261 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f7ef41e4fb6a3782f4d HTTP/1.1" 200 None 2026-03-31 07:49:44,263 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:49:44,293 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8d98df41e4fb6a3786a9a HTTP/1.1" 200 None 2026-03-31 07:49:44,295 - lava_callback - ERROR - Error processing callback for node 69c8d98df41e4fb6a3786a9a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:56022 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56028 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40810 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54656 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47230 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47234 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49320 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35270 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48174 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48176 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34028 - "GET / HTTP/1.0" 200 OK 2026-03-31 07:51:04,225 - lava_callback - INFO - Received callback for node 69c8d9daf41e4fb6a3786aa4 INFO: 172.20.0.1:33512 - "POST /node/69c8d9daf41e4fb6a3786aa4 HTTP/1.0" 202 Accepted 2026-03-31 07:51:04,230 - lava_callback - INFO - Starting async job submit for node 69c8d9daf41e4fb6a3786aa4 2026-03-31 07:51:04,231 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:51:04,247 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8d9daf41e4fb6a3786aa4 HTTP/1.1" 200 None 2026-03-31 07:51:04,248 - lava_callback - DEBUG - Processing node 69c8d9daf41e4fb6a3786aa4: baseline-arm64-clabbe 2026-03-31 07:51:04,248 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8d9daf41e4fb6a3786aa4 2026-03-31 07:51:04,248 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:51:04,262 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8d9daf41e4fb6a3786aa4 HTTP/1.1" 200 None 2026-03-31 07:51:04,263 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:51:04,277 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f7ef41e4fb6a3782f4d HTTP/1.1" 200 None 2026-03-31 07:51:04,278 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:51:04,299 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8d9daf41e4fb6a3786aa4 HTTP/1.1" 200 None 2026-03-31 07:51:04,305 - lava_callback - ERROR - Error processing callback for node 69c8d9daf41e4fb6a3786aa4: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:33528 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58552 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58558 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45648 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57574 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46008 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46018 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34638 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36204 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57656 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57660 - "GET / HTTP/1.0" 200 OK 2026-03-31 07:52:24,339 - lava_callback - INFO - Received callback for node 69c8da26f41e4fb6a3786ab3 INFO: 172.20.0.1:59810 - "POST /node/69c8da26f41e4fb6a3786ab3 HTTP/1.0" 202 Accepted 2026-03-31 07:52:24,345 - lava_callback - INFO - Starting async job submit for node 69c8da26f41e4fb6a3786ab3 2026-03-31 07:52:24,346 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:52:24,365 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8da26f41e4fb6a3786ab3 HTTP/1.1" 200 None 2026-03-31 07:52:24,365 - lava_callback - DEBUG - Processing node 69c8da26f41e4fb6a3786ab3: baseline-arm64-clabbe 2026-03-31 07:52:24,366 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8da26f41e4fb6a3786ab3 2026-03-31 07:52:24,366 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:52:24,412 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8da26f41e4fb6a3786ab3 HTTP/1.1" 200 None 2026-03-31 07:52:24,414 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:52:24,427 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f7ef41e4fb6a3782f4d HTTP/1.1" 200 None 2026-03-31 07:52:24,428 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:52:24,451 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8da26f41e4fb6a3786ab3 HTTP/1.1" 200 None 2026-03-31 07:52:24,461 - lava_callback - ERROR - Error processing callback for node 69c8da26f41e4fb6a3786ab3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:59812 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45110 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54530 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54532 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47448 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56028 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34574 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34588 - "GET / HTTP/1.0" 200 OK 2026-03-31 07:53:24,373 - lava_callback - INFO - Received callback for node 69c8da71f41e4fb6a3786ab9 INFO: 172.20.0.1:51966 - "POST /node/69c8da71f41e4fb6a3786ab9 HTTP/1.0" 202 Accepted 2026-03-31 07:53:24,378 - lava_callback - INFO - Starting async job submit for node 69c8da71f41e4fb6a3786ab9 2026-03-31 07:53:24,379 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:53:24,396 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8da71f41e4fb6a3786ab9 HTTP/1.1" 200 None 2026-03-31 07:53:24,397 - lava_callback - DEBUG - Processing node 69c8da71f41e4fb6a3786ab9: baseline-arm64-clabbe 2026-03-31 07:53:24,397 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8da71f41e4fb6a3786ab9 2026-03-31 07:53:24,398 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:53:24,411 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8da71f41e4fb6a3786ab9 HTTP/1.1" 200 None 2026-03-31 07:53:24,413 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:53:24,426 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f7ef41e4fb6a3782f4d HTTP/1.1" 200 None 2026-03-31 07:53:24,427 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:53:24,447 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8da71f41e4fb6a3786ab9 HTTP/1.1" 200 None 2026-03-31 07:53:24,449 - lava_callback - ERROR - Error processing callback for node 69c8da71f41e4fb6a3786ab9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:51974 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59824 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60682 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60696 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49088 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42676 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52448 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52458 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45338 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56482 - "GET / HTTP/1.0" 200 OK 2026-03-31 07:54:44,445 - lava_callback - INFO - Received callback for node 69c8dabbf41e4fb6a3786ac9 INFO: 172.20.0.1:60026 - "POST /node/69c8dabbf41e4fb6a3786ac9 HTTP/1.0" 202 Accepted 2026-03-31 07:54:44,451 - lava_callback - INFO - Starting async job submit for node 69c8dabbf41e4fb6a3786ac9 2026-03-31 07:54:44,451 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:54:44,466 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8dabbf41e4fb6a3786ac9 HTTP/1.1" 200 None 2026-03-31 07:54:44,467 - lava_callback - DEBUG - Processing node 69c8dabbf41e4fb6a3786ac9: baseline-arm64-clabbe 2026-03-31 07:54:44,467 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8dabbf41e4fb6a3786ac9 2026-03-31 07:54:44,468 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:54:44,481 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8dabbf41e4fb6a3786ac9 HTTP/1.1" 200 None 2026-03-31 07:54:44,482 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:54:44,496 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3295318a3717aa65d01c3 HTTP/1.1" 200 None 2026-03-31 07:54:44,497 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:54:44,519 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8dabbf41e4fb6a3786ac9 HTTP/1.1" 200 None 2026-03-31 07:54:44,529 - lava_callback - ERROR - Error processing callback for node 69c8dabbf41e4fb6a3786ac9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:60028 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60032 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53212 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33226 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57688 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57690 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59582 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47294 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45900 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45916 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33840 - "GET / HTTP/1.0" 200 OK 2026-03-31 07:56:04,488 - lava_callback - INFO - Received callback for node 69c8db06f41e4fb6a3786ad0 INFO: 172.20.0.1:51720 - "POST /node/69c8db06f41e4fb6a3786ad0 HTTP/1.0" 202 Accepted 2026-03-31 07:56:04,494 - lava_callback - INFO - Starting async job submit for node 69c8db06f41e4fb6a3786ad0 2026-03-31 07:56:04,495 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:56:04,509 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8db06f41e4fb6a3786ad0 HTTP/1.1" 200 None 2026-03-31 07:56:04,510 - lava_callback - DEBUG - Processing node 69c8db06f41e4fb6a3786ad0: baseline-arm64-clabbe 2026-03-31 07:56:04,510 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8db06f41e4fb6a3786ad0 2026-03-31 07:56:04,511 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:56:04,534 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8db06f41e4fb6a3786ad0 HTTP/1.1" 200 None 2026-03-31 07:56:04,535 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:56:04,548 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f7ef41e4fb6a3782f4d HTTP/1.1" 200 None 2026-03-31 07:56:04,550 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:56:04,570 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8db06f41e4fb6a3786ad0 HTTP/1.1" 200 None 2026-03-31 07:56:04,571 - lava_callback - ERROR - Error processing callback for node 69c8db06f41e4fb6a3786ad0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:51726 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57176 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57186 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44472 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32976 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34662 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34674 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52366 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33316 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51518 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51528 - "GET / HTTP/1.0" 200 OK 2026-03-31 07:57:24,583 - lava_callback - INFO - Received callback for node 69c8db50f41e4fb6a3786add INFO: 172.20.0.1:42566 - "POST /node/69c8db50f41e4fb6a3786add HTTP/1.0" 202 Accepted 2026-03-31 07:57:24,588 - lava_callback - INFO - Starting async job submit for node 69c8db50f41e4fb6a3786add 2026-03-31 07:57:24,589 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:57:24,612 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8db50f41e4fb6a3786add HTTP/1.1" 200 None 2026-03-31 07:57:24,613 - lava_callback - DEBUG - Processing node 69c8db50f41e4fb6a3786add: baseline-arm64-clabbe 2026-03-31 07:57:24,613 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8db50f41e4fb6a3786add 2026-03-31 07:57:24,614 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:57:24,627 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8db50f41e4fb6a3786add HTTP/1.1" 200 None 2026-03-31 07:57:24,629 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:57:24,642 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f7ef41e4fb6a3782f4d HTTP/1.1" 200 None 2026-03-31 07:57:24,644 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:57:24,668 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8db50f41e4fb6a3786add HTTP/1.1" 200 None 2026-03-31 07:57:24,670 - lava_callback - ERROR - Error processing callback for node 69c8db50f41e4fb6a3786add: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:42580 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56430 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40470 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40480 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40828 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54024 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42344 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42360 - "GET / HTTP/1.0" 200 OK 2026-03-31 07:58:24,039 - lava_callback - INFO - Received callback for node 69c8db93f41e4fb6a3786ae7 INFO: 172.20.0.1:48986 - "POST /node/69c8db93f41e4fb6a3786ae7 HTTP/1.0" 202 Accepted 2026-03-31 07:58:24,045 - lava_callback - INFO - Starting async job submit for node 69c8db93f41e4fb6a3786ae7 2026-03-31 07:58:24,046 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:58:24,060 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8db93f41e4fb6a3786ae7 HTTP/1.1" 200 None 2026-03-31 07:58:24,061 - lava_callback - DEBUG - Processing node 69c8db93f41e4fb6a3786ae7: baseline-arm64-baylibre 2026-03-31 07:58:24,061 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8db93f41e4fb6a3786ae7 2026-03-31 07:58:24,062 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:58:24,076 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8db93f41e4fb6a3786ae7 HTTP/1.1" 200 None 2026-03-31 07:58:24,077 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:58:24,090 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3295318a3717aa65d01c3 HTTP/1.1" 200 None 2026-03-31 07:58:24,091 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:58:24,119 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8db93f41e4fb6a3786ae7 HTTP/1.1" 200 None 2026-03-31 07:58:24,120 - lava_callback - ERROR - Error processing callback for node 69c8db93f41e4fb6a3786ae7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:49000 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53630 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60506 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60518 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52050 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52024 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47558 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47560 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56906 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55428 - "GET / HTTP/1.0" 200 OK 2026-03-31 07:59:44,099 - lava_callback - INFO - Received callback for node 69c8dbdff41e4fb6a3786af5 INFO: 172.20.0.1:45822 - "POST /node/69c8dbdff41e4fb6a3786af5 HTTP/1.0" 202 Accepted 2026-03-31 07:59:44,105 - lava_callback - INFO - Starting async job submit for node 69c8dbdff41e4fb6a3786af5 2026-03-31 07:59:44,106 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:59:44,121 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8dbdff41e4fb6a3786af5 HTTP/1.1" 200 None 2026-03-31 07:59:44,122 - lava_callback - DEBUG - Processing node 69c8dbdff41e4fb6a3786af5: baseline-arm64-baylibre 2026-03-31 07:59:44,122 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8dbdff41e4fb6a3786af5 2026-03-31 07:59:44,123 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:59:44,146 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8dbdff41e4fb6a3786af5 HTTP/1.1" 200 None 2026-03-31 07:59:44,148 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:59:44,163 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3295318a3717aa65d01c3 HTTP/1.1" 200 None 2026-03-31 07:59:44,165 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 07:59:44,188 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8dbdff41e4fb6a3786af5 HTTP/1.1" 200 None 2026-03-31 07:59:44,189 - lava_callback - ERROR - Error processing callback for node 69c8dbdff41e4fb6a3786af5: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:45836 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45850 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45898 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33352 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36440 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36448 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32830 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34426 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52562 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52578 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53650 - "GET / HTTP/1.0" 200 OK 2026-03-31 08:01:04,767 - lava_callback - INFO - Received callback for node 69c8dc31f41e4fb6a3786afc INFO: 172.20.0.1:47536 - "POST /node/69c8dc31f41e4fb6a3786afc HTTP/1.0" 202 Accepted 2026-03-31 08:01:04,772 - lava_callback - INFO - Starting async job submit for node 69c8dc31f41e4fb6a3786afc 2026-03-31 08:01:04,773 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:01:04,787 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8dc31f41e4fb6a3786afc HTTP/1.1" 200 None 2026-03-31 08:01:04,788 - lava_callback - DEBUG - Processing node 69c8dc31f41e4fb6a3786afc: baseline-arm64-clabbe 2026-03-31 08:01:04,788 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8dc31f41e4fb6a3786afc 2026-03-31 08:01:04,789 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:01:04,802 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8dc31f41e4fb6a3786afc HTTP/1.1" 200 None 2026-03-31 08:01:04,803 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:01:04,817 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3295318a3717aa65d01c3 HTTP/1.1" 200 None 2026-03-31 08:01:04,818 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:01:04,839 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8dc31f41e4fb6a3786afc HTTP/1.1" 200 None 2026-03-31 08:01:04,844 - lava_callback - ERROR - Error processing callback for node 69c8dc31f41e4fb6a3786afc: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:47552 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45696 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45698 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60242 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48454 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59712 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59714 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35796 - "GET / HTTP/1.0" 200 OK 2026-03-31 08:02:04,598 - lava_callback - INFO - Received callback for node 69c8dc76f41e4fb6a3786b09 INFO: 172.20.0.1:48962 - "POST /node/69c8dc76f41e4fb6a3786b09 HTTP/1.0" 202 Accepted 2026-03-31 08:02:04,604 - lava_callback - INFO - Starting async job submit for node 69c8dc76f41e4fb6a3786b09 2026-03-31 08:02:04,605 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:02:04,642 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8dc76f41e4fb6a3786b09 HTTP/1.1" 200 None 2026-03-31 08:02:04,643 - lava_callback - DEBUG - Processing node 69c8dc76f41e4fb6a3786b09: baseline-arm64-baylibre 2026-03-31 08:02:04,643 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8dc76f41e4fb6a3786b09 2026-03-31 08:02:04,644 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:02:04,658 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8dc76f41e4fb6a3786b09 HTTP/1.1" 200 None 2026-03-31 08:02:04,660 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:02:04,674 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3295318a3717aa65d01c3 HTTP/1.1" 200 None 2026-03-31 08:02:04,675 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:02:04,697 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8dc76f41e4fb6a3786b09 HTTP/1.1" 200 None 2026-03-31 08:02:04,699 - lava_callback - ERROR - Error processing callback for node 69c8dc76f41e4fb6a3786b09: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:48978 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43692 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43704 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47674 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52238 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45488 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45492 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48716 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37330 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55496 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55498 - "GET / HTTP/1.0" 200 OK 2026-03-31 08:03:24,287 - lava_callback - INFO - Received callback for node 69c8dcbff41e4fb6a3786b11 INFO: 172.20.0.1:60578 - "POST /node/69c8dcbff41e4fb6a3786b11 HTTP/1.0" 202 Accepted 2026-03-31 08:03:24,292 - lava_callback - INFO - Starting async job submit for node 69c8dcbff41e4fb6a3786b11 2026-03-31 08:03:24,293 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:03:24,307 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8dcbff41e4fb6a3786b11 HTTP/1.1" 200 None 2026-03-31 08:03:24,308 - lava_callback - DEBUG - Processing node 69c8dcbff41e4fb6a3786b11: baseline-arm64-baylibre 2026-03-31 08:03:24,308 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8dcbff41e4fb6a3786b11 2026-03-31 08:03:24,309 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:03:24,322 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8dcbff41e4fb6a3786b11 HTTP/1.1" 200 None 2026-03-31 08:03:24,323 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:03:24,337 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3295318a3717aa65d01c3 HTTP/1.1" 200 None 2026-03-31 08:03:24,338 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:03:24,358 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8dcbff41e4fb6a3786b11 HTTP/1.1" 200 None 2026-03-31 08:03:24,364 - lava_callback - ERROR - Error processing callback for node 69c8dcbff41e4fb6a3786b11: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:60588 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52778 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34358 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34370 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55790 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39878 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38106 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38110 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41524 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40738 - "GET / HTTP/1.0" 200 OK 2026-03-31 08:04:44,352 - lava_callback - INFO - Received callback for node 69c8dd0af41e4fb6a3786b19 INFO: 172.20.0.1:43336 - "POST /node/69c8dd0af41e4fb6a3786b19 HTTP/1.0" 202 Accepted 2026-03-31 08:04:44,357 - lava_callback - INFO - Starting async job submit for node 69c8dd0af41e4fb6a3786b19 2026-03-31 08:04:44,358 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:04:44,373 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8dd0af41e4fb6a3786b19 HTTP/1.1" 200 None 2026-03-31 08:04:44,374 - lava_callback - DEBUG - Processing node 69c8dd0af41e4fb6a3786b19: baseline-arm64-baylibre 2026-03-31 08:04:44,374 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8dd0af41e4fb6a3786b19 2026-03-31 08:04:44,374 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:04:44,410 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8dd0af41e4fb6a3786b19 HTTP/1.1" 200 None 2026-03-31 08:04:44,411 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:04:44,429 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3295318a3717aa65d01c3 HTTP/1.1" 200 None 2026-03-31 08:04:44,430 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:04:44,450 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8dd0af41e4fb6a3786b19 HTTP/1.1" 200 None 2026-03-31 08:04:44,456 - lava_callback - ERROR - Error processing callback for node 69c8dd0af41e4fb6a3786b19: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:43338 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43352 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53448 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36350 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47300 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47314 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41622 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54498 - "GET / HTTP/1.0" 200 OK 2026-03-31 08:05:44,377 - lava_callback - INFO - Received callback for node 69c8dd53f41e4fb6a3786b26 INFO: 172.20.0.1:45048 - "POST /node/69c8dd53f41e4fb6a3786b26 HTTP/1.0" 202 Accepted 2026-03-31 08:05:44,383 - lava_callback - INFO - Starting async job submit for node 69c8dd53f41e4fb6a3786b26 2026-03-31 08:05:44,384 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:05:44,398 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8dd53f41e4fb6a3786b26 HTTP/1.1" 200 None 2026-03-31 08:05:44,399 - lava_callback - DEBUG - Processing node 69c8dd53f41e4fb6a3786b26: baseline-arm64-baylibre 2026-03-31 08:05:44,399 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8dd53f41e4fb6a3786b26 2026-03-31 08:05:44,400 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:05:44,414 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8dd53f41e4fb6a3786b26 HTTP/1.1" 200 None 2026-03-31 08:05:44,415 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:05:44,428 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3295318a3717aa65d01c3 HTTP/1.1" 200 None 2026-03-31 08:05:44,429 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:05:44,450 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8dd53f41e4fb6a3786b26 HTTP/1.1" 200 None 2026-03-31 08:05:44,452 - lava_callback - ERROR - Error processing callback for node 69c8dd53f41e4fb6a3786b26: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:45062 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45072 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59270 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53266 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51032 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51048 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56758 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49010 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39812 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39826 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43274 - "GET / HTTP/1.0" 200 OK 2026-03-31 08:07:04,992 - lava_callback - INFO - Received callback for node 69c8dda4f41e4fb6a3786b34 INFO: 172.20.0.1:59458 - "POST /node/69c8dda4f41e4fb6a3786b34 HTTP/1.0" 202 Accepted 2026-03-31 08:07:04,997 - lava_callback - INFO - Starting async job submit for node 69c8dda4f41e4fb6a3786b34 2026-03-31 08:07:04,998 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:07:05,014 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8dda4f41e4fb6a3786b34 HTTP/1.1" 200 None 2026-03-31 08:07:05,015 - lava_callback - DEBUG - Processing node 69c8dda4f41e4fb6a3786b34: baseline-arm64-clabbe 2026-03-31 08:07:05,015 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8dda4f41e4fb6a3786b34 2026-03-31 08:07:05,016 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:07:05,076 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8dda4f41e4fb6a3786b34 HTTP/1.1" 200 None 2026-03-31 08:07:05,078 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:07:05,105 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3295318a3717aa65d01c3 HTTP/1.1" 200 None 2026-03-31 08:07:05,107 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:07:05,129 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8dda4f41e4fb6a3786b34 HTTP/1.1" 200 None 2026-03-31 08:07:05,131 - lava_callback - ERROR - Error processing callback for node 69c8dda4f41e4fb6a3786b34: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:59468 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44154 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44162 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49490 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42140 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59692 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59702 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47184 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39776 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60340 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60344 - "GET / HTTP/1.0" 200 OK 2026-03-31 08:08:25,073 - lava_callback - INFO - Received callback for node 69c8ddeef41e4fb6a3786b3b INFO: 172.20.0.1:39114 - "POST /node/69c8ddeef41e4fb6a3786b3b HTTP/1.0" 202 Accepted 2026-03-31 08:08:25,080 - lava_callback - INFO - Starting async job submit for node 69c8ddeef41e4fb6a3786b3b 2026-03-31 08:08:25,081 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:08:25,095 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8ddeef41e4fb6a3786b3b HTTP/1.1" 200 None 2026-03-31 08:08:25,096 - lava_callback - DEBUG - Processing node 69c8ddeef41e4fb6a3786b3b: baseline-arm64-clabbe 2026-03-31 08:08:25,096 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8ddeef41e4fb6a3786b3b 2026-03-31 08:08:25,097 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:08:25,110 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8ddeef41e4fb6a3786b3b HTTP/1.1" 200 None 2026-03-31 08:08:25,112 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:08:25,125 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3295318a3717aa65d01c3 HTTP/1.1" 200 None 2026-03-31 08:08:25,126 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:08:25,146 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8ddeef41e4fb6a3786b3b HTTP/1.1" 200 None 2026-03-31 08:08:25,150 - lava_callback - ERROR - Error processing callback for node 69c8ddeef41e4fb6a3786b3b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:39130 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59290 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45188 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45198 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58800 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33108 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41020 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41030 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57530 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37660 - "GET / HTTP/1.0" 200 OK 2026-03-31 08:09:44,646 - lava_callback - INFO - Received callback for node 69c8de34f41e4fb6a3786b41 INFO: 172.20.0.1:51774 - "POST /node/69c8de34f41e4fb6a3786b41 HTTP/1.0" 202 Accepted 2026-03-31 08:09:44,655 - lava_callback - INFO - Starting async job submit for node 69c8de34f41e4fb6a3786b41 2026-03-31 08:09:44,656 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:09:44,683 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8de34f41e4fb6a3786b41 HTTP/1.1" 200 None 2026-03-31 08:09:44,684 - lava_callback - DEBUG - Processing node 69c8de34f41e4fb6a3786b41: baseline-arm64-baylibre 2026-03-31 08:09:44,684 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8de34f41e4fb6a3786b41 2026-03-31 08:09:44,684 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:09:44,868 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8de34f41e4fb6a3786b41 HTTP/1.1" 200 None 2026-03-31 08:09:44,870 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:09:44,895 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3295318a3717aa65d01c3 HTTP/1.1" 200 None 2026-03-31 08:09:44,897 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:09:44,919 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8de34f41e4fb6a3786b41 HTTP/1.1" 200 None 2026-03-31 08:09:44,924 - lava_callback - ERROR - Error processing callback for node 69c8de34f41e4fb6a3786b41: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:51786 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51796 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34862 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54994 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50484 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50486 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39610 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45034 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44352 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44354 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38910 - "GET / HTTP/1.0" 200 OK 2026-03-31 08:11:05,192 - lava_callback - INFO - Received callback for node 69c8de84f41e4fb6a3786b4d INFO: 172.20.0.1:41080 - "POST /node/69c8de84f41e4fb6a3786b4d HTTP/1.0" 202 Accepted 2026-03-31 08:11:05,198 - lava_callback - INFO - Starting async job submit for node 69c8de84f41e4fb6a3786b4d 2026-03-31 08:11:05,199 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:11:05,223 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8de84f41e4fb6a3786b4d HTTP/1.1" 200 None 2026-03-31 08:11:05,224 - lava_callback - DEBUG - Processing node 69c8de84f41e4fb6a3786b4d: baseline-arm64-clabbe 2026-03-31 08:11:05,224 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8de84f41e4fb6a3786b4d 2026-03-31 08:11:05,225 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:11:05,239 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8de84f41e4fb6a3786b4d HTTP/1.1" 200 None 2026-03-31 08:11:05,240 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:11:05,255 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3295318a3717aa65d01c3 HTTP/1.1" 200 None 2026-03-31 08:11:05,257 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:11:05,277 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8de84f41e4fb6a3786b4d HTTP/1.1" 200 None 2026-03-31 08:11:05,287 - lava_callback - ERROR - Error processing callback for node 69c8de84f41e4fb6a3786b4d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:41094 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54002 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54004 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44250 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50870 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42796 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42800 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40608 - "GET / HTTP/1.0" 200 OK 2026-03-31 08:12:05,255 - lava_callback - INFO - Received callback for node 69c8decff41e4fb6a3786b58 INFO: 172.20.0.1:47708 - "POST /node/69c8decff41e4fb6a3786b58 HTTP/1.0" 202 Accepted 2026-03-31 08:12:05,261 - lava_callback - INFO - Starting async job submit for node 69c8decff41e4fb6a3786b58 2026-03-31 08:12:05,262 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:12:05,280 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8decff41e4fb6a3786b58 HTTP/1.1" 200 None 2026-03-31 08:12:05,281 - lava_callback - DEBUG - Processing node 69c8decff41e4fb6a3786b58: baseline-arm64-clabbe 2026-03-31 08:12:05,281 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8decff41e4fb6a3786b58 2026-03-31 08:12:05,282 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:12:05,298 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8decff41e4fb6a3786b58 HTTP/1.1" 200 None 2026-03-31 08:12:05,299 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:12:05,313 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3295318a3717aa65d01c3 HTTP/1.1" 200 None 2026-03-31 08:12:05,314 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:12:05,346 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8decff41e4fb6a3786b58 HTTP/1.1" 200 None 2026-03-31 08:12:05,348 - lava_callback - ERROR - Error processing callback for node 69c8decff41e4fb6a3786b58: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:47712 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58258 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58270 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52504 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41996 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47662 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47672 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59402 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34692 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45500 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45504 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42680 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39754 - "GET / HTTP/1.0" 200 OK INFO: Shutting down INFO: Waiting for application shutdown. INFO: Application shutdown complete. INFO: Finished server process [1]