INFO: Started server process [1] INFO: Waiting for application startup. INFO: Application startup complete. INFO: Uvicorn running on http://0.0.0.0:8000 (Press CTRL+C to quit) INFO: 172.21.0.1:37630 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45034 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48574 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45564 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45568 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48386 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48388 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50688 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34710 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56150 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43040 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44626 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44636 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60026 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60030 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56300 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38092 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56540 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38480 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44772 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44784 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47592 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47608 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54080 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51422 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44956 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59276 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55812 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55822 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42112 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42114 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43932 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51564 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42532 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43538 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40346 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40362 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60004 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60014 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38912 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36532 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55584 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46650 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56910 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56926 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54136 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54142 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50446 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34116 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52912 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43560 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45768 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45772 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34730 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34736 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50214 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58060 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51630 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50080 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43106 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43118 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48230 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48232 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46786 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40038 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37772 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57794 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52236 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52250 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54286 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54288 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48782 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55014 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36740 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35026 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46208 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46222 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50018 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50024 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60080 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39356 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42234 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46590 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56614 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56622 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58698 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58710 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58592 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54550 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34160 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59924 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39756 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39768 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53344 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53352 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59080 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39182 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40144 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40786 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58690 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58694 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53426 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53440 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33038 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45278 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56186 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47324 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41770 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41772 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36210 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36226 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45820 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57414 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36532 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35448 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55454 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55464 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36672 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36686 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45320 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34628 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60402 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47776 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45610 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45614 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57538 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57550 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44294 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43370 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34440 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53144 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35824 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35828 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43108 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43124 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37078 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52780 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42726 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59574 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38452 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38456 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56318 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56326 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43168 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60906 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36292 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49644 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41850 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41862 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46464 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46468 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35546 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38668 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42524 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51678 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43096 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43100 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58164 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58174 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40134 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47854 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44972 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37528 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44096 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44100 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52386 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52396 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45928 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35274 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57468 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58948 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34766 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34770 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51682 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51694 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42086 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33652 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56110 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33758 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51134 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51138 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55280 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55290 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56486 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42956 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59774 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39344 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45774 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45782 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42462 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42478 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48744 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52328 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55000 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40100 - "GET / HTTP/1.0" 200 OK 2026-03-26 16:39:01,513 - lava_callback - INFO - Received callback for node 69c2be186778148209fd33cf 2026-03-26 16:39:01,520 - lava_callback - INFO - Starting async job submit for node 69c2be186778148209fd33cf INFO: 172.21.0.1:34550 - "POST /node/69c2be186778148209fd33cf HTTP/1.0" 202 Accepted 2026-03-26 16:39:01,522 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:39:01,544 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2be186778148209fd33cf HTTP/1.1" 200 None 2026-03-26 16:39:01,545 - lava_callback - DEBUG - Processing node 69c2be186778148209fd33cf: baseline-arm-baylibre 2026-03-26 16:39:01,546 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2be186778148209fd33cf 2026-03-26 16:39:01,547 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:39:01,568 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2be186778148209fd33cf HTTP/1.1" 200 None 2026-03-26 16:39:01,569 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:39:01,591 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad09221a77312e2220ee1 HTTP/1.1" 200 None 2026-03-26 16:39:01,593 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:39:01,625 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2be186778148209fd33cf HTTP/1.1" 200 None 2026-03-26 16:39:01,628 - lava_callback - ERROR - Error processing callback for node 69c2be186778148209fd33cf: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:34560 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34564 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56588 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56596 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35564 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60662 - "GET / HTTP/1.0" 200 OK 2026-03-26 16:39:41,118 - lava_callback - INFO - Received callback for node 69c2be426778148209fd33d9 INFO: 172.21.0.1:53770 - "POST /node/69c2be426778148209fd33d9 HTTP/1.0" 202 Accepted 2026-03-26 16:39:41,124 - lava_callback - INFO - Starting async job submit for node 69c2be426778148209fd33d9 2026-03-26 16:39:41,124 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:39:41,139 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2be426778148209fd33d9 HTTP/1.1" 200 None 2026-03-26 16:39:41,140 - lava_callback - DEBUG - Processing node 69c2be426778148209fd33d9: baseline-arm-baylibre 2026-03-26 16:39:41,140 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2be426778148209fd33d9 2026-03-26 16:39:41,141 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:39:41,157 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2be426778148209fd33d9 HTTP/1.1" 200 None 2026-03-26 16:39:41,159 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:39:41,172 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad09221a77312e2220ee1 HTTP/1.1" 200 None 2026-03-26 16:39:41,174 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:39:41,194 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2be426778148209fd33d9 HTTP/1.1" 200 None 2026-03-26 16:39:41,200 - lava_callback - ERROR - Error processing callback for node 69c2be426778148209fd33d9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:53786 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47772 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36580 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36592 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43042 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43056 - "GET / HTTP/1.0" 200 OK 2026-03-26 16:40:21,168 - lava_callback - INFO - Received callback for node 69c2be6c6778148209fd33e4 INFO: 172.21.0.1:37926 - "POST /node/69c2be6c6778148209fd33e4 HTTP/1.0" 202 Accepted 2026-03-26 16:40:21,173 - lava_callback - INFO - Starting async job submit for node 69c2be6c6778148209fd33e4 2026-03-26 16:40:21,174 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:40:21,197 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2be6c6778148209fd33e4 HTTP/1.1" 200 None 2026-03-26 16:40:21,198 - lava_callback - DEBUG - Processing node 69c2be6c6778148209fd33e4: baseline-arm-baylibre 2026-03-26 16:40:21,198 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2be6c6778148209fd33e4 2026-03-26 16:40:21,199 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:40:21,213 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2be6c6778148209fd33e4 HTTP/1.1" 200 None 2026-03-26 16:40:21,215 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:40:21,394 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad09221a77312e2220ee1 HTTP/1.1" 200 None 2026-03-26 16:40:21,396 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:40:21,590 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2be6c6778148209fd33e4 HTTP/1.1" 200 None 2026-03-26 16:40:21,601 - lava_callback - ERROR - Error processing callback for node 69c2be6c6778148209fd33e4: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:37932 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39180 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48410 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36436 - "GET / HTTP/1.0" 200 OK 2026-03-26 16:41:01,167 - lava_callback - INFO - Received callback for node 69c2be966778148209fd33f3 INFO: 172.21.0.1:53098 - "POST /node/69c2be966778148209fd33f3 HTTP/1.0" 202 Accepted 2026-03-26 16:41:01,172 - lava_callback - INFO - Starting async job submit for node 69c2be966778148209fd33f3 2026-03-26 16:41:01,173 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:41:01,197 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2be966778148209fd33f3 HTTP/1.1" 200 None 2026-03-26 16:41:01,198 - lava_callback - DEBUG - Processing node 69c2be966778148209fd33f3: baseline-arm-baylibre 2026-03-26 16:41:01,198 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2be966778148209fd33f3 2026-03-26 16:41:01,198 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:41:01,225 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2be966778148209fd33f3 HTTP/1.1" 200 None 2026-03-26 16:41:01,227 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:41:01,261 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad09221a77312e2220ee1 HTTP/1.1" 200 None 2026-03-26 16:41:01,262 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:41:01,283 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2be966778148209fd33f3 HTTP/1.1" 200 None 2026-03-26 16:41:01,289 - lava_callback - ERROR - Error processing callback for node 69c2be966778148209fd33f3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:53104 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53110 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37004 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37008 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54824 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39228 - "GET / HTTP/1.0" 200 OK 2026-03-26 16:41:41,219 - lava_callback - INFO - Received callback for node 69c2beb36778148209fd341b INFO: 172.21.0.1:55602 - "POST /node/69c2beb36778148209fd341b HTTP/1.0" 202 Accepted 2026-03-26 16:41:41,224 - lava_callback - INFO - Starting async job submit for node 69c2beb36778148209fd341b 2026-03-26 16:41:41,225 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:41:41,240 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2beb36778148209fd341b HTTP/1.1" 200 None 2026-03-26 16:41:41,241 - lava_callback - DEBUG - Processing node 69c2beb36778148209fd341b: baseline-arm-baylibre 2026-03-26 16:41:41,241 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2beb36778148209fd341b 2026-03-26 16:41:41,242 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:41:41,256 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2beb36778148209fd341b HTTP/1.1" 200 None 2026-03-26 16:41:41,257 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:41:41,271 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b94b6778148209fd336c HTTP/1.1" 200 None 2026-03-26 16:41:41,273 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:41:41,303 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2beb36778148209fd341b HTTP/1.1" 200 None 2026-03-26 16:41:41,309 - lava_callback - ERROR - Error processing callback for node 69c2beb36778148209fd341b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-26 16:41:41,866 - lava_callback - INFO - Received callback for node 69c2beb46778148209fd341e INFO: 172.21.0.1:55612 - "POST /node/69c2beb46778148209fd341e HTTP/1.0" 202 Accepted 2026-03-26 16:41:41,871 - lava_callback - INFO - Starting async job submit for node 69c2beb46778148209fd341e 2026-03-26 16:41:41,872 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:41:41,886 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2beb46778148209fd341e HTTP/1.1" 200 None 2026-03-26 16:41:41,887 - lava_callback - DEBUG - Processing node 69c2beb46778148209fd341e: baseline-arm-baylibre 2026-03-26 16:41:41,887 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2beb46778148209fd341e 2026-03-26 16:41:41,888 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:41:41,902 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2beb46778148209fd341e HTTP/1.1" 200 None 2026-03-26 16:41:41,903 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:41:41,919 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b94b6778148209fd336c HTTP/1.1" 200 None 2026-03-26 16:41:41,921 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:41:41,942 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2beb46778148209fd341e HTTP/1.1" 200 None 2026-03-26 16:41:41,943 - lava_callback - ERROR - Error processing callback for node 69c2beb46778148209fd341e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-26 16:41:42,506 - lava_callback - INFO - Received callback for node 69c2beb66778148209fd3421 INFO: 172.21.0.1:55626 - "POST /node/69c2beb66778148209fd3421 HTTP/1.0" 202 Accepted 2026-03-26 16:41:42,512 - lava_callback - INFO - Starting async job submit for node 69c2beb66778148209fd3421 2026-03-26 16:41:42,512 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:41:42,528 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2beb66778148209fd3421 HTTP/1.1" 200 None 2026-03-26 16:41:42,529 - lava_callback - DEBUG - Processing node 69c2beb66778148209fd3421: baseline-arm-baylibre 2026-03-26 16:41:42,529 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2beb66778148209fd3421 2026-03-26 16:41:42,530 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:41:42,552 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2beb66778148209fd3421 HTTP/1.1" 200 None 2026-03-26 16:41:42,554 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:41:42,567 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b94b6778148209fd336c HTTP/1.1" 200 None 2026-03-26 16:41:42,569 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:41:42,593 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2beb66778148209fd3421 HTTP/1.1" 200 None 2026-03-26 16:41:42,595 - lava_callback - ERROR - Error processing callback for node 69c2beb66778148209fd3421: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-26 16:41:43,154 - lava_callback - INFO - Received callback for node 69c2beb76778148209fd3424 INFO: 172.21.0.1:55628 - "POST /node/69c2beb76778148209fd3424 HTTP/1.0" 202 Accepted 2026-03-26 16:41:43,160 - lava_callback - INFO - Starting async job submit for node 69c2beb76778148209fd3424 2026-03-26 16:41:43,161 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:41:43,177 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2beb76778148209fd3424 HTTP/1.1" 200 None 2026-03-26 16:41:43,177 - lava_callback - DEBUG - Processing node 69c2beb76778148209fd3424: baseline-arm-baylibre 2026-03-26 16:41:43,178 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2beb76778148209fd3424 2026-03-26 16:41:43,178 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:41:43,191 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2beb76778148209fd3424 HTTP/1.1" 200 None 2026-03-26 16:41:43,192 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:41:43,206 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b94b6778148209fd336c HTTP/1.1" 200 None 2026-03-26 16:41:43,207 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:41:43,227 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2beb76778148209fd3424 HTTP/1.1" 200 None 2026-03-26 16:41:43,237 - lava_callback - ERROR - Error processing callback for node 69c2beb76778148209fd3424: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-26 16:41:43,843 - lava_callback - INFO - Received callback for node 69c2beba6778148209fd3431 INFO: 172.21.0.1:55636 - "POST /node/69c2beba6778148209fd3431 HTTP/1.0" 202 Accepted 2026-03-26 16:41:43,849 - lava_callback - INFO - Starting async job submit for node 69c2beba6778148209fd3431 2026-03-26 16:41:43,850 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:41:43,866 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2beba6778148209fd3431 HTTP/1.1" 200 None 2026-03-26 16:41:43,867 - lava_callback - DEBUG - Processing node 69c2beba6778148209fd3431: baseline-arm-baylibre 2026-03-26 16:41:43,867 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2beba6778148209fd3431 2026-03-26 16:41:43,868 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:41:43,882 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2beba6778148209fd3431 HTTP/1.1" 200 None 2026-03-26 16:41:43,883 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:41:43,896 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b94b6778148209fd336c HTTP/1.1" 200 None 2026-03-26 16:41:43,898 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:41:43,921 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2beba6778148209fd3431 HTTP/1.1" 200 None 2026-03-26 16:41:43,923 - lava_callback - ERROR - Error processing callback for node 69c2beba6778148209fd3431: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-26 16:41:44,482 - lava_callback - INFO - Received callback for node 69c2bebc6778148209fd3434 INFO: 172.21.0.1:55650 - "POST /node/69c2bebc6778148209fd3434 HTTP/1.0" 202 Accepted 2026-03-26 16:41:44,488 - lava_callback - INFO - Starting async job submit for node 69c2bebc6778148209fd3434 2026-03-26 16:41:44,488 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:41:44,503 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2bebc6778148209fd3434 HTTP/1.1" 200 None 2026-03-26 16:41:44,504 - lava_callback - DEBUG - Processing node 69c2bebc6778148209fd3434: baseline-arm-baylibre 2026-03-26 16:41:44,504 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2bebc6778148209fd3434 2026-03-26 16:41:44,505 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:41:44,518 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2bebc6778148209fd3434 HTTP/1.1" 200 None 2026-03-26 16:41:44,519 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:41:44,532 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b94b6778148209fd336c HTTP/1.1" 200 None 2026-03-26 16:41:44,533 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:41:44,553 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2bebc6778148209fd3434 HTTP/1.1" 200 None 2026-03-26 16:41:44,554 - lava_callback - ERROR - Error processing callback for node 69c2bebc6778148209fd3434: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:55662 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35296 - "GET / HTTP/1.0" 200 OK 2026-03-26 16:41:59,713 - lava_callback - INFO - Received callback for node 69c2bec46778148209fd344d INFO: 172.21.0.1:46690 - "POST /node/69c2bec46778148209fd344d HTTP/1.0" 202 Accepted 2026-03-26 16:41:59,719 - lava_callback - INFO - Starting async job submit for node 69c2bec46778148209fd344d 2026-03-26 16:41:59,719 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:41:59,735 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2bec46778148209fd344d HTTP/1.1" 200 None 2026-03-26 16:41:59,736 - lava_callback - DEBUG - Processing node 69c2bec46778148209fd344d: baseline-arm-clabbe 2026-03-26 16:41:59,736 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2bec46778148209fd344d 2026-03-26 16:41:59,737 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:41:59,751 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2bec46778148209fd344d HTTP/1.1" 200 None 2026-03-26 16:41:59,753 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:41:59,768 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b94b6778148209fd336c HTTP/1.1" 200 None 2026-03-26 16:41:59,770 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:41:59,792 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2bec46778148209fd344d HTTP/1.1" 200 None 2026-03-26 16:41:59,793 - lava_callback - ERROR - Error processing callback for node 69c2bec46778148209fd344d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-26 16:42:00,405 - lava_callback - INFO - Received callback for node 69c2bec86778148209fd345c INFO: 172.21.0.1:46702 - "POST /node/69c2bec86778148209fd345c HTTP/1.0" 202 Accepted 2026-03-26 16:42:00,411 - lava_callback - INFO - Starting async job submit for node 69c2bec86778148209fd345c 2026-03-26 16:42:00,412 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:42:00,426 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2bec86778148209fd345c HTTP/1.1" 200 None 2026-03-26 16:42:00,427 - lava_callback - DEBUG - Processing node 69c2bec86778148209fd345c: baseline-arm-clabbe 2026-03-26 16:42:00,427 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2bec86778148209fd345c 2026-03-26 16:42:00,428 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:42:00,442 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2bec86778148209fd345c HTTP/1.1" 200 None 2026-03-26 16:42:00,444 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:42:00,460 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b94b6778148209fd336c HTTP/1.1" 200 None 2026-03-26 16:42:00,461 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:42:00,483 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2bec86778148209fd345c HTTP/1.1" 200 None 2026-03-26 16:42:00,489 - lava_callback - ERROR - Error processing callback for node 69c2bec86778148209fd345c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-26 16:42:01,099 - lava_callback - INFO - Received callback for node 69c2bec96778148209fd3461 INFO: 172.21.0.1:46706 - "POST /node/69c2bec96778148209fd3461 HTTP/1.0" 202 Accepted 2026-03-26 16:42:01,105 - lava_callback - INFO - Starting async job submit for node 69c2bec96778148209fd3461 2026-03-26 16:42:01,106 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:42:01,121 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2bec96778148209fd3461 HTTP/1.1" 200 None 2026-03-26 16:42:01,122 - lava_callback - DEBUG - Processing node 69c2bec96778148209fd3461: baseline-arm-clabbe 2026-03-26 16:42:01,122 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2bec96778148209fd3461 2026-03-26 16:42:01,123 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:42:01,147 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2bec96778148209fd3461 HTTP/1.1" 200 None 2026-03-26 16:42:01,149 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:42:01,163 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b94b6778148209fd336c HTTP/1.1" 200 None 2026-03-26 16:42:01,164 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:42:01,185 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2bec96778148209fd3461 HTTP/1.1" 200 None 2026-03-26 16:42:01,196 - lava_callback - ERROR - Error processing callback for node 69c2bec96778148209fd3461: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-26 16:42:01,789 - lava_callback - INFO - Received callback for node 69c2beca6778148209fd3488 INFO: 172.21.0.1:46716 - "POST /node/69c2beca6778148209fd3488 HTTP/1.0" 202 Accepted 2026-03-26 16:42:01,795 - lava_callback - INFO - Starting async job submit for node 69c2beca6778148209fd3488 2026-03-26 16:42:01,796 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:42:01,810 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2beca6778148209fd3488 HTTP/1.1" 200 None 2026-03-26 16:42:01,811 - lava_callback - DEBUG - Processing node 69c2beca6778148209fd3488: baseline-arm-clabbe 2026-03-26 16:42:01,812 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2beca6778148209fd3488 2026-03-26 16:42:01,812 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:42:01,835 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2beca6778148209fd3488 HTTP/1.1" 200 None 2026-03-26 16:42:01,837 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:42:01,850 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b94b6778148209fd336c HTTP/1.1" 200 None 2026-03-26 16:42:01,852 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:42:01,872 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2beca6778148209fd3488 HTTP/1.1" 200 None 2026-03-26 16:42:01,877 - lava_callback - ERROR - Error processing callback for node 69c2beca6778148209fd3488: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-26 16:42:02,494 - lava_callback - INFO - Received callback for node 69c2becc6778148209fd3491 INFO: 172.21.0.1:46718 - "POST /node/69c2becc6778148209fd3491 HTTP/1.0" 202 Accepted 2026-03-26 16:42:02,500 - lava_callback - INFO - Starting async job submit for node 69c2becc6778148209fd3491 2026-03-26 16:42:02,501 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:42:02,521 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2becc6778148209fd3491 HTTP/1.1" 200 None 2026-03-26 16:42:02,522 - lava_callback - DEBUG - Processing node 69c2becc6778148209fd3491: baseline-arm-clabbe 2026-03-26 16:42:02,522 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2becc6778148209fd3491 2026-03-26 16:42:02,523 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:42:02,546 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2becc6778148209fd3491 HTTP/1.1" 200 None 2026-03-26 16:42:02,548 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:42:02,561 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b94b6778148209fd336c HTTP/1.1" 200 None 2026-03-26 16:42:02,563 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:42:02,583 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2becc6778148209fd3491 HTTP/1.1" 200 None 2026-03-26 16:42:02,589 - lava_callback - ERROR - Error processing callback for node 69c2becc6778148209fd3491: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:46730 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46740 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36948 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36956 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57340 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60550 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47282 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45628 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49510 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49522 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32948 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:32960 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54212 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36028 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60646 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50572 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36922 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36930 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54494 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54502 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34036 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34544 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34520 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41684 - "GET / HTTP/1.0" 200 OK 2026-03-26 16:45:01,628 - lava_callback - INFO - Received callback for node 69c2bf896778148209fd358f INFO: 172.21.0.1:55596 - "POST /node/69c2bf896778148209fd358f HTTP/1.0" 202 Accepted 2026-03-26 16:45:01,635 - lava_callback - INFO - Starting async job submit for node 69c2bf896778148209fd358f 2026-03-26 16:45:01,637 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:45:01,656 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2bf896778148209fd358f HTTP/1.1" 200 None 2026-03-26 16:45:01,658 - lava_callback - DEBUG - Processing node 69c2bf896778148209fd358f: baseline-arm-baylibre 2026-03-26 16:45:01,658 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2bf896778148209fd358f 2026-03-26 16:45:01,659 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:45:01,677 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2bf896778148209fd358f HTTP/1.1" 200 None 2026-03-26 16:45:01,679 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:45:01,698 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad09221a77312e2220ee1 HTTP/1.1" 200 None 2026-03-26 16:45:01,699 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:45:01,733 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2bf896778148209fd358f HTTP/1.1" 200 None 2026-03-26 16:45:01,734 - lava_callback - ERROR - Error processing callback for node 69c2bf896778148209fd358f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:55612 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55618 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33798 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33802 - "GET / HTTP/1.0" 200 OK 2026-03-26 16:45:23,890 - lava_callback - INFO - Received callback for node 69c56288abf66a8f4d9bbdbe INFO: 172.21.0.1:49034 - "POST /node/69c56288abf66a8f4d9bbdbe HTTP/1.0" 202 Accepted 2026-03-26 16:45:23,900 - lava_callback - INFO - Starting async job submit for node 69c56288abf66a8f4d9bbdbe 2026-03-26 16:45:23,931 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:45:23,962 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c56288abf66a8f4d9bbdbe HTTP/1.1" 200 None 2026-03-26 16:45:23,963 - lava_callback - DEBUG - Processing node 69c56288abf66a8f4d9bbdbe: baseline-x86 2026-03-26 16:45:24,252 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 16:45:24,413 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 16:45:24,415 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69c56288abf66a8f4d9bbdbe/log.txt.gz 2026-03-26 16:45:24,535 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 16:45:24,936 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 16:45:24,939 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69c56288abf66a8f4d9bbdbe/lava-logs.yaml 2026-03-26 16:45:24,944 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 16:45:25,078 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 16:45:25,080 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69c56288abf66a8f4d9bbdbe/lava_callback.json.gz 2026-03-26 16:45:25,098 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:45:25,112 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c56288abf66a8f4d9bbdbe HTTP/1.1" 200 None 2026-03-26 16:45:25,114 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:45:25,128 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c55c92abf66a8f4d9b967b HTTP/1.1" 200 None 2026-03-26 16:45:25,129 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:45:25,156 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c56288abf66a8f4d9bbdbe HTTP/1.1" 200 None 2026-03-26 16:45:25,158 - lava_callback - INFO - Completed processing callback for node 69c56288abf66a8f4d9bbdbe INFO: 172.21.0.1:49046 - "GET / HTTP/1.0" 200 OK 2026-03-26 16:45:31,635 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:45:31,655 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:48216 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58716 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36636 - "GET / HTTP/1.0" 200 OK 2026-03-26 16:45:58,168 - lava_callback - INFO - Received callback for node 69c5628aabf66a8f4d9bbdc7 INFO: 172.21.0.1:36646 - "POST /node/69c5628aabf66a8f4d9bbdc7 HTTP/1.0" 202 Accepted 2026-03-26 16:45:58,176 - lava_callback - INFO - Starting async job submit for node 69c5628aabf66a8f4d9bbdc7 2026-03-26 16:45:58,198 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:45:58,214 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5628aabf66a8f4d9bbdc7 HTTP/1.1" 200 None 2026-03-26 16:45:58,215 - lava_callback - DEBUG - Processing node 69c5628aabf66a8f4d9bbdc7: baseline-x86 2026-03-26 16:45:58,439 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 16:45:58,718 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 16:45:58,720 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69c5628aabf66a8f4d9bbdc7/log.txt.gz 2026-03-26 16:45:58,847 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 16:45:59,287 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 16:45:59,289 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69c5628aabf66a8f4d9bbdc7/lava-logs.yaml 2026-03-26 16:45:59,296 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 16:45:59,509 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 16:45:59,511 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69c5628aabf66a8f4d9bbdc7/lava_callback.json.gz 2026-03-26 16:45:59,576 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:45:59,601 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5628aabf66a8f4d9bbdc7 HTTP/1.1" 200 None 2026-03-26 16:45:59,603 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:45:59,617 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c55c92abf66a8f4d9b967b HTTP/1.1" 200 None 2026-03-26 16:45:59,619 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:45:59,650 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5628aabf66a8f4d9bbdc7 HTTP/1.1" 200 None 2026-03-26 16:45:59,662 - lava_callback - INFO - Completed processing callback for node 69c5628aabf66a8f4d9bbdc7 2026-03-26 16:46:01,658 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:46:01,681 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:55986 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55992 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39912 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39920 - "GET / HTTP/1.0" 200 OK 2026-03-26 16:46:24,331 - lava_callback - INFO - Received callback for node 69c56289abf66a8f4d9bbdc4 INFO: 172.21.0.1:33606 - "POST /node/69c56289abf66a8f4d9bbdc4 HTTP/1.0" 202 Accepted 2026-03-26 16:46:24,337 - lava_callback - INFO - Starting async job submit for node 69c56289abf66a8f4d9bbdc4 2026-03-26 16:46:24,357 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:46:24,371 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c56289abf66a8f4d9bbdc4 HTTP/1.1" 200 None 2026-03-26 16:46:24,372 - lava_callback - DEBUG - Processing node 69c56289abf66a8f4d9bbdc4: baseline-x86 2026-03-26 16:46:24,769 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 16:46:25,002 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 16:46:25,004 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69c56289abf66a8f4d9bbdc4/log.txt.gz 2026-03-26 16:46:25,222 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 16:46:25,439 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 16:46:25,442 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69c56289abf66a8f4d9bbdc4/lava-logs.yaml 2026-03-26 16:46:25,451 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 16:46:25,712 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 16:46:25,714 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69c56289abf66a8f4d9bbdc4/lava_callback.json.gz 2026-03-26 16:46:25,735 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:46:25,755 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c56289abf66a8f4d9bbdc4 HTTP/1.1" 200 None 2026-03-26 16:46:25,757 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:46:25,777 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c55c92abf66a8f4d9b967b HTTP/1.1" 200 None 2026-03-26 16:46:25,779 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:46:25,812 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c56289abf66a8f4d9bbdc4 HTTP/1.1" 200 None 2026-03-26 16:46:25,814 - lava_callback - INFO - Completed processing callback for node 69c56289abf66a8f4d9bbdc4 INFO: 172.21.0.1:33610 - "GET / HTTP/1.0" 200 OK 2026-03-26 16:46:26,599 - lava_callback - INFO - Received callback for node 69c562d5abf66a8f4d9bbe09 INFO: 172.21.0.1:33612 - "POST /node/69c562d5abf66a8f4d9bbe09 HTTP/1.0" 202 Accepted 2026-03-26 16:46:26,605 - lava_callback - INFO - Starting async job submit for node 69c562d5abf66a8f4d9bbe09 2026-03-26 16:46:26,608 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:46:26,622 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c562d5abf66a8f4d9bbe09 HTTP/1.1" 200 None 2026-03-26 16:46:26,623 - lava_callback - DEBUG - Processing node 69c562d5abf66a8f4d9bbe09: baseline-arm 2026-03-26 16:46:26,628 - lava_callback - WARNING - Failed to upload log 2026-03-26 16:46:26,631 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 16:46:26,819 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 16:46:26,821 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c562d5abf66a8f4d9bbe09/lava-logs.yaml 2026-03-26 16:46:26,822 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 16:46:26,930 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 16:46:26,932 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c562d5abf66a8f4d9bbe09/lava_callback.json.gz 2026-03-26 16:46:26,935 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:46:26,949 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c562d5abf66a8f4d9bbe09 HTTP/1.1" 200 None 2026-03-26 16:46:26,950 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:46:26,963 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c55c70abf66a8f4d9b9655 HTTP/1.1" 200 None 2026-03-26 16:46:26,965 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:46:26,992 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c562d5abf66a8f4d9bbe09 HTTP/1.1" 200 None 2026-03-26 16:46:27,001 - lava_callback - INFO - Completed processing callback for node 69c562d5abf66a8f4d9bbe09 2026-03-26 16:46:31,029 - lava_callback - INFO - Received callback for node 69c56289abf66a8f4d9bbdc1 INFO: 172.21.0.1:50276 - "POST /node/69c56289abf66a8f4d9bbdc1 HTTP/1.0" 202 Accepted 2026-03-26 16:46:31,036 - lava_callback - INFO - Starting async job submit for node 69c56289abf66a8f4d9bbdc1 2026-03-26 16:46:31,055 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:46:31,071 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c56289abf66a8f4d9bbdc1 HTTP/1.1" 200 None 2026-03-26 16:46:31,072 - lava_callback - DEBUG - Processing node 69c56289abf66a8f4d9bbdc1: baseline-x86 2026-03-26 16:46:31,342 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 16:46:31,508 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 16:46:31,509 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69c56289abf66a8f4d9bbdc1/log.txt.gz 2026-03-26 16:46:31,688 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:46:31,742 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 16:46:31,747 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-26 16:46:31,966 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 16:46:31,968 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69c56289abf66a8f4d9bbdc1/lava-logs.yaml 2026-03-26 16:46:31,977 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 16:46:32,127 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 16:46:32,129 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69c56289abf66a8f4d9bbdc1/lava_callback.json.gz 2026-03-26 16:46:32,156 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:46:32,205 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c56289abf66a8f4d9bbdc1 HTTP/1.1" 200 None 2026-03-26 16:46:32,209 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:46:32,237 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c55c92abf66a8f4d9b967b HTTP/1.1" 200 None 2026-03-26 16:46:32,239 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:46:32,282 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c56289abf66a8f4d9bbdc1 HTTP/1.1" 200 None 2026-03-26 16:46:32,286 - lava_callback - INFO - Completed processing callback for node 69c56289abf66a8f4d9bbdc1 INFO: 172.21.0.1:50282 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34302 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33638 - "GET / HTTP/1.0" 200 OK 2026-03-26 16:46:58,676 - lava_callback - INFO - Received callback for node 69c5628cabf66a8f4d9bbdca INFO: 172.21.0.1:33648 - "POST /node/69c5628cabf66a8f4d9bbdca HTTP/1.0" 202 Accepted 2026-03-26 16:46:58,682 - lava_callback - INFO - Starting async job submit for node 69c5628cabf66a8f4d9bbdca 2026-03-26 16:46:58,700 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:46:58,715 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5628cabf66a8f4d9bbdca HTTP/1.1" 200 None 2026-03-26 16:46:58,716 - lava_callback - DEBUG - Processing node 69c5628cabf66a8f4d9bbdca: baseline-x86 2026-03-26 16:46:59,007 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 16:46:59,167 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 16:46:59,169 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69c5628cabf66a8f4d9bbdca/log.txt.gz 2026-03-26 16:46:59,205 - lava_callback - INFO - Received callback for node 69c562d9abf66a8f4d9bbe15 2026-03-26 16:46:59,258 - lava_callback - INFO - Starting async job submit for node 69c562d9abf66a8f4d9bbe15 INFO: 172.21.0.1:33650 - "POST /node/69c562d9abf66a8f4d9bbe15 HTTP/1.0" 202 Accepted 2026-03-26 16:46:59,299 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:46:59,361 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c562d9abf66a8f4d9bbe15 HTTP/1.1" 200 None 2026-03-26 16:46:59,368 - lava_callback - DEBUG - Processing node 69c562d9abf66a8f4d9bbe15: baseline-arm 2026-03-26 16:46:59,406 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 16:46:59,544 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 16:46:59,729 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 16:46:59,731 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69c562d9abf66a8f4d9bbe15/log.txt.gz 2026-03-26 16:46:59,749 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 16:46:59,785 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69c5628cabf66a8f4d9bbdca/lava-logs.yaml 2026-03-26 16:46:59,812 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 16:46:59,813 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 16:46:59,990 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 16:46:59,992 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c562d9abf66a8f4d9bbe15/lava-logs.yaml 2026-03-26 16:46:59,996 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 16:47:00,016 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 16:47:00,019 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69c5628cabf66a8f4d9bbdca/lava_callback.json.gz 2026-03-26 16:47:00,047 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:47:00,063 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5628cabf66a8f4d9bbdca HTTP/1.1" 200 None 2026-03-26 16:47:00,065 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:47:00,080 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c55c92abf66a8f4d9b967b HTTP/1.1" 200 None 2026-03-26 16:47:00,082 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:47:00,111 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5628cabf66a8f4d9bbdca HTTP/1.1" 200 None 2026-03-26 16:47:00,115 - lava_callback - INFO - Completed processing callback for node 69c5628cabf66a8f4d9bbdca 2026-03-26 16:47:00,146 - lava_callback - INFO - Received callback for node 69c562d6abf66a8f4d9bbe0c 2026-03-26 16:47:00,155 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 16:47:00,157 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c562d9abf66a8f4d9bbe15/lava_callback.json.gz INFO: 172.21.0.1:41914 - "POST /node/69c562d6abf66a8f4d9bbe0c HTTP/1.0" 202 Accepted 2026-03-26 16:47:00,204 - lava_callback - INFO - Starting async job submit for node 69c562d6abf66a8f4d9bbe0c 2026-03-26 16:47:00,210 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:47:00,225 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:47:00,245 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c562d9abf66a8f4d9bbe15 HTTP/1.1" 200 None 2026-03-26 16:47:00,247 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:47:00,247 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c562d6abf66a8f4d9bbe0c HTTP/1.1" 200 None 2026-03-26 16:47:00,249 - lava_callback - DEBUG - Processing node 69c562d6abf66a8f4d9bbe0c: baseline-arm 2026-03-26 16:47:00,344 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c55c70abf66a8f4d9b9655 HTTP/1.1" 200 None 2026-03-26 16:47:00,378 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:47:00,427 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 16:47:00,442 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c562d9abf66a8f4d9bbe15 HTTP/1.1" 200 None 2026-03-26 16:47:00,444 - lava_callback - INFO - Completed processing callback for node 69c562d9abf66a8f4d9bbe15 2026-03-26 16:47:00,680 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 16:47:00,683 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69c562d6abf66a8f4d9bbe0c/log.txt.gz 2026-03-26 16:47:00,771 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 16:47:00,924 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 16:47:00,926 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c562d6abf66a8f4d9bbe0c/lava-logs.yaml 2026-03-26 16:47:00,930 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 16:47:01,041 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 16:47:01,043 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c562d6abf66a8f4d9bbe0c/lava_callback.json.gz 2026-03-26 16:47:01,062 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:47:01,081 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c562d6abf66a8f4d9bbe0c HTTP/1.1" 200 None 2026-03-26 16:47:01,082 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:47:01,096 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c55c70abf66a8f4d9b9655 HTTP/1.1" 200 None 2026-03-26 16:47:01,097 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:47:01,121 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c562d6abf66a8f4d9bbe0c HTTP/1.1" 200 None 2026-03-26 16:47:01,122 - lava_callback - INFO - Completed processing callback for node 69c562d6abf66a8f4d9bbe0c 2026-03-26 16:47:01,751 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:47:01,783 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 INFO: 172.21.0.1:41926 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41942 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35574 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-26 16:47:15,429 - lava_callback - INFO - Received callback for node 69c562d8abf66a8f4d9bbe12 INFO: 172.21.0.1:35586 - "POST /node/69c562d8abf66a8f4d9bbe12 HTTP/1.0" 202 Accepted 2026-03-26 16:47:15,436 - lava_callback - INFO - Starting async job submit for node 69c562d8abf66a8f4d9bbe12 2026-03-26 16:47:15,458 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:47:15,481 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c562d8abf66a8f4d9bbe12 HTTP/1.1" 200 None 2026-03-26 16:47:15,482 - lava_callback - DEBUG - Processing node 69c562d8abf66a8f4d9bbe12: baseline-arm 2026-03-26 16:47:15,694 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 16:47:15,822 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 16:47:15,823 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69c562d8abf66a8f4d9bbe12/log.txt.gz 2026-03-26 16:47:16,024 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 16:47:16,220 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 16:47:16,222 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c562d8abf66a8f4d9bbe12/lava-logs.yaml 2026-03-26 16:47:16,228 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 16:47:16,366 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 16:47:16,368 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c562d8abf66a8f4d9bbe12/lava_callback.json.gz 2026-03-26 16:47:16,392 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:47:16,408 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c562d8abf66a8f4d9bbe12 HTTP/1.1" 200 None 2026-03-26 16:47:16,409 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:47:16,428 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c55c70abf66a8f4d9b9655 HTTP/1.1" 200 None 2026-03-26 16:47:16,430 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:47:16,454 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c562d8abf66a8f4d9bbe12 HTTP/1.1" 200 None 2026-03-26 16:47:16,456 - lava_callback - INFO - Completed processing callback for node 69c562d8abf66a8f4d9bbe12 INFO: 172.21.0.1:35596 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37038 - "GET / HTTP/1.0" 200 OK 2026-03-26 16:47:30,890 - lava_callback - INFO - Received callback for node 69c562d5abf66a8f4d9bbe06 INFO: 172.21.0.1:46034 - "POST /node/69c562d5abf66a8f4d9bbe06 HTTP/1.0" 202 Accepted 2026-03-26 16:47:30,900 - lava_callback - INFO - Starting async job submit for node 69c562d5abf66a8f4d9bbe06 2026-03-26 16:47:31,012 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:47:31,045 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c562d5abf66a8f4d9bbe06 HTTP/1.1" 200 None 2026-03-26 16:47:31,046 - lava_callback - DEBUG - Processing node 69c562d5abf66a8f4d9bbe06: baseline-arm 2026-03-26 16:47:31,455 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 16:47:31,632 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 16:47:31,634 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69c562d5abf66a8f4d9bbe06/log.txt.gz 2026-03-26 16:47:31,791 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:47:31,851 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 16:47:31,869 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-26 16:47:32,187 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 16:47:32,190 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c562d5abf66a8f4d9bbe06/lava-logs.yaml 2026-03-26 16:47:32,197 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 16:47:32,411 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 16:47:32,413 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c562d5abf66a8f4d9bbe06/lava_callback.json.gz 2026-03-26 16:47:32,467 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:47:32,480 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c562d5abf66a8f4d9bbe06 HTTP/1.1" 200 None 2026-03-26 16:47:32,482 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:47:32,496 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c55c70abf66a8f4d9b9655 HTTP/1.1" 200 None 2026-03-26 16:47:32,497 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:47:32,521 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c562d5abf66a8f4d9bbe06 HTTP/1.1" 200 None 2026-03-26 16:47:32,526 - lava_callback - INFO - Completed processing callback for node 69c562d5abf66a8f4d9bbe06 INFO: 172.21.0.1:46050 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52074 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46156 - "GET / HTTP/1.0" 200 OK 2026-03-26 16:47:57,269 - lava_callback - INFO - Received callback for node 69c562e8abf66a8f4d9bbe5b INFO: 172.21.0.1:46172 - "POST /node/69c562e8abf66a8f4d9bbe5b HTTP/1.0" 202 Accepted 2026-03-26 16:47:57,275 - lava_callback - INFO - Starting async job submit for node 69c562e8abf66a8f4d9bbe5b 2026-03-26 16:47:57,298 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:47:57,319 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c562e8abf66a8f4d9bbe5b HTTP/1.1" 200 None 2026-03-26 16:47:57,320 - lava_callback - DEBUG - Processing node 69c562e8abf66a8f4d9bbe5b: baseline-arm-broonie 2026-03-26 16:47:57,529 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 16:47:57,685 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 16:47:57,687 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69c562e8abf66a8f4d9bbe5b/log.txt.gz 2026-03-26 16:47:57,813 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 16:47:57,992 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 16:47:57,994 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69c562e8abf66a8f4d9bbe5b/lava-logs.yaml 2026-03-26 16:47:57,999 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 16:47:58,130 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 16:47:58,132 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69c562e8abf66a8f4d9bbe5b/lava_callback.json.gz 2026-03-26 16:47:58,155 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:47:58,177 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c562e8abf66a8f4d9bbe5b HTTP/1.1" 200 None 2026-03-26 16:47:58,178 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:47:58,192 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c55c70abf66a8f4d9b9655 HTTP/1.1" 200 None 2026-03-26 16:47:58,193 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:47:58,217 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c562e8abf66a8f4d9bbe5b HTTP/1.1" 200 None 2026-03-26 16:47:58,219 - lava_callback - INFO - Completed processing callback for node 69c562e8abf66a8f4d9bbe5b 2026-03-26 16:48:01,871 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:48:01,888 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-26 16:48:04,043 - lava_callback - INFO - Received callback for node 69c562fcabf66a8f4d9bbe9e INFO: 172.21.0.1:36724 - "POST /node/69c562fcabf66a8f4d9bbe9e HTTP/1.0" 202 Accepted 2026-03-26 16:48:04,090 - lava_callback - INFO - Starting async job submit for node 69c562fcabf66a8f4d9bbe9e 2026-03-26 16:48:04,109 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:48:04,134 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c562fcabf66a8f4d9bbe9e HTTP/1.1" 200 None 2026-03-26 16:48:04,135 - lava_callback - DEBUG - Processing node 69c562fcabf66a8f4d9bbe9e: baseline-arm-pengutronix 2026-03-26 16:48:04,240 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 16:48:04,384 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 16:48:04,386 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c562fcabf66a8f4d9bbe9e/log.txt.gz 2026-03-26 16:48:04,452 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 16:48:04,647 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 16:48:04,649 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c562fcabf66a8f4d9bbe9e/lava-logs.yaml 2026-03-26 16:48:04,652 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 16:48:04,805 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 16:48:04,806 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c562fcabf66a8f4d9bbe9e/lava_callback.json.gz 2026-03-26 16:48:04,827 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:48:04,842 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c562fcabf66a8f4d9bbe9e HTTP/1.1" 200 None 2026-03-26 16:48:04,843 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:48:04,856 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c55c70abf66a8f4d9b9655 HTTP/1.1" 200 None 2026-03-26 16:48:04,858 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:48:04,882 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c562fcabf66a8f4d9bbe9e HTTP/1.1" 200 None 2026-03-26 16:48:04,887 - lava_callback - INFO - Completed processing callback for node 69c562fcabf66a8f4d9bbe9e INFO: 172.21.0.1:36738 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36744 - "GET / HTTP/1.0" 200 OK 2026-03-26 16:48:09,578 - lava_callback - INFO - Received callback for node 69c562f9abf66a8f4d9bbe95 INFO: 172.21.0.1:36754 - "POST /node/69c562f9abf66a8f4d9bbe95 HTTP/1.0" 202 Accepted 2026-03-26 16:48:09,584 - lava_callback - INFO - Starting async job submit for node 69c562f9abf66a8f4d9bbe95 2026-03-26 16:48:09,602 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:48:09,621 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c562f9abf66a8f4d9bbe95 HTTP/1.1" 200 None 2026-03-26 16:48:09,622 - lava_callback - DEBUG - Processing node 69c562f9abf66a8f4d9bbe95: baseline-arm-pengutronix 2026-03-26 16:48:09,762 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 16:48:09,875 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 16:48:09,876 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c562f9abf66a8f4d9bbe95/log.txt.gz 2026-03-26 16:48:09,960 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 16:48:10,163 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 16:48:10,165 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c562f9abf66a8f4d9bbe95/lava-logs.yaml 2026-03-26 16:48:10,169 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 16:48:10,319 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 16:48:10,321 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c562f9abf66a8f4d9bbe95/lava_callback.json.gz 2026-03-26 16:48:10,339 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:48:10,354 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c562f9abf66a8f4d9bbe95 HTTP/1.1" 200 None 2026-03-26 16:48:10,355 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:48:10,368 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c55c70abf66a8f4d9b9655 HTTP/1.1" 200 None 2026-03-26 16:48:10,370 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:48:10,394 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c562f9abf66a8f4d9bbe95 HTTP/1.1" 200 None 2026-03-26 16:48:10,395 - lava_callback - INFO - Completed processing callback for node 69c562f9abf66a8f4d9bbe95 2026-03-26 16:48:10,776 - lava_callback - INFO - Received callback for node 69c562faabf66a8f4d9bbe98 INFO: 172.21.0.1:53158 - "POST /node/69c562faabf66a8f4d9bbe98 HTTP/1.0" 202 Accepted 2026-03-26 16:48:10,783 - lava_callback - INFO - Starting async job submit for node 69c562faabf66a8f4d9bbe98 2026-03-26 16:48:10,804 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:48:10,821 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c562faabf66a8f4d9bbe98 HTTP/1.1" 200 None 2026-03-26 16:48:10,822 - lava_callback - DEBUG - Processing node 69c562faabf66a8f4d9bbe98: baseline-arm-pengutronix 2026-03-26 16:48:10,996 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 16:48:11,116 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 16:48:11,118 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c562faabf66a8f4d9bbe98/log.txt.gz 2026-03-26 16:48:11,207 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 16:48:11,379 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 16:48:11,381 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c562faabf66a8f4d9bbe98/lava-logs.yaml 2026-03-26 16:48:11,386 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 16:48:11,512 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 16:48:11,514 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c562faabf66a8f4d9bbe98/lava_callback.json.gz 2026-03-26 16:48:11,534 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:48:11,559 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c562faabf66a8f4d9bbe98 HTTP/1.1" 200 None 2026-03-26 16:48:11,560 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:48:11,574 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c55c70abf66a8f4d9b9655 HTTP/1.1" 200 None 2026-03-26 16:48:11,575 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:48:11,601 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c562faabf66a8f4d9bbe98 HTTP/1.1" 200 None 2026-03-26 16:48:11,602 - lava_callback - INFO - Completed processing callback for node 69c562faabf66a8f4d9bbe98 INFO: 172.21.0.1:53164 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-26 16:48:16,067 - lava_callback - INFO - Received callback for node 69c562e6abf66a8f4d9bbe3b INFO: 172.21.0.1:53166 - "POST /node/69c562e6abf66a8f4d9bbe3b HTTP/1.0" 202 Accepted 2026-03-26 16:48:16,073 - lava_callback - INFO - Starting async job submit for node 69c562e6abf66a8f4d9bbe3b 2026-03-26 16:48:16,096 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:48:16,111 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c562e6abf66a8f4d9bbe3b HTTP/1.1" 200 None 2026-03-26 16:48:16,112 - lava_callback - DEBUG - Processing node 69c562e6abf66a8f4d9bbe3b: baseline-arm-broonie 2026-03-26 16:48:16,312 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 16:48:16,431 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 16:48:16,433 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69c562e6abf66a8f4d9bbe3b/log.txt.gz INFO: 172.21.0.1:53180 - "GET / HTTP/1.0" 200 OK 2026-03-26 16:48:16,551 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 16:48:16,722 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 16:48:16,724 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69c562e6abf66a8f4d9bbe3b/lava-logs.yaml 2026-03-26 16:48:16,729 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 16:48:16,871 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 16:48:16,873 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69c562e6abf66a8f4d9bbe3b/lava_callback.json.gz 2026-03-26 16:48:16,896 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:48:16,922 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c562e6abf66a8f4d9bbe3b HTTP/1.1" 200 None 2026-03-26 16:48:16,923 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:48:16,941 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c55c70abf66a8f4d9b9655 HTTP/1.1" 200 None 2026-03-26 16:48:16,943 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:48:16,967 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c562e6abf66a8f4d9bbe3b HTTP/1.1" 200 None 2026-03-26 16:48:16,968 - lava_callback - INFO - Completed processing callback for node 69c562e6abf66a8f4d9bbe3b 2026-03-26 16:48:25,991 - lava_callback - INFO - Received callback for node 69c562e9abf66a8f4d9bbe62 INFO: 172.21.0.1:33516 - "POST /node/69c562e9abf66a8f4d9bbe62 HTTP/1.0" 202 Accepted 2026-03-26 16:48:25,997 - lava_callback - INFO - Starting async job submit for node 69c562e9abf66a8f4d9bbe62 2026-03-26 16:48:26,020 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:48:26,035 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c562e9abf66a8f4d9bbe62 HTTP/1.1" 200 None 2026-03-26 16:48:26,036 - lava_callback - DEBUG - Processing node 69c562e9abf66a8f4d9bbe62: baseline-arm-broonie 2026-03-26 16:48:26,261 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 16:48:26,386 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 16:48:26,387 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69c562e9abf66a8f4d9bbe62/log.txt.gz INFO: 172.21.0.1:33522 - "GET / HTTP/1.0" 200 OK 2026-03-26 16:48:26,528 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 16:48:26,699 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 16:48:26,701 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69c562e9abf66a8f4d9bbe62/lava-logs.yaml 2026-03-26 16:48:26,707 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 16:48:26,838 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 16:48:26,840 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69c562e9abf66a8f4d9bbe62/lava_callback.json.gz 2026-03-26 16:48:26,863 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:48:26,878 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c562e9abf66a8f4d9bbe62 HTTP/1.1" 200 None 2026-03-26 16:48:26,880 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:48:26,895 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c55c70abf66a8f4d9b9655 HTTP/1.1" 200 None 2026-03-26 16:48:26,896 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:48:26,926 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c562e9abf66a8f4d9bbe62 HTTP/1.1" 200 None 2026-03-26 16:48:26,927 - lava_callback - INFO - Completed processing callback for node 69c562e9abf66a8f4d9bbe62 2026-03-26 16:48:29,204 - lava_callback - INFO - Received callback for node 69c562eeabf66a8f4d9bbe71 INFO: 172.21.0.1:33526 - "POST /node/69c562eeabf66a8f4d9bbe71 HTTP/1.0" 202 Accepted 2026-03-26 16:48:29,210 - lava_callback - INFO - Starting async job submit for node 69c562eeabf66a8f4d9bbe71 2026-03-26 16:48:29,232 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:48:29,249 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c562eeabf66a8f4d9bbe71 HTTP/1.1" 200 None 2026-03-26 16:48:29,249 - lava_callback - DEBUG - Processing node 69c562eeabf66a8f4d9bbe71: baseline-arm-clabbe 2026-03-26 16:48:29,483 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 16:48:29,599 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 16:48:29,601 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69c562eeabf66a8f4d9bbe71/log.txt.gz 2026-03-26 16:48:29,741 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 16:48:29,927 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 16:48:29,929 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69c562eeabf66a8f4d9bbe71/lava-logs.yaml 2026-03-26 16:48:29,936 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 16:48:30,072 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 16:48:30,074 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69c562eeabf66a8f4d9bbe71/lava_callback.json.gz 2026-03-26 16:48:30,100 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:48:30,124 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c562eeabf66a8f4d9bbe71 HTTP/1.1" 200 None 2026-03-26 16:48:30,125 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:48:30,140 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c55c70abf66a8f4d9b9655 HTTP/1.1" 200 None 2026-03-26 16:48:30,142 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:48:30,166 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c562eeabf66a8f4d9bbe71 HTTP/1.1" 200 None 2026-03-26 16:48:30,171 - lava_callback - INFO - Completed processing callback for node 69c562eeabf66a8f4d9bbe71 2026-03-26 16:48:31,890 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:48:31,906 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 INFO: 172.21.0.1:42838 - "GET / HTTP/1.0" 200 OK 2026-03-26 16:48:46,047 - lava_callback - INFO - Received callback for node 69c562e5abf66a8f4d9bbe38 INFO: 172.21.0.1:35348 - "POST /node/69c562e5abf66a8f4d9bbe38 HTTP/1.0" 202 Accepted 2026-03-26 16:48:46,053 - lava_callback - INFO - Starting async job submit for node 69c562e5abf66a8f4d9bbe38 2026-03-26 16:48:46,075 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:48:46,090 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c562e5abf66a8f4d9bbe38 HTTP/1.1" 200 None 2026-03-26 16:48:46,091 - lava_callback - DEBUG - Processing node 69c562e5abf66a8f4d9bbe38: baseline-arm-broonie 2026-03-26 16:48:46,297 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 16:48:46,415 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 16:48:46,417 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69c562e5abf66a8f4d9bbe38/log.txt.gz INFO: 172.21.0.1:35362 - "GET / HTTP/1.0" 200 OK 2026-03-26 16:48:46,544 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 16:48:46,758 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 16:48:46,759 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69c562e5abf66a8f4d9bbe38/lava-logs.yaml 2026-03-26 16:48:46,765 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 16:48:46,901 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 16:48:46,903 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69c562e5abf66a8f4d9bbe38/lava_callback.json.gz 2026-03-26 16:48:46,926 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:48:46,940 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c562e5abf66a8f4d9bbe38 HTTP/1.1" 200 None 2026-03-26 16:48:46,941 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:48:46,954 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c55c70abf66a8f4d9b9655 HTTP/1.1" 200 None 2026-03-26 16:48:46,956 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:48:46,984 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c562e5abf66a8f4d9bbe38 HTTP/1.1" 200 None 2026-03-26 16:48:46,985 - lava_callback - INFO - Completed processing callback for node 69c562e5abf66a8f4d9bbe38 INFO: 172.21.0.1:50378 - "GET / HTTP/1.0" 200 OK 2026-03-26 16:49:01,908 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:49:01,924 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:41816 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41818 - "GET / HTTP/1.0" 200 OK 2026-03-26 16:49:07,133 - lava_callback - INFO - Received callback for node 69c56313abf66a8f4d9bbeea INFO: 172.21.0.1:41834 - "POST /node/69c56313abf66a8f4d9bbeea HTTP/1.0" 202 Accepted 2026-03-26 16:49:07,140 - lava_callback - INFO - Starting async job submit for node 69c56313abf66a8f4d9bbeea 2026-03-26 16:49:07,163 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:49:07,178 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c56313abf66a8f4d9bbeea HTTP/1.1" 200 None 2026-03-26 16:49:07,179 - lava_callback - DEBUG - Processing node 69c56313abf66a8f4d9bbeea: ltp-dio 2026-03-26 16:49:07,565 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 16:49:07,707 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 16:49:07,708 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-dio-69c56313abf66a8f4d9bbeea/log.txt.gz 2026-03-26 16:49:07,924 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 16:49:08,162 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 16:49:08,164 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-dio-69c56313abf66a8f4d9bbeea/lava-logs.yaml 2026-03-26 16:49:08,173 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 16:49:08,324 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 16:49:08,326 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-dio-69c56313abf66a8f4d9bbeea/lava_callback.json.gz 2026-03-26 16:49:08,351 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:49:08,365 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c56313abf66a8f4d9bbeea HTTP/1.1" 200 None 2026-03-26 16:49:08,366 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:49:08,380 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c55c70abf66a8f4d9b9655 HTTP/1.1" 200 None 2026-03-26 16:49:08,382 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:49:08,406 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c56313abf66a8f4d9bbeea HTTP/1.1" 200 None 2026-03-26 16:49:08,408 - lava_callback - INFO - Completed processing callback for node 69c56313abf66a8f4d9bbeea INFO: 172.21.0.1:39408 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39416 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34854 - "GET / HTTP/1.0" 200 OK 2026-03-26 16:49:31,926 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:49:31,942 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:49790 - "GET / HTTP/1.0" 200 OK 2026-03-26 16:49:41,991 - lava_callback - INFO - Received callback for node 69c2c0a06778148209fd364e INFO: 172.21.0.1:59346 - "POST /node/69c2c0a06778148209fd364e HTTP/1.0" 202 Accepted 2026-03-26 16:49:41,997 - lava_callback - INFO - Starting async job submit for node 69c2c0a06778148209fd364e 2026-03-26 16:49:41,998 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:49:42,015 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2c0a06778148209fd364e HTTP/1.1" 200 None 2026-03-26 16:49:42,016 - lava_callback - DEBUG - Processing node 69c2c0a06778148209fd364e: baseline-arm-baylibre 2026-03-26 16:49:42,016 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2c0a06778148209fd364e 2026-03-26 16:49:42,017 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:49:42,030 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2c0a06778148209fd364e HTTP/1.1" 200 None 2026-03-26 16:49:42,031 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:49:42,044 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad09221a77312e2220ee1 HTTP/1.1" 200 None 2026-03-26 16:49:42,046 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:49:42,066 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2c0a06778148209fd364e HTTP/1.1" 200 None 2026-03-26 16:49:42,067 - lava_callback - ERROR - Error processing callback for node 69c2c0a06778148209fd364e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:59348 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33532 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52388 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52390 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50598 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50600 - "GET / HTTP/1.0" 200 OK 2026-03-26 16:50:16,543 - lava_callback - INFO - Received callback for node 69c562efabf66a8f4d9bbe75 INFO: 172.21.0.1:50606 - "POST /node/69c562efabf66a8f4d9bbe75 HTTP/1.0" 202 Accepted 2026-03-26 16:50:16,550 - lava_callback - INFO - Starting async job submit for node 69c562efabf66a8f4d9bbe75 2026-03-26 16:50:16,571 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:50:16,587 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c562efabf66a8f4d9bbe75 HTTP/1.1" 200 None 2026-03-26 16:50:16,588 - lava_callback - DEBUG - Processing node 69c562efabf66a8f4d9bbe75: baseline-arm-clabbe 2026-03-26 16:50:16,801 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 16:50:16,971 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 16:50:16,973 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69c562efabf66a8f4d9bbe75/log.txt.gz 2026-03-26 16:50:17,095 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 16:50:17,285 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 16:50:17,287 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69c562efabf66a8f4d9bbe75/lava-logs.yaml 2026-03-26 16:50:17,292 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 16:50:17,455 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 16:50:17,457 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69c562efabf66a8f4d9bbe75/lava_callback.json.gz 2026-03-26 16:50:17,479 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:50:17,493 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c562efabf66a8f4d9bbe75 HTTP/1.1" 200 None 2026-03-26 16:50:17,495 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:50:17,509 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c55c70abf66a8f4d9b9655 HTTP/1.1" 200 None 2026-03-26 16:50:17,510 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:50:17,562 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c562efabf66a8f4d9bbe75 HTTP/1.1" 200 None 2026-03-26 16:50:17,564 - lava_callback - INFO - Completed processing callback for node 69c562efabf66a8f4d9bbe75 INFO: 172.21.0.1:49794 - "GET / HTTP/1.0" 200 OK 2026-03-26 16:50:31,944 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:50:31,967 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:37428 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35532 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53086 - "GET / HTTP/1.0" 200 OK 2026-03-26 16:51:03,377 - lava_callback - INFO - Received callback for node 69c563efabf66a8f4d9bbfb8 INFO: 172.21.0.1:50402 - "POST /node/69c563efabf66a8f4d9bbfb8 HTTP/1.0" 202 Accepted 2026-03-26 16:51:03,383 - lava_callback - INFO - Starting async job submit for node 69c563efabf66a8f4d9bbfb8 2026-03-26 16:51:03,386 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:51:03,400 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c563efabf66a8f4d9bbfb8 HTTP/1.1" 200 None 2026-03-26 16:51:03,401 - lava_callback - DEBUG - Processing node 69c563efabf66a8f4d9bbfb8: baseline-arm 2026-03-26 16:51:03,406 - lava_callback - WARNING - Failed to upload log 2026-03-26 16:51:03,409 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 16:51:03,574 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 16:51:03,576 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c563efabf66a8f4d9bbfb8/lava-logs.yaml 2026-03-26 16:51:03,577 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 16:51:03,697 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 16:51:03,699 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c563efabf66a8f4d9bbfb8/lava_callback.json.gz 2026-03-26 16:51:03,702 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:51:03,717 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c563efabf66a8f4d9bbfb8 HTTP/1.1" 200 None 2026-03-26 16:51:03,718 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:51:03,732 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c55c70abf66a8f4d9b9655 HTTP/1.1" 200 None 2026-03-26 16:51:03,733 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:51:03,754 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c563efabf66a8f4d9bbfb8 HTTP/1.1" 200 None 2026-03-26 16:51:03,760 - lava_callback - INFO - Completed processing callback for node 69c563efabf66a8f4d9bbfb8 INFO: 172.21.0.1:50418 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50422 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56084 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56096 - "GET / HTTP/1.0" 200 OK 2026-03-26 16:51:20,288 - lava_callback - INFO - Received callback for node 69c2c0f56778148209fd5d2b INFO: 172.21.0.1:40256 - "POST /node/69c2c0f56778148209fd5d2b HTTP/1.0" 202 Accepted 2026-03-26 16:51:20,298 - lava_callback - INFO - Starting async job submit for node 69c2c0f56778148209fd5d2b 2026-03-26 16:51:20,299 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:51:20,337 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2c0f56778148209fd5d2b HTTP/1.1" 200 None 2026-03-26 16:51:20,339 - lava_callback - DEBUG - Processing node 69c2c0f56778148209fd5d2b: baseline-arm-clabbe 2026-03-26 16:51:20,339 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2c0f56778148209fd5d2b 2026-03-26 16:51:20,340 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:51:20,360 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2c0f56778148209fd5d2b HTTP/1.1" 200 None 2026-03-26 16:51:20,362 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:51:20,376 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad09221a77312e2220ee1 HTTP/1.1" 200 None 2026-03-26 16:51:20,377 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:51:20,417 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2c0f56778148209fd5d2b HTTP/1.1" 200 None 2026-03-26 16:51:20,419 - lava_callback - ERROR - Error processing callback for node 69c2c0f56778148209fd5d2b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:40272 - "GET / HTTP/1.0" 200 OK 2026-03-26 16:51:31,970 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:51:32,011 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-26 16:51:36,353 - lava_callback - INFO - Received callback for node 69c5630dabf66a8f4d9bbede INFO: 172.21.0.1:40864 - "POST /node/69c5630dabf66a8f4d9bbede HTTP/1.0" 202 Accepted 2026-03-26 16:51:36,363 - lava_callback - INFO - Starting async job submit for node 69c5630dabf66a8f4d9bbede 2026-03-26 16:51:36,402 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:51:36,424 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5630dabf66a8f4d9bbede HTTP/1.1" 200 None 2026-03-26 16:51:36,425 - lava_callback - DEBUG - Processing node 69c5630dabf66a8f4d9bbede: ltp-cap_bounds INFO: 172.21.0.1:40872 - "GET / HTTP/1.0" 200 OK 2026-03-26 16:51:36,921 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 16:51:37,444 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 16:51:37,446 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-cap_bounds-69c5630dabf66a8f4d9bbede/log.txt.gz 2026-03-26 16:51:37,747 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 16:51:38,321 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 16:51:38,324 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-cap_bounds-69c5630dabf66a8f4d9bbede/lava-logs.yaml 2026-03-26 16:51:38,335 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 16:51:38,886 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 16:51:38,888 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-cap_bounds-69c5630dabf66a8f4d9bbede/lava_callback.json.gz 2026-03-26 16:51:38,919 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:51:38,941 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5630dabf66a8f4d9bbede HTTP/1.1" 200 None 2026-03-26 16:51:38,944 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:51:38,966 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c55c70abf66a8f4d9b9655 HTTP/1.1" 200 None 2026-03-26 16:51:38,969 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:51:39,008 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5630dabf66a8f4d9bbede HTTP/1.1" 200 None 2026-03-26 16:51:39,010 - lava_callback - INFO - Completed processing callback for node 69c5630dabf66a8f4d9bbede INFO: 172.21.0.1:47320 - "GET / HTTP/1.0" 200 OK 2026-03-26 16:51:55,363 - lava_callback - INFO - Received callback for node 69c562f3abf66a8f4d9bbe81 INFO: 172.21.0.1:46772 - "POST /node/69c562f3abf66a8f4d9bbe81 HTTP/1.0" 202 Accepted 2026-03-26 16:51:55,372 - lava_callback - INFO - Starting async job submit for node 69c562f3abf66a8f4d9bbe81 2026-03-26 16:51:55,408 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:51:55,438 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c562f3abf66a8f4d9bbe81 HTTP/1.1" 200 None 2026-03-26 16:51:55,439 - lava_callback - DEBUG - Processing node 69c562f3abf66a8f4d9bbe81: baseline-arm-clabbe 2026-03-26 16:51:55,743 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 16:51:56,298 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 16:51:56,300 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69c562f3abf66a8f4d9bbe81/log.txt.gz 2026-03-26 16:51:56,465 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.21.0.1:46778 - "GET / HTTP/1.0" 200 OK 2026-03-26 16:51:56,976 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 16:51:56,979 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69c562f3abf66a8f4d9bbe81/lava-logs.yaml 2026-03-26 16:51:56,986 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 16:51:57,441 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 16:51:57,443 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69c562f3abf66a8f4d9bbe81/lava_callback.json.gz 2026-03-26 16:51:57,471 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:51:57,494 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c562f3abf66a8f4d9bbe81 HTTP/1.1" 200 None 2026-03-26 16:51:57,496 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:51:57,517 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c55c70abf66a8f4d9b9655 HTTP/1.1" 200 None 2026-03-26 16:51:57,519 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:51:57,556 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c562f3abf66a8f4d9bbe81 HTTP/1.1" 200 None 2026-03-26 16:51:57,558 - lava_callback - INFO - Completed processing callback for node 69c562f3abf66a8f4d9bbe81 2026-03-26 16:52:02,014 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:52:02,053 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-26 16:52:06,098 - lava_callback - INFO - Received callback for node 69c5630eabf66a8f4d9bbee1 INFO: 172.21.0.1:53448 - "POST /node/69c5630eabf66a8f4d9bbee1 HTTP/1.0" 202 Accepted 2026-03-26 16:52:06,109 - lava_callback - INFO - Starting async job submit for node 69c5630eabf66a8f4d9bbee1 2026-03-26 16:52:06,146 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:52:06,166 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5630eabf66a8f4d9bbee1 HTTP/1.1" 200 None 2026-03-26 16:52:06,167 - lava_callback - DEBUG - Processing node 69c5630eabf66a8f4d9bbee1: ltp-containers INFO: 172.21.0.1:53454 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53456 - "GET / HTTP/1.0" 200 OK 2026-03-26 16:52:06,719 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 16:52:07,240 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 16:52:07,242 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-containers-69c5630eabf66a8f4d9bbee1/log.txt.gz 2026-03-26 16:52:07,596 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 16:52:07,720 - lava_callback - INFO - Received callback for node 69c56320abf66a8f4d9bbf09 INFO: 172.21.0.1:53466 - "POST /node/69c56320abf66a8f4d9bbf09 HTTP/1.0" 202 Accepted 2026-03-26 16:52:07,732 - lava_callback - INFO - Starting async job submit for node 69c56320abf66a8f4d9bbf09 2026-03-26 16:52:07,791 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:52:07,812 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c56320abf66a8f4d9bbf09 HTTP/1.1" 200 None 2026-03-26 16:52:07,813 - lava_callback - DEBUG - Processing node 69c56320abf66a8f4d9bbf09: ltp-syscalls 2026-03-26 16:52:08,276 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 16:52:08,309 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-containers-69c5630eabf66a8f4d9bbee1/lava-logs.yaml 2026-03-26 16:52:08,396 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 16:52:09,040 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 16:52:09,080 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 16:52:09,082 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-containers-69c5630eabf66a8f4d9bbee1/lava_callback.json.gz 2026-03-26 16:52:09,120 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:52:09,144 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5630eabf66a8f4d9bbee1 HTTP/1.1" 200 None 2026-03-26 16:52:09,146 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:52:09,162 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c55c70abf66a8f4d9b9655 HTTP/1.1" 200 None 2026-03-26 16:52:09,165 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:52:09,205 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5630eabf66a8f4d9bbee1 HTTP/1.1" 200 None 2026-03-26 16:52:09,207 - lava_callback - INFO - Completed processing callback for node 69c5630eabf66a8f4d9bbee1 2026-03-26 16:52:09,743 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 16:52:09,745 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-syscalls-69c56320abf66a8f4d9bbf09/log.txt.gz 2026-03-26 16:52:10,444 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 16:52:10,985 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 16:52:10,988 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-syscalls-69c56320abf66a8f4d9bbf09/lava-logs.yaml 2026-03-26 16:52:11,009 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 16:52:11,599 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 16:52:11,601 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-syscalls-69c56320abf66a8f4d9bbf09/lava_callback.json.gz 2026-03-26 16:52:11,646 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:52:11,675 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c56320abf66a8f4d9bbf09 HTTP/1.1" 200 None 2026-03-26 16:52:11,677 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:52:11,694 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c55c70abf66a8f4d9b9655 HTTP/1.1" 200 None 2026-03-26 16:52:11,695 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:52:11,727 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c56320abf66a8f4d9bbf09 HTTP/1.1" 200 None 2026-03-26 16:52:11,741 - lava_callback - INFO - Completed processing callback for node 69c56320abf66a8f4d9bbf09 INFO: 172.21.0.1:37736 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37752 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51592 - "GET / HTTP/1.0" 200 OK 2026-03-26 16:52:32,056 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:52:32,083 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:47696 - "GET / HTTP/1.0" 200 OK 2026-03-26 16:52:40,037 - lava_callback - INFO - Received callback for node 69c2c1466778148209fd5d50 INFO: 172.21.0.1:38170 - "POST /node/69c2c1466778148209fd5d50 HTTP/1.0" 202 Accepted 2026-03-26 16:52:40,043 - lava_callback - INFO - Starting async job submit for node 69c2c1466778148209fd5d50 2026-03-26 16:52:40,044 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:52:40,058 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2c1466778148209fd5d50 HTTP/1.1" 200 None 2026-03-26 16:52:40,059 - lava_callback - DEBUG - Processing node 69c2c1466778148209fd5d50: baseline-arm-clabbe 2026-03-26 16:52:40,059 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2c1466778148209fd5d50 2026-03-26 16:52:40,060 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:52:40,077 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2c1466778148209fd5d50 HTTP/1.1" 200 None 2026-03-26 16:52:40,079 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:52:40,094 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad09221a77312e2220ee1 HTTP/1.1" 200 None 2026-03-26 16:52:40,095 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:52:40,123 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2c1466778148209fd5d50 HTTP/1.1" 200 None 2026-03-26 16:52:40,128 - lava_callback - ERROR - Error processing callback for node 69c2c1466778148209fd5d50: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:38176 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50932 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50858 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50870 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52648 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52656 - "GET / HTTP/1.0" 200 OK 2026-03-26 16:53:21,926 - lava_callback - INFO - Received callback for node 69c2c16e6778148209fd5d59 INFO: 172.21.0.1:59160 - "POST /node/69c2c16e6778148209fd5d59 HTTP/1.0" 202 Accepted 2026-03-26 16:53:21,932 - lava_callback - INFO - Starting async job submit for node 69c2c16e6778148209fd5d59 2026-03-26 16:53:21,933 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:53:21,947 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2c16e6778148209fd5d59 HTTP/1.1" 200 None 2026-03-26 16:53:21,948 - lava_callback - DEBUG - Processing node 69c2c16e6778148209fd5d59: baseline-arm-baylibre 2026-03-26 16:53:21,948 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2c16e6778148209fd5d59 2026-03-26 16:53:21,949 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:53:21,962 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2c16e6778148209fd5d59 HTTP/1.1" 200 None 2026-03-26 16:53:21,963 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:53:22,019 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01633c3d73288b0d052f0 HTTP/1.1" 200 None 2026-03-26 16:53:22,021 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:53:22,041 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2c16e6778148209fd5d59 HTTP/1.1" 200 None 2026-03-26 16:53:22,046 - lava_callback - ERROR - Error processing callback for node 69c2c16e6778148209fd5d59: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:59166 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37286 - "GET / HTTP/1.0" 200 OK 2026-03-26 16:53:38,466 - lava_callback - INFO - Received callback for node 69c562f4abf66a8f4d9bbe84 INFO: 172.21.0.1:37298 - "POST /node/69c562f4abf66a8f4d9bbe84 HTTP/1.0" 202 Accepted 2026-03-26 16:53:38,473 - lava_callback - INFO - Starting async job submit for node 69c562f4abf66a8f4d9bbe84 2026-03-26 16:53:38,499 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:53:38,514 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c562f4abf66a8f4d9bbe84 HTTP/1.1" 200 None 2026-03-26 16:53:38,515 - lava_callback - DEBUG - Processing node 69c562f4abf66a8f4d9bbe84: baseline-arm-clabbe 2026-03-26 16:53:38,719 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 16:53:38,877 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 16:53:38,879 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69c562f4abf66a8f4d9bbe84/log.txt.gz 2026-03-26 16:53:39,010 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 16:53:39,203 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 16:53:39,205 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69c562f4abf66a8f4d9bbe84/lava-logs.yaml 2026-03-26 16:53:39,211 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 16:53:39,357 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 16:53:39,359 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69c562f4abf66a8f4d9bbe84/lava_callback.json.gz 2026-03-26 16:53:39,388 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:53:39,402 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c562f4abf66a8f4d9bbe84 HTTP/1.1" 200 None 2026-03-26 16:53:39,403 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:53:39,417 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c55c70abf66a8f4d9b9655 HTTP/1.1" 200 None 2026-03-26 16:53:39,419 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:53:39,443 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c562f4abf66a8f4d9bbe84 HTTP/1.1" 200 None 2026-03-26 16:53:39,445 - lava_callback - INFO - Completed processing callback for node 69c562f4abf66a8f4d9bbe84 2026-03-26 16:53:46,498 - lava_callback - INFO - Received callback for node 69c5630fabf66a8f4d9bbee4 INFO: 172.21.0.1:35672 - "POST /node/69c5630fabf66a8f4d9bbee4 HTTP/1.0" 202 Accepted 2026-03-26 16:53:46,505 - lava_callback - INFO - Starting async job submit for node 69c5630fabf66a8f4d9bbee4 INFO: 172.21.0.1:35682 - "GET / HTTP/1.0" 200 OK 2026-03-26 16:53:46,531 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:53:46,545 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5630fabf66a8f4d9bbee4 HTTP/1.1" 200 None 2026-03-26 16:53:46,545 - lava_callback - DEBUG - Processing node 69c5630fabf66a8f4d9bbee4: ltp-controllers 2026-03-26 16:53:47,022 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 16:53:47,160 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 16:53:47,161 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-controllers-69c5630fabf66a8f4d9bbee4/log.txt.gz 2026-03-26 16:53:47,431 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 16:53:47,803 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 16:53:47,806 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-controllers-69c5630fabf66a8f4d9bbee4/lava-logs.yaml 2026-03-26 16:53:47,817 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 16:53:47,980 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 16:53:47,981 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-controllers-69c5630fabf66a8f4d9bbee4/lava_callback.json.gz 2026-03-26 16:53:48,009 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:53:48,023 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5630fabf66a8f4d9bbee4 HTTP/1.1" 200 None 2026-03-26 16:53:48,025 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:53:48,038 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c55c70abf66a8f4d9b9655 HTTP/1.1" 200 None 2026-03-26 16:53:48,040 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:53:48,064 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5630fabf66a8f4d9bbee4 HTTP/1.1" 200 None 2026-03-26 16:53:48,066 - lava_callback - INFO - Completed processing callback for node 69c5630fabf66a8f4d9bbee4 INFO: 172.21.0.1:57134 - "GET / HTTP/1.0" 200 OK 2026-03-26 16:54:01,833 - lava_callback - INFO - Received callback for node 69c2c1976778148209fd5d65 INFO: 172.21.0.1:39744 - "POST /node/69c2c1976778148209fd5d65 HTTP/1.0" 202 Accepted 2026-03-26 16:54:01,839 - lava_callback - INFO - Starting async job submit for node 69c2c1976778148209fd5d65 2026-03-26 16:54:01,839 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:54:01,854 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2c1976778148209fd5d65 HTTP/1.1" 200 None 2026-03-26 16:54:01,855 - lava_callback - DEBUG - Processing node 69c2c1976778148209fd5d65: baseline-arm-baylibre 2026-03-26 16:54:01,855 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2c1976778148209fd5d65 2026-03-26 16:54:01,856 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:54:01,879 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2c1976778148209fd5d65 HTTP/1.1" 200 None 2026-03-26 16:54:01,880 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:54:01,894 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01633c3d73288b0d052f0 HTTP/1.1" 200 None 2026-03-26 16:54:01,895 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:54:01,915 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2c1976778148209fd5d65 HTTP/1.1" 200 None 2026-03-26 16:54:01,921 - lava_callback - ERROR - Error processing callback for node 69c2c1976778148209fd5d65: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-26 16:54:02,087 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:54:02,104 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:39746 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39754 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58912 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58928 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53900 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50478 - "GET / HTTP/1.0" 200 OK 2026-03-26 16:54:42,261 - lava_callback - INFO - Received callback for node 69c2c1c06778148209fd5d6f INFO: 172.21.0.1:54310 - "POST /node/69c2c1c06778148209fd5d6f HTTP/1.0" 202 Accepted 2026-03-26 16:54:42,267 - lava_callback - INFO - Starting async job submit for node 69c2c1c06778148209fd5d6f 2026-03-26 16:54:42,268 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:54:42,286 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2c1c06778148209fd5d6f HTTP/1.1" 200 None 2026-03-26 16:54:42,287 - lava_callback - DEBUG - Processing node 69c2c1c06778148209fd5d6f: baseline-arm-baylibre 2026-03-26 16:54:42,287 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2c1c06778148209fd5d6f 2026-03-26 16:54:42,288 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:54:42,303 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2c1c06778148209fd5d6f HTTP/1.1" 200 None 2026-03-26 16:54:42,305 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:54:42,318 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01633c3d73288b0d052f0 HTTP/1.1" 200 None 2026-03-26 16:54:42,319 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:54:42,340 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2c1c06778148209fd5d6f HTTP/1.1" 200 None 2026-03-26 16:54:42,345 - lava_callback - ERROR - Error processing callback for node 69c2c1c06778148209fd5d6f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:54318 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35838 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51376 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51378 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46184 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46188 - "GET / HTTP/1.0" 200 OK 2026-03-26 16:55:21,921 - lava_callback - INFO - Received callback for node 69c2c1ea6778148209fd5d7b INFO: 172.21.0.1:60336 - "POST /node/69c2c1ea6778148209fd5d7b HTTP/1.0" 202 Accepted 2026-03-26 16:55:21,927 - lava_callback - INFO - Starting async job submit for node 69c2c1ea6778148209fd5d7b 2026-03-26 16:55:21,928 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:55:21,942 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2c1ea6778148209fd5d7b HTTP/1.1" 200 None 2026-03-26 16:55:21,943 - lava_callback - DEBUG - Processing node 69c2c1ea6778148209fd5d7b: baseline-arm-baylibre 2026-03-26 16:55:21,943 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2c1ea6778148209fd5d7b 2026-03-26 16:55:21,944 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:55:21,956 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2c1ea6778148209fd5d7b HTTP/1.1" 200 None 2026-03-26 16:55:21,958 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:55:21,974 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01633c3d73288b0d052f0 HTTP/1.1" 200 None 2026-03-26 16:55:21,976 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:55:22,003 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2c1ea6778148209fd5d7b HTTP/1.1" 200 None 2026-03-26 16:55:22,010 - lava_callback - ERROR - Error processing callback for node 69c2c1ea6778148209fd5d7b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:60352 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57032 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38114 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38420 - "GET / HTTP/1.0" 200 OK 2026-03-26 16:56:02,310 - lava_callback - INFO - Received callback for node 69c2c2116778148209fd5d85 INFO: 172.21.0.1:50324 - "POST /node/69c2c2116778148209fd5d85 HTTP/1.0" 202 Accepted 2026-03-26 16:56:02,317 - lava_callback - INFO - Starting async job submit for node 69c2c2116778148209fd5d85 2026-03-26 16:56:02,318 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:56:02,333 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2c2116778148209fd5d85 HTTP/1.1" 200 None 2026-03-26 16:56:02,334 - lava_callback - DEBUG - Processing node 69c2c2116778148209fd5d85: baseline-arm-baylibre 2026-03-26 16:56:02,334 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2c2116778148209fd5d85 2026-03-26 16:56:02,335 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:56:02,350 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2c2116778148209fd5d85 HTTP/1.1" 200 None 2026-03-26 16:56:02,352 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:56:02,372 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01633c3d73288b0d052f0 HTTP/1.1" 200 None 2026-03-26 16:56:02,374 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:56:02,396 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2c2116778148209fd5d85 HTTP/1.1" 200 None 2026-03-26 16:56:02,407 - lava_callback - ERROR - Error processing callback for node 69c2c2116778148209fd5d85: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:50332 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50340 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59512 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59520 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40622 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33054 - "GET / HTTP/1.0" 200 OK 2026-03-26 16:56:42,029 - lava_callback - INFO - Received callback for node 69c2c23b6778148209fd5d91 INFO: 172.21.0.1:48488 - "POST /node/69c2c23b6778148209fd5d91 HTTP/1.0" 202 Accepted 2026-03-26 16:56:42,035 - lava_callback - INFO - Starting async job submit for node 69c2c23b6778148209fd5d91 2026-03-26 16:56:42,036 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:56:42,063 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2c23b6778148209fd5d91 HTTP/1.1" 200 None 2026-03-26 16:56:42,064 - lava_callback - DEBUG - Processing node 69c2c23b6778148209fd5d91: baseline-arm-baylibre 2026-03-26 16:56:42,064 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2c23b6778148209fd5d91 2026-03-26 16:56:42,065 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:56:42,080 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2c23b6778148209fd5d91 HTTP/1.1" 200 None 2026-03-26 16:56:42,082 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:56:42,095 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01633c3d73288b0d052f0 HTTP/1.1" 200 None 2026-03-26 16:56:42,096 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:56:42,117 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2c23b6778148209fd5d91 HTTP/1.1" 200 None 2026-03-26 16:56:42,118 - lava_callback - ERROR - Error processing callback for node 69c2c23b6778148209fd5d91: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:48504 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56546 - "GET / HTTP/1.0" 200 OK 2026-03-26 16:57:05,437 - lava_callback - INFO - Received callback for node 69c56547abf66a8f4d9bc05a INFO: 172.21.0.1:56452 - "POST /node/69c56547abf66a8f4d9bc05a HTTP/1.0" 202 Accepted 2026-03-26 16:57:05,446 - lava_callback - INFO - Starting async job submit for node 69c56547abf66a8f4d9bc05a 2026-03-26 16:57:05,451 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:57:05,473 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c56547abf66a8f4d9bc05a HTTP/1.1" 200 None 2026-03-26 16:57:05,474 - lava_callback - DEBUG - Processing node 69c56547abf66a8f4d9bc05a: baseline-arm 2026-03-26 16:57:05,482 - lava_callback - WARNING - Failed to upload log 2026-03-26 16:57:05,487 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 16:57:05,654 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 16:57:05,656 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c56547abf66a8f4d9bc05a/lava-logs.yaml 2026-03-26 16:57:05,657 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 16:57:05,828 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 16:57:05,830 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c56547abf66a8f4d9bc05a/lava_callback.json.gz 2026-03-26 16:57:05,833 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:57:05,852 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c56547abf66a8f4d9bc05a HTTP/1.1" 200 None 2026-03-26 16:57:05,854 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:57:05,869 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c55c70abf66a8f4d9b9655 HTTP/1.1" 200 None 2026-03-26 16:57:05,871 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:57:05,895 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c56547abf66a8f4d9bc05a HTTP/1.1" 200 None 2026-03-26 16:57:05,907 - lava_callback - INFO - Completed processing callback for node 69c56547abf66a8f4d9bc05a INFO: 172.21.0.1:56458 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56468 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45578 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45588 - "GET / HTTP/1.0" 200 OK 2026-03-26 16:57:20,337 - lava_callback - INFO - Received callback for node 69c2c2676778148209fd5d9d INFO: 172.21.0.1:41732 - "POST /node/69c2c2676778148209fd5d9d HTTP/1.0" 202 Accepted 2026-03-26 16:57:20,343 - lava_callback - INFO - Starting async job submit for node 69c2c2676778148209fd5d9d 2026-03-26 16:57:20,344 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:57:20,364 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2c2676778148209fd5d9d HTTP/1.1" 200 None 2026-03-26 16:57:20,366 - lava_callback - DEBUG - Processing node 69c2c2676778148209fd5d9d: baseline-arm-clabbe 2026-03-26 16:57:20,366 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2c2676778148209fd5d9d 2026-03-26 16:57:20,367 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:57:20,390 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2c2676778148209fd5d9d HTTP/1.1" 200 None 2026-03-26 16:57:20,393 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:57:20,408 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad09221a77312e2220ee1 HTTP/1.1" 200 None 2026-03-26 16:57:20,411 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:57:20,445 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2c2676778148209fd5d9d HTTP/1.1" 200 None 2026-03-26 16:57:20,447 - lava_callback - ERROR - Error processing callback for node 69c2c2676778148209fd5d9d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:41740 - "GET / HTTP/1.0" 200 OK 2026-03-26 16:57:32,107 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:57:32,136 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:44280 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60328 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60342 - "GET / HTTP/1.0" 200 OK 2026-03-26 16:58:00,310 - lava_callback - INFO - Received callback for node 69c2c2926778148209fd5daa INFO: 172.21.0.1:51736 - "POST /node/69c2c2926778148209fd5daa HTTP/1.0" 202 Accepted 2026-03-26 16:58:00,319 - lava_callback - INFO - Starting async job submit for node 69c2c2926778148209fd5daa 2026-03-26 16:58:00,325 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:58:00,350 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2c2926778148209fd5daa HTTP/1.1" 200 None 2026-03-26 16:58:00,352 - lava_callback - DEBUG - Processing node 69c2c2926778148209fd5daa: baseline-arm-clabbe 2026-03-26 16:58:00,353 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2c2926778148209fd5daa 2026-03-26 16:58:00,354 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:58:00,381 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2c2926778148209fd5daa HTTP/1.1" 200 None 2026-03-26 16:58:00,384 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:58:00,417 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01633c3d73288b0d052f0 HTTP/1.1" 200 None 2026-03-26 16:58:00,420 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:58:00,464 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2c2926778148209fd5daa HTTP/1.1" 200 None 2026-03-26 16:58:00,466 - lava_callback - ERROR - Error processing callback for node 69c2c2926778148209fd5daa: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:51752 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51756 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60986 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60998 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45968 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47336 - "GET / HTTP/1.0" 200 OK 2026-03-26 16:58:40,539 - lava_callback - INFO - Received callback for node 69c2c2bd6778148209fd5dfb INFO: 172.21.0.1:52472 - "POST /node/69c2c2bd6778148209fd5dfb HTTP/1.0" 202 Accepted 2026-03-26 16:58:40,548 - lava_callback - INFO - Starting async job submit for node 69c2c2bd6778148209fd5dfb 2026-03-26 16:58:40,549 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:58:40,574 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2c2bd6778148209fd5dfb HTTP/1.1" 200 None 2026-03-26 16:58:40,575 - lava_callback - DEBUG - Processing node 69c2c2bd6778148209fd5dfb: baseline-arm-clabbe 2026-03-26 16:58:40,575 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2c2bd6778148209fd5dfb 2026-03-26 16:58:40,576 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:58:40,607 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2c2bd6778148209fd5dfb HTTP/1.1" 200 None 2026-03-26 16:58:40,609 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:58:40,633 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01633c3d73288b0d052f0 HTTP/1.1" 200 None 2026-03-26 16:58:40,635 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:58:40,670 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2c2bd6778148209fd5dfb HTTP/1.1" 200 None 2026-03-26 16:58:40,672 - lava_callback - ERROR - Error processing callback for node 69c2c2bd6778148209fd5dfb: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:52486 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33056 - "GET / HTTP/1.0" 200 OK 2026-03-26 16:59:02,197 - lava_callback - INFO - Received callback for node 69c2c2ce6778148209fd5e48 INFO: 172.21.0.1:51742 - "POST /node/69c2c2ce6778148209fd5e48 HTTP/1.0" 202 Accepted 2026-03-26 16:59:02,206 - lava_callback - INFO - Starting async job submit for node 69c2c2ce6778148209fd5e48 2026-03-26 16:59:02,209 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:59:02,235 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2c2ce6778148209fd5e48 HTTP/1.1" 200 None 2026-03-26 16:59:02,236 - lava_callback - DEBUG - Processing node 69c2c2ce6778148209fd5e48: baseline-arm64-baylibre 2026-03-26 16:59:02,236 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2c2ce6778148209fd5e48 2026-03-26 16:59:02,237 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:59:02,258 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2c2ce6778148209fd5e48 HTTP/1.1" 200 None 2026-03-26 16:59:02,260 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:59:02,280 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b9626778148209fd337b HTTP/1.1" 200 None 2026-03-26 16:59:02,282 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:59:02,315 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2c2ce6778148209fd5e48 HTTP/1.1" 200 None 2026-03-26 16:59:02,316 - lava_callback - ERROR - Error processing callback for node 69c2c2ce6778148209fd5e48: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-26 16:59:02,870 - lava_callback - INFO - Received callback for node 69c2c2d06778148209fd5e4b INFO: 172.21.0.1:51746 - "POST /node/69c2c2d06778148209fd5e4b HTTP/1.0" 202 Accepted 2026-03-26 16:59:02,880 - lava_callback - INFO - Starting async job submit for node 69c2c2d06778148209fd5e4b 2026-03-26 16:59:02,882 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:59:02,911 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2c2d06778148209fd5e4b HTTP/1.1" 200 None 2026-03-26 16:59:02,913 - lava_callback - DEBUG - Processing node 69c2c2d06778148209fd5e4b: baseline-arm64-baylibre 2026-03-26 16:59:02,913 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2c2d06778148209fd5e4b 2026-03-26 16:59:02,914 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:59:02,955 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2c2d06778148209fd5e4b HTTP/1.1" 200 None 2026-03-26 16:59:02,957 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:59:02,995 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b9626778148209fd337b HTTP/1.1" 200 None 2026-03-26 16:59:02,998 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:59:03,037 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2c2d06778148209fd5e4b HTTP/1.1" 200 None 2026-03-26 16:59:03,039 - lava_callback - ERROR - Error processing callback for node 69c2c2d06778148209fd5e4b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-26 16:59:03,568 - lava_callback - INFO - Received callback for node 69c2c2d16778148209fd5e57 INFO: 172.21.0.1:51762 - "POST /node/69c2c2d16778148209fd5e57 HTTP/1.0" 202 Accepted 2026-03-26 16:59:03,578 - lava_callback - INFO - Starting async job submit for node 69c2c2d16778148209fd5e57 2026-03-26 16:59:03,580 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:59:03,603 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2c2d16778148209fd5e57 HTTP/1.1" 200 None 2026-03-26 16:59:03,604 - lava_callback - DEBUG - Processing node 69c2c2d16778148209fd5e57: baseline-arm64-baylibre 2026-03-26 16:59:03,604 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2c2d16778148209fd5e57 2026-03-26 16:59:03,605 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:59:03,631 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2c2d16778148209fd5e57 HTTP/1.1" 200 None 2026-03-26 16:59:03,633 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:59:03,655 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b9626778148209fd337b HTTP/1.1" 200 None 2026-03-26 16:59:03,657 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:59:03,699 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2c2d16778148209fd5e57 HTTP/1.1" 200 None 2026-03-26 16:59:03,701 - lava_callback - ERROR - Error processing callback for node 69c2c2d16778148209fd5e57: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-26 16:59:04,240 - lava_callback - INFO - Received callback for node 69c2c2d46778148209fd5e5e INFO: 172.21.0.1:51776 - "POST /node/69c2c2d46778148209fd5e5e HTTP/1.0" 202 Accepted 2026-03-26 16:59:04,249 - lava_callback - INFO - Starting async job submit for node 69c2c2d46778148209fd5e5e 2026-03-26 16:59:04,256 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:59:04,284 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2c2d46778148209fd5e5e HTTP/1.1" 200 None 2026-03-26 16:59:04,285 - lava_callback - DEBUG - Processing node 69c2c2d46778148209fd5e5e: baseline-arm64-baylibre 2026-03-26 16:59:04,285 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2c2d46778148209fd5e5e 2026-03-26 16:59:04,286 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:59:04,334 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2c2d46778148209fd5e5e HTTP/1.1" 200 None 2026-03-26 16:59:04,336 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:59:04,369 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b9626778148209fd337b HTTP/1.1" 200 None 2026-03-26 16:59:04,371 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:59:04,404 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2c2d46778148209fd5e5e HTTP/1.1" 200 None 2026-03-26 16:59:04,406 - lava_callback - ERROR - Error processing callback for node 69c2c2d46778148209fd5e5e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:51782 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51786 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60086 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60090 - "GET / HTTP/1.0" 200 OK 2026-03-26 16:59:22,158 - lava_callback - INFO - Received callback for node 69c2c2d66778148209fd5e61 INFO: 172.21.0.1:60044 - "POST /node/69c2c2d66778148209fd5e61 HTTP/1.0" 202 Accepted 2026-03-26 16:59:22,168 - lava_callback - INFO - Starting async job submit for node 69c2c2d66778148209fd5e61 2026-03-26 16:59:22,169 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:59:22,195 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2c2d66778148209fd5e61 HTTP/1.1" 200 None 2026-03-26 16:59:22,197 - lava_callback - DEBUG - Processing node 69c2c2d66778148209fd5e61: baseline-arm64-baylibre 2026-03-26 16:59:22,197 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2c2d66778148209fd5e61 2026-03-26 16:59:22,198 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:59:22,225 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2c2d66778148209fd5e61 HTTP/1.1" 200 None 2026-03-26 16:59:22,227 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:59:22,263 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b9626778148209fd337b HTTP/1.1" 200 None 2026-03-26 16:59:22,266 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:59:22,321 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2c2d66778148209fd5e61 HTTP/1.1" 200 None 2026-03-26 16:59:22,322 - lava_callback - ERROR - Error processing callback for node 69c2c2d66778148209fd5e61: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-26 16:59:22,859 - lava_callback - INFO - Received callback for node 69c2c2d76778148209fd5e64 INFO: 172.21.0.1:60048 - "POST /node/69c2c2d76778148209fd5e64 HTTP/1.0" 202 Accepted 2026-03-26 16:59:22,867 - lava_callback - INFO - Starting async job submit for node 69c2c2d76778148209fd5e64 2026-03-26 16:59:22,872 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:59:22,904 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2c2d76778148209fd5e64 HTTP/1.1" 200 None 2026-03-26 16:59:22,905 - lava_callback - DEBUG - Processing node 69c2c2d76778148209fd5e64: baseline-arm64-baylibre 2026-03-26 16:59:22,905 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2c2d76778148209fd5e64 2026-03-26 16:59:22,906 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:59:22,931 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2c2d76778148209fd5e64 HTTP/1.1" 200 None 2026-03-26 16:59:22,933 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:59:22,957 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b9626778148209fd337b HTTP/1.1" 200 None 2026-03-26 16:59:22,959 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:59:22,992 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2c2d76778148209fd5e64 HTTP/1.1" 200 None 2026-03-26 16:59:22,993 - lava_callback - ERROR - Error processing callback for node 69c2c2d76778148209fd5e64: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-26 16:59:24,814 - lava_callback - INFO - Received callback for node 69c46ad8944faaf0e7f62007 INFO: 172.21.0.1:60062 - "POST /node/69c46ad8944faaf0e7f62007 HTTP/1.0" 202 Accepted 2026-03-26 16:59:24,826 - lava_callback - INFO - Starting async job submit for node 69c46ad8944faaf0e7f62007 2026-03-26 16:59:24,880 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:59:24,912 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c46ad8944faaf0e7f62007 HTTP/1.1" 200 None 2026-03-26 16:59:24,914 - lava_callback - DEBUG - Processing node 69c46ad8944faaf0e7f62007: baseline-arm64 2026-03-26 16:59:25,859 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.21.0.1:60076 - "GET / HTTP/1.0" 200 OK 2026-03-26 16:59:26,716 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 16:59:26,719 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c46ad8944faaf0e7f62007/log.txt.gz 2026-03-26 16:59:27,162 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 16:59:28,794 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 16:59:28,796 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c46ad8944faaf0e7f62007/lava-logs.yaml 2026-03-26 16:59:28,813 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 16:59:30,047 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 16:59:30,050 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c46ad8944faaf0e7f62007/lava_callback.json.gz 2026-03-26 16:59:30,096 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:59:30,133 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c46ad8944faaf0e7f62007 HTTP/1.1" 200 None 2026-03-26 16:59:30,136 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:59:30,160 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ad9944faaf0e7f5e41d HTTP/1.1" 200 None 2026-03-26 16:59:30,162 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:59:30,209 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c46ad8944faaf0e7f62007 HTTP/1.1" 200 None 2026-03-26 16:59:30,281 - lava_callback - INFO - Completed processing callback for node 69c46ad8944faaf0e7f62007 2026-03-26 16:59:32,139 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:59:32,170 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:42290 - "GET / HTTP/1.0" 200 OK 2026-03-26 16:59:40,359 - lava_callback - INFO - Received callback for node 69c2c2e96778148209fd5ea8 INFO: 172.21.0.1:37678 - "POST /node/69c2c2e96778148209fd5ea8 HTTP/1.0" 202 Accepted 2026-03-26 16:59:40,369 - lava_callback - INFO - Starting async job submit for node 69c2c2e96778148209fd5ea8 2026-03-26 16:59:40,370 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:59:40,399 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2c2e96778148209fd5ea8 HTTP/1.1" 200 None 2026-03-26 16:59:40,401 - lava_callback - DEBUG - Processing node 69c2c2e96778148209fd5ea8: baseline-arm64-clabbe 2026-03-26 16:59:40,401 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2c2e96778148209fd5ea8 2026-03-26 16:59:40,403 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:59:40,427 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2c2e96778148209fd5ea8 HTTP/1.1" 200 None 2026-03-26 16:59:40,430 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:59:40,455 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b9626778148209fd337b HTTP/1.1" 200 None 2026-03-26 16:59:40,458 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:59:40,505 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2c2e96778148209fd5ea8 HTTP/1.1" 200 None 2026-03-26 16:59:40,506 - lava_callback - ERROR - Error processing callback for node 69c2c2e96778148209fd5ea8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-26 16:59:41,020 - lava_callback - INFO - Received callback for node 69c2c2e96778148209fd5eaa INFO: 172.21.0.1:37692 - "POST /node/69c2c2e96778148209fd5eaa HTTP/1.0" 202 Accepted 2026-03-26 16:59:41,029 - lava_callback - INFO - Starting async job submit for node 69c2c2e96778148209fd5eaa 2026-03-26 16:59:41,035 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:59:41,064 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2c2e96778148209fd5eaa HTTP/1.1" 200 None 2026-03-26 16:59:41,066 - lava_callback - DEBUG - Processing node 69c2c2e96778148209fd5eaa: baseline-arm-clabbe 2026-03-26 16:59:41,066 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2c2e96778148209fd5eaa 2026-03-26 16:59:41,067 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:59:41,092 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2c2e96778148209fd5eaa HTTP/1.1" 200 None 2026-03-26 16:59:41,095 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:59:41,117 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01633c3d73288b0d052f0 HTTP/1.1" 200 None 2026-03-26 16:59:41,120 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:59:41,158 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2c2e96778148209fd5eaa HTTP/1.1" 200 None 2026-03-26 16:59:41,159 - lava_callback - ERROR - Error processing callback for node 69c2c2e96778148209fd5eaa: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-26 16:59:41,726 - lava_callback - INFO - Received callback for node 69c2c2eb6778148209fd5ead INFO: 172.21.0.1:37704 - "POST /node/69c2c2eb6778148209fd5ead HTTP/1.0" 202 Accepted 2026-03-26 16:59:41,735 - lava_callback - INFO - Starting async job submit for node 69c2c2eb6778148209fd5ead 2026-03-26 16:59:41,737 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:59:41,763 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2c2eb6778148209fd5ead HTTP/1.1" 200 None 2026-03-26 16:59:41,764 - lava_callback - DEBUG - Processing node 69c2c2eb6778148209fd5ead: baseline-arm64-clabbe 2026-03-26 16:59:41,765 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2c2eb6778148209fd5ead 2026-03-26 16:59:41,765 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:59:41,794 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2c2eb6778148209fd5ead HTTP/1.1" 200 None 2026-03-26 16:59:41,796 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:59:41,823 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b9626778148209fd337b HTTP/1.1" 200 None 2026-03-26 16:59:41,825 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:59:41,867 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2c2eb6778148209fd5ead HTTP/1.1" 200 None 2026-03-26 16:59:41,869 - lava_callback - ERROR - Error processing callback for node 69c2c2eb6778148209fd5ead: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-26 16:59:42,415 - lava_callback - INFO - Received callback for node 69c2c2ed6778148209fd5eb1 INFO: 172.21.0.1:37710 - "POST /node/69c2c2ed6778148209fd5eb1 HTTP/1.0" 202 Accepted 2026-03-26 16:59:42,424 - lava_callback - INFO - Starting async job submit for node 69c2c2ed6778148209fd5eb1 2026-03-26 16:59:42,426 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:59:42,452 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2c2ed6778148209fd5eb1 HTTP/1.1" 200 None 2026-03-26 16:59:42,453 - lava_callback - DEBUG - Processing node 69c2c2ed6778148209fd5eb1: baseline-arm64-clabbe 2026-03-26 16:59:42,454 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2c2ed6778148209fd5eb1 2026-03-26 16:59:42,455 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:59:42,485 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2c2ed6778148209fd5eb1 HTTP/1.1" 200 None 2026-03-26 16:59:42,487 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:59:42,517 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b9626778148209fd337b HTTP/1.1" 200 None 2026-03-26 16:59:42,520 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:59:42,555 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2c2ed6778148209fd5eb1 HTTP/1.1" 200 None 2026-03-26 16:59:42,556 - lava_callback - ERROR - Error processing callback for node 69c2c2ed6778148209fd5eb1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-26 16:59:43,081 - lava_callback - INFO - Received callback for node 69c2c2ee6778148209fd5eb4 INFO: 172.21.0.1:37718 - "POST /node/69c2c2ee6778148209fd5eb4 HTTP/1.0" 202 Accepted 2026-03-26 16:59:43,091 - lava_callback - INFO - Starting async job submit for node 69c2c2ee6778148209fd5eb4 2026-03-26 16:59:43,094 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:59:43,123 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2c2ee6778148209fd5eb4 HTTP/1.1" 200 None 2026-03-26 16:59:43,125 - lava_callback - DEBUG - Processing node 69c2c2ee6778148209fd5eb4: baseline-arm64-clabbe 2026-03-26 16:59:43,125 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2c2ee6778148209fd5eb4 2026-03-26 16:59:43,126 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:59:43,147 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2c2ee6778148209fd5eb4 HTTP/1.1" 200 None 2026-03-26 16:59:43,149 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:59:43,173 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b9626778148209fd337b HTTP/1.1" 200 None 2026-03-26 16:59:43,175 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:59:43,211 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2c2ee6778148209fd5eb4 HTTP/1.1" 200 None 2026-03-26 16:59:43,212 - lava_callback - ERROR - Error processing callback for node 69c2c2ee6778148209fd5eb4: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-26 16:59:43,757 - lava_callback - INFO - Received callback for node 69c2c2f06778148209fd5eba INFO: 172.21.0.1:37728 - "POST /node/69c2c2f06778148209fd5eba HTTP/1.0" 202 Accepted 2026-03-26 16:59:43,766 - lava_callback - INFO - Starting async job submit for node 69c2c2f06778148209fd5eba 2026-03-26 16:59:43,768 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:59:43,811 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2c2f06778148209fd5eba HTTP/1.1" 200 None 2026-03-26 16:59:43,813 - lava_callback - DEBUG - Processing node 69c2c2f06778148209fd5eba: baseline-arm64-clabbe 2026-03-26 16:59:43,813 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2c2f06778148209fd5eba 2026-03-26 16:59:43,814 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:59:43,843 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2c2f06778148209fd5eba HTTP/1.1" 200 None 2026-03-26 16:59:43,846 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:59:43,867 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b9626778148209fd337b HTTP/1.1" 200 None 2026-03-26 16:59:43,869 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:59:43,923 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2c2f06778148209fd5eba HTTP/1.1" 200 None 2026-03-26 16:59:43,925 - lava_callback - ERROR - Error processing callback for node 69c2c2f06778148209fd5eba: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-26 16:59:44,417 - lava_callback - INFO - Received callback for node 69c2c2f16778148209fd5ebd INFO: 172.21.0.1:37744 - "POST /node/69c2c2f16778148209fd5ebd HTTP/1.0" 202 Accepted 2026-03-26 16:59:44,426 - lava_callback - INFO - Starting async job submit for node 69c2c2f16778148209fd5ebd 2026-03-26 16:59:44,428 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:59:44,458 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2c2f16778148209fd5ebd HTTP/1.1" 200 None 2026-03-26 16:59:44,459 - lava_callback - DEBUG - Processing node 69c2c2f16778148209fd5ebd: baseline-arm64-clabbe 2026-03-26 16:59:44,460 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2c2f16778148209fd5ebd 2026-03-26 16:59:44,460 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:59:44,483 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2c2f16778148209fd5ebd HTTP/1.1" 200 None 2026-03-26 16:59:44,485 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:59:44,510 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b9626778148209fd337b HTTP/1.1" 200 None 2026-03-26 16:59:44,512 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 16:59:44,545 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2c2f16778148209fd5ebd HTTP/1.1" 200 None 2026-03-26 16:59:44,547 - lava_callback - ERROR - Error processing callback for node 69c2c2f16778148209fd5ebd: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:37754 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52338 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58602 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58608 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60468 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60476 - "GET / HTTP/1.0" 200 OK 2026-03-26 17:00:20,574 - lava_callback - INFO - Received callback for node 69c2c3166778148209fd5f08 INFO: 172.21.0.1:55360 - "POST /node/69c2c3166778148209fd5f08 HTTP/1.0" 202 Accepted 2026-03-26 17:00:20,581 - lava_callback - INFO - Starting async job submit for node 69c2c3166778148209fd5f08 2026-03-26 17:00:20,582 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 17:00:20,597 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2c3166778148209fd5f08 HTTP/1.1" 200 None 2026-03-26 17:00:20,599 - lava_callback - DEBUG - Processing node 69c2c3166778148209fd5f08: baseline-arm-clabbe 2026-03-26 17:00:20,599 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2c3166778148209fd5f08 2026-03-26 17:00:20,600 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 17:00:20,627 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2c3166778148209fd5f08 HTTP/1.1" 200 None 2026-03-26 17:00:20,630 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 17:00:20,648 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01633c3d73288b0d052f0 HTTP/1.1" 200 None 2026-03-26 17:00:20,650 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 17:00:20,696 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2c3166778148209fd5f08 HTTP/1.1" 200 None 2026-03-26 17:00:20,698 - lava_callback - ERROR - Error processing callback for node 69c2c3166778148209fd5f08: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:55376 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46328 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57648 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39722 - "GET / HTTP/1.0" 200 OK 2026-03-26 17:01:00,455 - lava_callback - INFO - Received callback for node 69c2c3496778148209fd5f65 INFO: 172.21.0.1:39428 - "POST /node/69c2c3496778148209fd5f65 HTTP/1.0" 202 Accepted 2026-03-26 17:01:00,462 - lava_callback - INFO - Starting async job submit for node 69c2c3496778148209fd5f65 2026-03-26 17:01:00,463 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 17:01:00,489 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2c3496778148209fd5f65 HTTP/1.1" 200 None 2026-03-26 17:01:00,491 - lava_callback - DEBUG - Processing node 69c2c3496778148209fd5f65: baseline-arm-clabbe 2026-03-26 17:01:00,491 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2c3496778148209fd5f65 2026-03-26 17:01:00,492 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 17:01:00,520 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2c3496778148209fd5f65 HTTP/1.1" 200 None 2026-03-26 17:01:00,523 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 17:01:00,545 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01633c3d73288b0d052f0 HTTP/1.1" 200 None 2026-03-26 17:01:00,547 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 17:01:00,576 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2c3496778148209fd5f65 HTTP/1.1" 200 None 2026-03-26 17:01:00,578 - lava_callback - ERROR - Error processing callback for node 69c2c3496778148209fd5f65: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:39434 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39444 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43532 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43544 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38002 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47600 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41580 - "GET / HTTP/1.0" 200 OK 2026-03-26 17:01:52,045 - lava_callback - INFO - Received callback for node 69c46b32944faaf0e7f62014 INFO: 172.21.0.1:46602 - "POST /node/69c46b32944faaf0e7f62014 HTTP/1.0" 202 Accepted 2026-03-26 17:01:52,056 - lava_callback - INFO - Starting async job submit for node 69c46b32944faaf0e7f62014 2026-03-26 17:01:52,099 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 17:01:52,121 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c46b32944faaf0e7f62014 HTTP/1.1" 200 None 2026-03-26 17:01:52,122 - lava_callback - DEBUG - Processing node 69c46b32944faaf0e7f62014: baseline-nfs-arm64 2026-03-26 17:01:52,929 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 17:01:53,811 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 17:01:53,814 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c46b32944faaf0e7f62014/log.txt.gz 2026-03-26 17:01:54,215 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 17:01:55,121 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 17:01:55,124 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c46b32944faaf0e7f62014/lava-logs.yaml 2026-03-26 17:01:55,137 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 17:01:55,905 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 17:01:55,906 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c46b32944faaf0e7f62014/lava_callback.json.gz 2026-03-26 17:01:55,937 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 17:01:55,960 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c46b32944faaf0e7f62014 HTTP/1.1" 200 None 2026-03-26 17:01:55,962 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 17:01:55,983 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ad9944faaf0e7f5e41d HTTP/1.1" 200 None 2026-03-26 17:01:55,985 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 17:01:56,017 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c46b32944faaf0e7f62014 HTTP/1.1" 200 None 2026-03-26 17:01:56,097 - lava_callback - INFO - Completed processing callback for node 69c46b32944faaf0e7f62014 INFO: 172.21.0.1:46616 - "GET / HTTP/1.0" 200 OK 2026-03-26 17:02:02,172 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 17:02:02,202 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:59206 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59218 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49002 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49006 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48378 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35922 - "GET / HTTP/1.0" 200 OK 2026-03-26 17:02:40,531 - lava_callback - INFO - Received callback for node 69c2c39d6778148209fd5fb2 INFO: 172.21.0.1:50282 - "POST /node/69c2c39d6778148209fd5fb2 HTTP/1.0" 202 Accepted 2026-03-26 17:02:40,537 - lava_callback - INFO - Starting async job submit for node 69c2c39d6778148209fd5fb2 2026-03-26 17:02:40,538 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 17:02:40,552 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2c39d6778148209fd5fb2 HTTP/1.1" 200 None 2026-03-26 17:02:40,553 - lava_callback - DEBUG - Processing node 69c2c39d6778148209fd5fb2: baseline-arm-clabbe 2026-03-26 17:02:40,553 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2c39d6778148209fd5fb2 2026-03-26 17:02:40,554 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 17:02:40,569 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2c39d6778148209fd5fb2 HTTP/1.1" 200 None 2026-03-26 17:02:40,571 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 17:02:40,586 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad09221a77312e2220ee1 HTTP/1.1" 200 None 2026-03-26 17:02:40,587 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 17:02:40,627 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2c39d6778148209fd5fb2 HTTP/1.1" 200 None 2026-03-26 17:02:40,628 - lava_callback - ERROR - Error processing callback for node 69c2c39d6778148209fd5fb2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:50284 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42072 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56308 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56316 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35264 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35270 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35702 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51562 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39576 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50988 - "GET / HTTP/1.0" 200 OK 2026-03-26 17:04:00,577 - lava_callback - INFO - Received callback for node 69c2c3f36778148209fd5fd1 INFO: 172.21.0.1:36098 - "POST /node/69c2c3f36778148209fd5fd1 HTTP/1.0" 202 Accepted 2026-03-26 17:04:00,583 - lava_callback - INFO - Starting async job submit for node 69c2c3f36778148209fd5fd1 2026-03-26 17:04:00,584 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 17:04:00,598 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2c3f36778148209fd5fd1 HTTP/1.1" 200 None 2026-03-26 17:04:00,599 - lava_callback - DEBUG - Processing node 69c2c3f36778148209fd5fd1: baseline-arm-clabbe 2026-03-26 17:04:00,600 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2c3f36778148209fd5fd1 2026-03-26 17:04:00,600 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 17:04:00,613 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2c3f36778148209fd5fd1 HTTP/1.1" 200 None 2026-03-26 17:04:00,615 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 17:04:00,628 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad09221a77312e2220ee1 HTTP/1.1" 200 None 2026-03-26 17:04:00,630 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 17:04:00,665 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2c3f36778148209fd5fd1 HTTP/1.1" 200 None 2026-03-26 17:04:00,666 - lava_callback - ERROR - Error processing callback for node 69c2c3f36778148209fd5fd1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:36114 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36116 - "GET / HTTP/1.0" 200 OK 2026-03-26 17:04:10,113 - lava_callback - INFO - Received callback for node 69c562d7abf66a8f4d9bbe0f INFO: 172.21.0.1:36172 - "POST /node/69c562d7abf66a8f4d9bbe0f HTTP/1.0" 202 Accepted 2026-03-26 17:04:10,120 - lava_callback - INFO - Starting async job submit for node 69c562d7abf66a8f4d9bbe0f 2026-03-26 17:04:10,172 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 17:04:10,188 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c562d7abf66a8f4d9bbe0f HTTP/1.1" 200 None 2026-03-26 17:04:10,188 - lava_callback - DEBUG - Processing node 69c562d7abf66a8f4d9bbe0f: baseline-arm 2026-03-26 17:04:10,797 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 17:04:10,944 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 17:04:10,946 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69c562d7abf66a8f4d9bbe0f/log.txt.gz 2026-03-26 17:04:11,286 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 17:04:11,669 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 17:04:11,671 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c562d7abf66a8f4d9bbe0f/lava-logs.yaml 2026-03-26 17:04:11,685 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 17:04:12,003 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 17:04:12,005 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c562d7abf66a8f4d9bbe0f/lava_callback.json.gz DEBUG: Setup failed for 69c562d7abf66a8f4d9bbe0f. Transitting job node result: incomplete -> fail 2026-03-26 17:04:12,124 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 17:04:12,139 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c562d7abf66a8f4d9bbe0f HTTP/1.1" 200 None 2026-03-26 17:04:12,141 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 17:04:12,155 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c55c70abf66a8f4d9b9655 HTTP/1.1" 200 None 2026-03-26 17:04:12,157 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 17:04:12,182 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c562d7abf66a8f4d9bbe0f HTTP/1.1" 200 None 2026-03-26 17:04:12,289 - lava_callback - INFO - Completed processing callback for node 69c562d7abf66a8f4d9bbe0f INFO: 172.21.0.1:36178 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36194 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38616 - "GET / HTTP/1.0" 200 OK 2026-03-26 17:04:32,204 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 17:04:32,224 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:49016 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41730 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39702 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52796 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52802 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45210 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45216 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54516 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47726 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49116 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60794 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54776 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54788 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48328 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48334 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54404 - "GET / HTTP/1.0" 200 OK 2026-03-26 17:06:31,281 - lava_callback - INFO - Received callback for node 69c562fbabf66a8f4d9bbe9b INFO: 172.21.0.1:48730 - "POST /node/69c562fbabf66a8f4d9bbe9b HTTP/1.0" 202 Accepted 2026-03-26 17:06:31,332 - lava_callback - INFO - Starting async job submit for node 69c562fbabf66a8f4d9bbe9b 2026-03-26 17:06:31,368 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 17:06:31,388 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c562fbabf66a8f4d9bbe9b HTTP/1.1" 200 None 2026-03-26 17:06:31,388 - lava_callback - DEBUG - Processing node 69c562fbabf66a8f4d9bbe9b: baseline-arm-pengutronix 2026-03-26 17:06:31,495 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 17:06:31,696 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 17:06:31,698 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c562fbabf66a8f4d9bbe9b/log.txt.gz 2026-03-26 17:06:31,814 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 17:06:32,011 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 17:06:32,013 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c562fbabf66a8f4d9bbe9b/lava-logs.yaml 2026-03-26 17:06:32,016 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 17:06:32,156 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 17:06:32,157 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c562fbabf66a8f4d9bbe9b/lava_callback.json.gz DEBUG: Setup failed for 69c562fbabf66a8f4d9bbe9b. Transitting job node result: incomplete -> fail 2026-03-26 17:06:32,238 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 17:06:32,273 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c562fbabf66a8f4d9bbe9b HTTP/1.1" 200 None 2026-03-26 17:06:32,274 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 17:06:32,289 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c55c70abf66a8f4d9b9655 HTTP/1.1" 200 None 2026-03-26 17:06:32,291 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 17:06:32,315 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c562fbabf66a8f4d9bbe9b HTTP/1.1" 200 None 2026-03-26 17:06:32,386 - lava_callback - INFO - Completed processing callback for node 69c562fbabf66a8f4d9bbe9b INFO: 172.21.0.1:48738 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56504 - "GET / HTTP/1.0" 200 OK 2026-03-26 17:06:54,760 - lava_callback - INFO - Received callback for node 69c56797abf66a8f4d9bc149 INFO: 172.21.0.1:47004 - "POST /node/69c56797abf66a8f4d9bc149 HTTP/1.0" 202 Accepted 2026-03-26 17:06:54,766 - lava_callback - INFO - Starting async job submit for node 69c56797abf66a8f4d9bc149 2026-03-26 17:06:54,769 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 17:06:54,820 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c56797abf66a8f4d9bc149 HTTP/1.1" 200 None 2026-03-26 17:06:54,821 - lava_callback - DEBUG - Processing node 69c56797abf66a8f4d9bc149: baseline-arm 2026-03-26 17:06:54,826 - lava_callback - WARNING - Failed to upload log 2026-03-26 17:06:54,830 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 17:06:55,112 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 17:06:55,114 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c56797abf66a8f4d9bc149/lava-logs.yaml 2026-03-26 17:06:55,115 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 17:06:55,242 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 17:06:55,244 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c56797abf66a8f4d9bc149/lava_callback.json.gz 2026-03-26 17:06:55,246 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 17:06:55,261 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c56797abf66a8f4d9bc149 HTTP/1.1" 200 None 2026-03-26 17:06:55,263 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 17:06:55,287 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c55c70abf66a8f4d9b9655 HTTP/1.1" 200 None 2026-03-26 17:06:55,288 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 17:06:55,310 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c56797abf66a8f4d9bc149 HTTP/1.1" 200 None 2026-03-26 17:06:55,321 - lava_callback - INFO - Completed processing callback for node 69c56797abf66a8f4d9bc149 INFO: 172.21.0.1:47008 - "GET / HTTP/1.0" 200 OK 2026-03-26 17:07:02,233 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 17:07:02,253 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:44904 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44910 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37916 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37924 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38730 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44650 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47922 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56714 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51796 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51806 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33524 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33540 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52650 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49062 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41936 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41494 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50608 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50612 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40940 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40946 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42750 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37094 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33236 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35962 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40386 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40400 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43826 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43840 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44954 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49576 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59418 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53402 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42246 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42256 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51808 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51814 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36764 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50900 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35062 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43944 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51574 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51582 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53718 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53724 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55246 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44050 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58516 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35400 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41450 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41464 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60596 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60604 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45672 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42432 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58154 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60062 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60230 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60240 - "GET / HTTP/1.0" 200 OK 2026-03-26 17:14:11,285 - lava_callback - INFO - Received callback for node 69c4f4ccf63ce05331e2ae83 INFO: 172.21.0.1:34052 - "POST /node/69c4f4ccf63ce05331e2ae83 HTTP/1.0" 202 Accepted 2026-03-26 17:14:11,291 - lava_callback - INFO - Starting async job submit for node 69c4f4ccf63ce05331e2ae83 2026-03-26 17:14:11,315 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 17:14:11,329 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f4ccf63ce05331e2ae83 HTTP/1.1" 200 None 2026-03-26 17:14:11,330 - lava_callback - DEBUG - Processing node 69c4f4ccf63ce05331e2ae83: baseline-arm64 2026-03-26 17:14:11,821 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 17:14:11,982 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 17:14:11,984 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c4f4ccf63ce05331e2ae83/log.txt.gz 2026-03-26 17:14:12,239 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 17:14:12,463 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 17:14:12,466 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c4f4ccf63ce05331e2ae83/lava-logs.yaml 2026-03-26 17:14:12,477 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 17:14:12,664 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 17:14:12,666 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c4f4ccf63ce05331e2ae83/lava_callback.json.gz 2026-03-26 17:14:12,690 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 17:14:12,718 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f4ccf63ce05331e2ae83 HTTP/1.1" 200 None 2026-03-26 17:14:12,720 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 17:14:12,734 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebd8f63ce05331e2a492 HTTP/1.1" 200 None 2026-03-26 17:14:12,736 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 17:14:12,757 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4f4ccf63ce05331e2ae83 HTTP/1.1" 200 None 2026-03-26 17:14:12,820 - lava_callback - INFO - Completed processing callback for node 69c4f4ccf63ce05331e2ae83 INFO: 172.21.0.1:34060 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34076 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47226 - "GET / HTTP/1.0" 200 OK 2026-03-26 17:14:32,257 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 17:14:32,272 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:38092 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34322 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57040 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51700 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51702 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44252 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44260 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50002 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37492 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38600 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59590 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44752 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44758 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54680 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54692 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36590 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49224 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60832 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40876 - "GET / HTTP/1.0" 200 OK 2026-03-26 17:17:03,255 - lava_callback - INFO - Received callback for node 69c2c70d6778148209fd60df INFO: 172.21.0.1:39274 - "POST /node/69c2c70d6778148209fd60df HTTP/1.0" 202 Accepted 2026-03-26 17:17:03,261 - lava_callback - INFO - Starting async job submit for node 69c2c70d6778148209fd60df 2026-03-26 17:17:03,262 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 17:17:03,276 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2c70d6778148209fd60df HTTP/1.1" 200 None 2026-03-26 17:17:03,277 - lava_callback - DEBUG - Processing node 69c2c70d6778148209fd60df: baseline-arm64-baylibre 2026-03-26 17:17:03,277 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2c70d6778148209fd60df 2026-03-26 17:17:03,278 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 17:17:03,291 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2c70d6778148209fd60df HTTP/1.1" 200 None 2026-03-26 17:17:03,292 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 17:17:03,307 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd73420e93e2314723e2c4 HTTP/1.1" 200 None 2026-03-26 17:17:03,309 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 17:17:03,330 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2c70d6778148209fd60df HTTP/1.1" 200 None 2026-03-26 17:17:03,331 - lava_callback - ERROR - Error processing callback for node 69c2c70d6778148209fd60df: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:39288 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39294 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33888 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33896 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49352 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60872 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40834 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50928 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50760 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50766 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52292 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52300 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42988 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33666 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51986 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49348 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37194 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37198 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58980 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58988 - "GET / HTTP/1.0" 200 OK 2026-03-26 17:19:18,189 - lava_callback - INFO - Received callback for node 69c565ccabf66a8f4d9bc08f INFO: 172.21.0.1:59004 - "POST /node/69c565ccabf66a8f4d9bc08f HTTP/1.0" 202 Accepted 2026-03-26 17:19:18,196 - lava_callback - INFO - Starting async job submit for node 69c565ccabf66a8f4d9bc08f 2026-03-26 17:19:18,219 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 17:19:18,235 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c565ccabf66a8f4d9bc08f HTTP/1.1" 200 None 2026-03-26 17:19:18,236 - lava_callback - DEBUG - Processing node 69c565ccabf66a8f4d9bc08f: baseline-arm 2026-03-26 17:19:18,444 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 17:19:18,594 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 17:19:18,595 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69c565ccabf66a8f4d9bc08f/log.txt.gz 2026-03-26 17:19:18,712 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 17:19:18,904 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 17:19:18,906 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c565ccabf66a8f4d9bc08f/lava-logs.yaml 2026-03-26 17:19:18,911 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 17:19:19,062 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 17:19:19,064 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c565ccabf66a8f4d9bc08f/lava_callback.json.gz 2026-03-26 17:19:19,087 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 17:19:19,101 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c565ccabf66a8f4d9bc08f HTTP/1.1" 200 None 2026-03-26 17:19:19,103 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 17:19:19,121 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c55c76abf66a8f4d9b965b HTTP/1.1" 200 None 2026-03-26 17:19:19,123 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 17:19:19,162 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c565ccabf66a8f4d9bc08f HTTP/1.1" 200 None 2026-03-26 17:19:19,171 - lava_callback - INFO - Completed processing callback for node 69c565ccabf66a8f4d9bc08f INFO: 172.21.0.1:60678 - "GET / HTTP/1.0" 200 OK 2026-03-26 17:19:32,275 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 17:19:32,291 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:59174 - "GET / HTTP/1.0" 200 OK 2026-03-26 17:19:41,344 - lava_callback - INFO - Received callback for node 69c2c7986778148209fd60fc INFO: 172.21.0.1:40756 - "POST /node/69c2c7986778148209fd60fc HTTP/1.0" 202 Accepted 2026-03-26 17:19:41,350 - lava_callback - INFO - Starting async job submit for node 69c2c7986778148209fd60fc 2026-03-26 17:19:41,351 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 17:19:41,365 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2c7986778148209fd60fc HTTP/1.1" 200 None 2026-03-26 17:19:41,366 - lava_callback - DEBUG - Processing node 69c2c7986778148209fd60fc: baseline-arm64-clabbe 2026-03-26 17:19:41,366 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2c7986778148209fd60fc 2026-03-26 17:19:41,367 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 17:19:41,380 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2c7986778148209fd60fc HTTP/1.1" 200 None 2026-03-26 17:19:41,381 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 17:19:41,394 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd73420e93e2314723e2c4 HTTP/1.1" 200 None 2026-03-26 17:19:41,396 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 17:19:41,415 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2c7986778148209fd60fc HTTP/1.1" 200 None 2026-03-26 17:19:41,438 - lava_callback - ERROR - Error processing callback for node 69c2c7986778148209fd60fc: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:40758 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38500 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51172 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51178 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52868 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52874 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38878 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34144 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33238 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48472 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41626 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41636 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55220 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55236 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34014 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48592 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48616 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53712 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34324 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34336 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44746 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44758 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37662 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57268 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52316 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35176 - "GET / HTTP/1.0" 200 OK 2026-03-26 17:23:01,496 - lava_callback - INFO - Received callback for node 69c2c8726778148209fd6122 INFO: 172.21.0.1:48534 - "POST /node/69c2c8726778148209fd6122 HTTP/1.0" 202 Accepted 2026-03-26 17:23:01,502 - lava_callback - INFO - Starting async job submit for node 69c2c8726778148209fd6122 2026-03-26 17:23:01,502 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 17:23:01,516 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2c8726778148209fd6122 HTTP/1.1" 200 None 2026-03-26 17:23:01,517 - lava_callback - DEBUG - Processing node 69c2c8726778148209fd6122: baseline-arm64-clabbe 2026-03-26 17:23:01,517 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2c8726778148209fd6122 2026-03-26 17:23:01,518 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 17:23:01,531 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2c8726778148209fd6122 HTTP/1.1" 200 None 2026-03-26 17:23:01,532 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 17:23:01,546 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd73420e93e2314723e2c4 HTTP/1.1" 200 None 2026-03-26 17:23:01,547 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 17:23:01,577 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2c8726778148209fd6122 HTTP/1.1" 200 None 2026-03-26 17:23:01,578 - lava_callback - ERROR - Error processing callback for node 69c2c8726778148209fd6122: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:48544 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48556 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55680 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55684 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44924 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34250 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35482 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57266 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56876 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56884 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55364 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55366 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57864 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54808 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50562 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38838 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36430 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36442 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48626 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48642 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52680 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38592 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41456 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50278 - "GET / HTTP/1.0" 200 OK 2026-03-26 17:26:01,585 - lava_callback - INFO - Received callback for node 69c2c9196778148209fd6141 INFO: 172.21.0.1:50328 - "POST /node/69c2c9196778148209fd6141 HTTP/1.0" 202 Accepted 2026-03-26 17:26:01,591 - lava_callback - INFO - Starting async job submit for node 69c2c9196778148209fd6141 2026-03-26 17:26:01,592 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 17:26:01,606 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2c9196778148209fd6141 HTTP/1.1" 200 None 2026-03-26 17:26:01,607 - lava_callback - DEBUG - Processing node 69c2c9196778148209fd6141: baseline-arm64-clabbe 2026-03-26 17:26:01,607 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2c9196778148209fd6141 2026-03-26 17:26:01,608 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 17:26:01,630 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2c9196778148209fd6141 HTTP/1.1" 200 None 2026-03-26 17:26:01,631 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 17:26:01,644 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd73420e93e2314723e2c4 HTTP/1.1" 200 None 2026-03-26 17:26:01,646 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 17:26:01,671 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2c9196778148209fd6141 HTTP/1.1" 200 None 2026-03-26 17:26:01,672 - lava_callback - ERROR - Error processing callback for node 69c2c9196778148209fd6141: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:50344 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50358 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45092 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45106 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48244 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51676 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36926 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43852 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37702 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37708 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49686 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49694 - "GET / HTTP/1.0" 200 OK 2026-03-26 17:27:21,697 - lava_callback - INFO - Received callback for node 69c2c96f6778148209fd6153 INFO: 172.21.0.1:47470 - "POST /node/69c2c96f6778148209fd6153 HTTP/1.0" 202 Accepted 2026-03-26 17:27:21,704 - lava_callback - INFO - Starting async job submit for node 69c2c96f6778148209fd6153 2026-03-26 17:27:21,705 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 17:27:21,728 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2c96f6778148209fd6153 HTTP/1.1" 200 None 2026-03-26 17:27:21,729 - lava_callback - DEBUG - Processing node 69c2c96f6778148209fd6153: baseline-arm64-clabbe 2026-03-26 17:27:21,729 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2c96f6778148209fd6153 2026-03-26 17:27:21,730 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 17:27:21,743 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2c96f6778148209fd6153 HTTP/1.1" 200 None 2026-03-26 17:27:21,745 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 17:27:21,759 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd73420e93e2314723e2c4 HTTP/1.1" 200 None 2026-03-26 17:27:21,760 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 17:27:21,781 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2c96f6778148209fd6153 HTTP/1.1" 200 None 2026-03-26 17:27:21,783 - lava_callback - ERROR - Error processing callback for node 69c2c96f6778148209fd6153: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:47482 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53724 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43784 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58008 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42292 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42306 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54790 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54804 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35004 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44304 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45694 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42490 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33608 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33622 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43792 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43794 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44470 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46926 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40026 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34778 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48508 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48510 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59452 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59468 - "GET / HTTP/1.0" 200 OK 2026-03-26 17:30:21,774 - lava_callback - INFO - Received callback for node 69c2ca186778148209fd6185 INFO: 172.21.0.1:43116 - "POST /node/69c2ca186778148209fd6185 HTTP/1.0" 202 Accepted 2026-03-26 17:30:21,781 - lava_callback - INFO - Starting async job submit for node 69c2ca186778148209fd6185 2026-03-26 17:30:21,782 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 17:30:21,796 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2ca186778148209fd6185 HTTP/1.1" 200 None 2026-03-26 17:30:21,796 - lava_callback - DEBUG - Processing node 69c2ca186778148209fd6185: baseline-arm64-clabbe 2026-03-26 17:30:21,797 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2ca186778148209fd6185 2026-03-26 17:30:21,797 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 17:30:21,823 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2ca186778148209fd6185 HTTP/1.1" 200 None 2026-03-26 17:30:21,824 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 17:30:21,842 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd73420e93e2314723e2c4 HTTP/1.1" 200 None 2026-03-26 17:30:21,843 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 17:30:21,863 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2ca186778148209fd6185 HTTP/1.1" 200 None 2026-03-26 17:30:21,874 - lava_callback - ERROR - Error processing callback for node 69c2ca186778148209fd6185: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:43128 - "GET /node/6992f07d31166481c8ddaa6c%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.21.0.1:43136 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48552 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37180 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46882 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36202 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36214 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33216 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33230 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39670 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51088 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46988 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59896 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36980 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36984 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48492 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48494 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38716 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52136 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44360 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48022 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54872 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54884 - "GET / HTTP/1.0" 200 OK 2026-03-26 17:33:07,734 - lava_callback - INFO - Received callback for node 69c56315abf66a8f4d9bbef1 INFO: 172.21.0.1:54896 - "POST /node/69c56315abf66a8f4d9bbef1 HTTP/1.0" 202 Accepted 2026-03-26 17:33:07,740 - lava_callback - INFO - Starting async job submit for node 69c56315abf66a8f4d9bbef1 2026-03-26 17:33:07,765 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 17:33:07,786 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c56315abf66a8f4d9bbef1 HTTP/1.1" 200 None 2026-03-26 17:33:07,787 - lava_callback - DEBUG - Processing node 69c56315abf66a8f4d9bbef1: ltp-fs_bind 2026-03-26 17:33:08,158 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 17:33:08,335 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 17:33:08,337 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-fs_bind-69c56315abf66a8f4d9bbef1/log.txt.gz 2026-03-26 17:33:08,565 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 17:33:08,801 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 17:33:08,803 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-fs_bind-69c56315abf66a8f4d9bbef1/lava-logs.yaml 2026-03-26 17:33:08,812 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 17:33:08,966 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 17:33:08,968 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-fs_bind-69c56315abf66a8f4d9bbef1/lava_callback.json.gz 2026-03-26 17:33:08,994 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 17:33:09,009 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c56315abf66a8f4d9bbef1 HTTP/1.1" 200 None 2026-03-26 17:33:09,011 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 17:33:09,025 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c55c70abf66a8f4d9b9655 HTTP/1.1" 200 None 2026-03-26 17:33:09,027 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 17:33:09,054 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c56315abf66a8f4d9bbef1 HTTP/1.1" 200 None 2026-03-26 17:33:09,055 - lava_callback - INFO - Completed processing callback for node 69c56315abf66a8f4d9bbef1 INFO: 172.21.0.1:33648 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33654 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36480 - "GET / HTTP/1.0" 200 OK 2026-03-26 17:33:28,664 - lava_callback - INFO - Received callback for node 69c56312abf66a8f4d9bbee7 INFO: 172.21.0.1:36496 - "POST /node/69c56312abf66a8f4d9bbee7 HTTP/1.0" 202 Accepted 2026-03-26 17:33:28,671 - lava_callback - INFO - Starting async job submit for node 69c56312abf66a8f4d9bbee7 2026-03-26 17:33:28,696 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 17:33:28,710 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c56312abf66a8f4d9bbee7 HTTP/1.1" 200 None 2026-03-26 17:33:28,710 - lava_callback - DEBUG - Processing node 69c56312abf66a8f4d9bbee7: ltp-cve 2026-03-26 17:33:29,104 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 17:33:29,247 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 17:33:29,249 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-cve-69c56312abf66a8f4d9bbee7/log.txt.gz 2026-03-26 17:33:29,480 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 17:33:29,678 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 17:33:29,680 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-cve-69c56312abf66a8f4d9bbee7/lava-logs.yaml 2026-03-26 17:33:29,689 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 17:33:29,841 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 17:33:29,843 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-cve-69c56312abf66a8f4d9bbee7/lava_callback.json.gz 2026-03-26 17:33:29,868 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 17:33:29,883 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c56312abf66a8f4d9bbee7 HTTP/1.1" 200 None 2026-03-26 17:33:29,884 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 17:33:29,901 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c55c70abf66a8f4d9b9655 HTTP/1.1" 200 None 2026-03-26 17:33:29,903 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 17:33:29,928 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c56312abf66a8f4d9bbee7 HTTP/1.1" 200 None 2026-03-26 17:33:29,929 - lava_callback - INFO - Completed processing callback for node 69c56312abf66a8f4d9bbee7 2026-03-26 17:33:32,296 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 17:33:32,329 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:38898 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53734 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35034 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58854 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58860 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49194 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49202 - "GET / HTTP/1.0" 200 OK 2026-03-26 17:34:21,988 - lava_callback - INFO - Received callback for node 69c2cb126778148209fd61c3 INFO: 172.21.0.1:52098 - "POST /node/69c2cb126778148209fd61c3 HTTP/1.0" 202 Accepted 2026-03-26 17:34:21,994 - lava_callback - INFO - Starting async job submit for node 69c2cb126778148209fd61c3 2026-03-26 17:34:21,995 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 17:34:22,010 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2cb126778148209fd61c3 HTTP/1.1" 200 None 2026-03-26 17:34:22,011 - lava_callback - DEBUG - Processing node 69c2cb126778148209fd61c3: baseline-arm64-clabbe 2026-03-26 17:34:22,011 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2cb126778148209fd61c3 2026-03-26 17:34:22,011 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 17:34:22,030 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2cb126778148209fd61c3 HTTP/1.1" 200 None 2026-03-26 17:34:22,032 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 17:34:22,047 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd73420e93e2314723e2c4 HTTP/1.1" 200 None 2026-03-26 17:34:22,049 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 17:34:22,072 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2cb126778148209fd61c3 HTTP/1.1" 200 None 2026-03-26 17:34:22,073 - lava_callback - ERROR - Error processing callback for node 69c2cb126778148209fd61c3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:52114 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53224 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48506 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59444 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56316 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56320 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35556 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35558 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36550 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59218 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36408 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49168 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45982 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45988 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45392 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45406 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44754 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34736 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56822 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51832 - "GET / HTTP/1.0" 200 OK 2026-03-26 17:36:57,744 - lava_callback - INFO - Received callback for node 69c565ceabf66a8f4d9bc092 INFO: 172.21.0.1:51836 - "POST /node/69c565ceabf66a8f4d9bc092 HTTP/1.0" 202 Accepted 2026-03-26 17:36:57,751 - lava_callback - INFO - Starting async job submit for node 69c565ceabf66a8f4d9bc092 2026-03-26 17:36:57,819 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 17:36:57,834 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c565ceabf66a8f4d9bc092 HTTP/1.1" 200 None 2026-03-26 17:36:57,835 - lava_callback - DEBUG - Processing node 69c565ceabf66a8f4d9bc092: ltp-smoke 2026-03-26 17:36:58,327 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 17:36:58,618 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 17:36:58,620 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoke-69c565ceabf66a8f4d9bc092/log.txt.gz 2026-03-26 17:36:58,898 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 17:36:59,282 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 17:36:59,284 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-smoke-69c565ceabf66a8f4d9bc092/lava-logs.yaml 2026-03-26 17:36:59,295 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 17:36:59,467 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 17:36:59,469 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoke-69c565ceabf66a8f4d9bc092/lava_callback.json.gz 2026-03-26 17:36:59,585 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 17:36:59,608 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c565ceabf66a8f4d9bc092 HTTP/1.1" 200 None 2026-03-26 17:36:59,610 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 17:36:59,624 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c55c76abf66a8f4d9b965b HTTP/1.1" 200 None 2026-03-26 17:36:59,626 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 17:36:59,656 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c565ceabf66a8f4d9bc092 HTTP/1.1" 200 None 2026-03-26 17:36:59,789 - lava_callback - INFO - Completed processing callback for node 69c565ceabf66a8f4d9bc092 2026-03-26 17:37:02,332 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 17:37:02,365 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:49052 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49060 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53588 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53600 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35698 - "GET /node/69876b95e31b58f834f19815%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.21.0.1:35704 - "GET /node/6988693ce408e9ea60faf7f6%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.21.0.1:35710 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35416 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48598 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57902 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55610 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55622 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58616 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58618 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40812 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58420 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60228 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36872 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39114 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39118 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59552 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59558 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60652 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54320 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38150 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41098 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58372 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58380 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54210 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54226 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34852 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48792 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46876 - "GET / HTTP/1.0" 200 OK 2026-03-26 17:40:48,077 - lava_callback - INFO - Received callback for node 69c56317abf66a8f4d9bbef4 INFO: 172.21.0.1:46888 - "POST /node/69c56317abf66a8f4d9bbef4 HTTP/1.0" 202 Accepted 2026-03-26 17:40:48,084 - lava_callback - INFO - Starting async job submit for node 69c56317abf66a8f4d9bbef4 2026-03-26 17:40:48,110 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 17:40:48,126 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c56317abf66a8f4d9bbef4 HTTP/1.1" 200 None 2026-03-26 17:40:48,128 - lava_callback - DEBUG - Processing node 69c56317abf66a8f4d9bbef4: ltp-fsx 2026-03-26 17:40:48,620 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 17:40:48,800 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 17:40:48,802 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-fsx-69c56317abf66a8f4d9bbef4/log.txt.gz 2026-03-26 17:40:49,071 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 17:40:49,436 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 17:40:49,438 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-fsx-69c56317abf66a8f4d9bbef4/lava-logs.yaml 2026-03-26 17:40:49,448 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 17:40:49,640 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 17:40:49,642 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-fsx-69c56317abf66a8f4d9bbef4/lava_callback.json.gz 2026-03-26 17:40:49,684 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 17:40:49,714 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c56317abf66a8f4d9bbef4 HTTP/1.1" 200 None 2026-03-26 17:40:49,717 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 17:40:49,738 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c55c70abf66a8f4d9b9655 HTTP/1.1" 200 None 2026-03-26 17:40:49,740 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 17:40:49,773 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c56317abf66a8f4d9bbef4 HTTP/1.1" 200 None 2026-03-26 17:40:49,774 - lava_callback - INFO - Completed processing callback for node 69c56317abf66a8f4d9bbef4 INFO: 172.21.0.1:36834 - "GET / HTTP/1.0" 200 OK 2026-03-26 17:41:02,368 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 17:41:02,401 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:43324 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43340 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57862 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57874 - "GET / HTTP/1.0" 200 OK 2026-03-26 17:41:18,261 - lava_callback - INFO - Received callback for node 69c56318abf66a8f4d9bbef7 INFO: 172.21.0.1:57890 - "POST /node/69c56318abf66a8f4d9bbef7 HTTP/1.0" 202 Accepted 2026-03-26 17:41:18,271 - lava_callback - INFO - Starting async job submit for node 69c56318abf66a8f4d9bbef7 2026-03-26 17:41:18,308 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 17:41:18,332 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c56318abf66a8f4d9bbef7 HTTP/1.1" 200 None 2026-03-26 17:41:18,333 - lava_callback - DEBUG - Processing node 69c56318abf66a8f4d9bbef7: ltp-input 2026-03-26 17:41:18,722 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 17:41:18,884 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 17:41:18,886 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-input-69c56318abf66a8f4d9bbef7/log.txt.gz 2026-03-26 17:41:19,124 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 17:41:19,361 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 17:41:19,363 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-input-69c56318abf66a8f4d9bbef7/lava-logs.yaml 2026-03-26 17:41:19,372 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 17:41:19,557 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 17:41:19,559 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-input-69c56318abf66a8f4d9bbef7/lava_callback.json.gz 2026-03-26 17:41:19,602 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 17:41:19,624 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c56318abf66a8f4d9bbef7 HTTP/1.1" 200 None 2026-03-26 17:41:19,626 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 17:41:19,644 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c55c70abf66a8f4d9b9655 HTTP/1.1" 200 None 2026-03-26 17:41:19,646 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 17:41:19,678 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c56318abf66a8f4d9bbef7 HTTP/1.1" 200 None 2026-03-26 17:41:19,680 - lava_callback - INFO - Completed processing callback for node 69c56318abf66a8f4d9bbef7 INFO: 172.21.0.1:48264 - "GET / HTTP/1.0" 200 OK 2026-03-26 17:41:32,404 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 17:41:32,424 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:47002 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49252 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60596 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43582 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43598 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49570 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49578 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33858 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43226 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50342 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34334 - "GET / HTTP/1.0" 200 OK 2026-03-26 17:43:02,361 - lava_callback - INFO - Received callback for node 69c2cd156778148209fd6252 INFO: 172.21.0.1:47506 - "POST /node/69c2cd156778148209fd6252 HTTP/1.0" 202 Accepted 2026-03-26 17:43:02,367 - lava_callback - INFO - Starting async job submit for node 69c2cd156778148209fd6252 2026-03-26 17:43:02,368 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 17:43:02,389 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2cd156778148209fd6252 HTTP/1.1" 200 None 2026-03-26 17:43:02,391 - lava_callback - DEBUG - Processing node 69c2cd156778148209fd6252: baseline-arm-clabbe 2026-03-26 17:43:02,391 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2cd156778148209fd6252 2026-03-26 17:43:02,392 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 17:43:02,412 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2cd156778148209fd6252 HTTP/1.1" 200 None 2026-03-26 17:43:02,414 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 17:43:02,440 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd732d0e93e2314723e2b5 HTTP/1.1" 200 None 2026-03-26 17:43:02,442 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 17:43:02,476 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2cd156778148209fd6252 HTTP/1.1" 200 None 2026-03-26 17:43:02,477 - lava_callback - ERROR - Error processing callback for node 69c2cd156778148209fd6252: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:47508 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47514 - "GET / HTTP/1.0" 200 OK 2026-03-26 17:43:08,167 - lava_callback - INFO - Received callback for node 69c5631aabf66a8f4d9bbefc INFO: 172.21.0.1:47522 - "POST /node/69c5631aabf66a8f4d9bbefc HTTP/1.0" 202 Accepted 2026-03-26 17:43:08,175 - lava_callback - INFO - Starting async job submit for node 69c5631aabf66a8f4d9bbefc 2026-03-26 17:43:08,205 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 17:43:08,226 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5631aabf66a8f4d9bbefc HTTP/1.1" 200 None 2026-03-26 17:43:08,227 - lava_callback - DEBUG - Processing node 69c5631aabf66a8f4d9bbefc: ltp-ipc 2026-03-26 17:43:08,595 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 17:43:08,753 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 17:43:08,755 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-ipc-69c5631aabf66a8f4d9bbefc/log.txt.gz 2026-03-26 17:43:09,110 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 17:43:09,381 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 17:43:09,383 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-ipc-69c5631aabf66a8f4d9bbefc/lava-logs.yaml 2026-03-26 17:43:09,392 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 17:43:09,531 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 17:43:09,533 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-ipc-69c5631aabf66a8f4d9bbefc/lava_callback.json.gz 2026-03-26 17:43:09,559 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 17:43:09,574 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5631aabf66a8f4d9bbefc HTTP/1.1" 200 None 2026-03-26 17:43:09,575 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 17:43:09,591 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c55c70abf66a8f4d9b9655 HTTP/1.1" 200 None 2026-03-26 17:43:09,593 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 17:43:09,617 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5631aabf66a8f4d9bbefc HTTP/1.1" 200 None 2026-03-26 17:43:09,619 - lava_callback - INFO - Completed processing callback for node 69c5631aabf66a8f4d9bbefc INFO: 172.21.0.1:36416 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36430 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46934 - "GET / HTTP/1.0" 200 OK 2026-03-26 17:43:32,427 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 17:43:32,443 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:60760 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42794 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37322 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54510 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54526 - "GET / HTTP/1.0" 200 OK 2026-03-26 17:44:08,136 - lava_callback - INFO - Received callback for node 69c5631babf66a8f4d9bbeff INFO: 172.21.0.1:54528 - "POST /node/69c5631babf66a8f4d9bbeff HTTP/1.0" 202 Accepted 2026-03-26 17:44:08,143 - lava_callback - INFO - Starting async job submit for node 69c5631babf66a8f4d9bbeff 2026-03-26 17:44:08,167 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 17:44:08,181 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5631babf66a8f4d9bbeff HTTP/1.1" 200 None 2026-03-26 17:44:08,182 - lava_callback - DEBUG - Processing node 69c5631babf66a8f4d9bbeff: ltp-pty 2026-03-26 17:44:08,521 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 17:44:08,632 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 17:44:08,634 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-pty-69c5631babf66a8f4d9bbeff/log.txt.gz 2026-03-26 17:44:08,868 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 17:44:09,054 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 17:44:09,056 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-pty-69c5631babf66a8f4d9bbeff/lava-logs.yaml 2026-03-26 17:44:09,065 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 17:44:09,372 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 17:44:09,374 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-pty-69c5631babf66a8f4d9bbeff/lava_callback.json.gz 2026-03-26 17:44:09,400 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 17:44:09,414 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5631babf66a8f4d9bbeff HTTP/1.1" 200 None 2026-03-26 17:44:09,415 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 17:44:09,431 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c55c70abf66a8f4d9b9655 HTTP/1.1" 200 None 2026-03-26 17:44:09,433 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 17:44:09,459 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5631babf66a8f4d9bbeff HTTP/1.1" 200 None 2026-03-26 17:44:09,460 - lava_callback - INFO - Completed processing callback for node 69c5631babf66a8f4d9bbeff INFO: 172.21.0.1:52132 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52134 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33682 - "GET / HTTP/1.0" 200 OK 2026-03-26 17:44:32,445 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 17:44:32,461 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:58700 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33492 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60624 - "GET / HTTP/1.0" 200 OK 2026-03-26 17:45:02,410 - lava_callback - INFO - Received callback for node 69c2cd8e6778148209fd6262 INFO: 172.21.0.1:41584 - "POST /node/69c2cd8e6778148209fd6262 HTTP/1.0" 202 Accepted 2026-03-26 17:45:02,416 - lava_callback - INFO - Starting async job submit for node 69c2cd8e6778148209fd6262 2026-03-26 17:45:02,417 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 17:45:02,442 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2cd8e6778148209fd6262 HTTP/1.1" 200 None 2026-03-26 17:45:02,443 - lava_callback - DEBUG - Processing node 69c2cd8e6778148209fd6262: baseline-arm-clabbe 2026-03-26 17:45:02,443 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2cd8e6778148209fd6262 2026-03-26 17:45:02,444 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 17:45:02,456 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2cd8e6778148209fd6262 HTTP/1.1" 200 None 2026-03-26 17:45:02,458 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 17:45:02,471 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd732d0e93e2314723e2b5 HTTP/1.1" 200 None 2026-03-26 17:45:02,472 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 17:45:02,497 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2cd8e6778148209fd6262 HTTP/1.1" 200 None 2026-03-26 17:45:02,499 - lava_callback - ERROR - Error processing callback for node 69c2cd8e6778148209fd6262: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:41600 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41614 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49706 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49720 - "GET / HTTP/1.0" 200 OK 2026-03-26 17:45:17,728 - lava_callback - INFO - Received callback for node 69c5631eabf66a8f4d9bbf06 INFO: 172.21.0.1:49734 - "POST /node/69c5631eabf66a8f4d9bbf06 HTTP/1.0" 202 Accepted 2026-03-26 17:45:17,735 - lava_callback - INFO - Starting async job submit for node 69c5631eabf66a8f4d9bbf06 2026-03-26 17:45:17,759 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 17:45:17,774 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5631eabf66a8f4d9bbf06 HTTP/1.1" 200 None 2026-03-26 17:45:17,774 - lava_callback - DEBUG - Processing node 69c5631eabf66a8f4d9bbf06: ltp-smoke 2026-03-26 17:45:18,115 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 17:45:18,443 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 17:45:18,445 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoke-69c5631eabf66a8f4d9bbf06/log.txt.gz 2026-03-26 17:45:18,653 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 17:45:18,892 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 17:45:18,894 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-smoke-69c5631eabf66a8f4d9bbf06/lava-logs.yaml 2026-03-26 17:45:18,903 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 17:45:19,081 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 17:45:19,083 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoke-69c5631eabf66a8f4d9bbf06/lava_callback.json.gz 2026-03-26 17:45:19,109 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 17:45:19,130 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5631eabf66a8f4d9bbf06 HTTP/1.1" 200 None 2026-03-26 17:45:19,132 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 17:45:19,146 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c55c70abf66a8f4d9b9655 HTTP/1.1" 200 None 2026-03-26 17:45:19,148 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 17:45:19,172 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5631eabf66a8f4d9bbf06 HTTP/1.1" 200 None 2026-03-26 17:45:19,173 - lava_callback - INFO - Completed processing callback for node 69c5631eabf66a8f4d9bbf06 INFO: 172.21.0.1:39314 - "GET / HTTP/1.0" 200 OK 2026-03-26 17:45:28,774 - lava_callback - INFO - Received callback for node 69c5631dabf66a8f4d9bbf03 INFO: 172.21.0.1:39322 - "POST /node/69c5631dabf66a8f4d9bbf03 HTTP/1.0" 202 Accepted 2026-03-26 17:45:28,781 - lava_callback - INFO - Starting async job submit for node 69c5631dabf66a8f4d9bbf03 2026-03-26 17:45:28,813 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 17:45:28,839 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5631dabf66a8f4d9bbf03 HTTP/1.1" 200 None 2026-03-26 17:45:28,840 - lava_callback - DEBUG - Processing node 69c5631dabf66a8f4d9bbf03: ltp-sched 2026-03-26 17:45:29,229 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 17:45:29,387 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 17:45:29,388 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-sched-69c5631dabf66a8f4d9bbf03/log.txt.gz 2026-03-26 17:45:29,607 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 17:45:29,833 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 17:45:29,835 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-sched-69c5631dabf66a8f4d9bbf03/lava-logs.yaml 2026-03-26 17:45:29,844 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 17:45:30,029 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 17:45:30,030 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-sched-69c5631dabf66a8f4d9bbf03/lava_callback.json.gz 2026-03-26 17:45:30,056 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 17:45:30,076 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5631dabf66a8f4d9bbf03 HTTP/1.1" 200 None 2026-03-26 17:45:30,078 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 17:45:30,093 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c55c70abf66a8f4d9b9655 HTTP/1.1" 200 None 2026-03-26 17:45:30,094 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 17:45:30,120 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5631dabf66a8f4d9bbf03 HTTP/1.1" 200 None 2026-03-26 17:45:30,121 - lava_callback - INFO - Completed processing callback for node 69c5631dabf66a8f4d9bbf03 2026-03-26 17:45:32,463 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 17:45:32,483 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:51772 - "GET / HTTP/1.0" 200 OK 2026-03-26 17:45:42,448 - lava_callback - INFO - Received callback for node 69c2cdb86778148209fd6273 INFO: 172.21.0.1:60994 - "POST /node/69c2cdb86778148209fd6273 HTTP/1.0" 202 Accepted 2026-03-26 17:45:42,453 - lava_callback - INFO - Starting async job submit for node 69c2cdb86778148209fd6273 2026-03-26 17:45:42,454 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 17:45:42,469 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2cdb86778148209fd6273 HTTP/1.1" 200 None 2026-03-26 17:45:42,470 - lava_callback - DEBUG - Processing node 69c2cdb86778148209fd6273: baseline-arm-clabbe 2026-03-26 17:45:42,470 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2cdb86778148209fd6273 2026-03-26 17:45:42,470 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 17:45:42,484 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2cdb86778148209fd6273 HTTP/1.1" 200 None 2026-03-26 17:45:42,485 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 17:45:42,498 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd732d0e93e2314723e2b5 HTTP/1.1" 200 None 2026-03-26 17:45:42,500 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 17:45:42,521 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2cdb86778148209fd6273 HTTP/1.1" 200 None 2026-03-26 17:45:42,526 - lava_callback - ERROR - Error processing callback for node 69c2cdb86778148209fd6273: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:32770 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59100 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42938 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42950 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35238 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35240 - "GET / HTTP/1.0" 200 OK 2026-03-26 17:46:22,626 - lava_callback - INFO - Received callback for node 69c2cde36778148209fd627e INFO: 172.21.0.1:46086 - "POST /node/69c2cde36778148209fd627e HTTP/1.0" 202 Accepted 2026-03-26 17:46:22,631 - lava_callback - INFO - Starting async job submit for node 69c2cde36778148209fd627e 2026-03-26 17:46:22,632 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 17:46:22,649 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2cde36778148209fd627e HTTP/1.1" 200 None 2026-03-26 17:46:22,650 - lava_callback - DEBUG - Processing node 69c2cde36778148209fd627e: baseline-arm-clabbe 2026-03-26 17:46:22,650 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2cde36778148209fd627e 2026-03-26 17:46:22,651 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 17:46:22,669 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2cde36778148209fd627e HTTP/1.1" 200 None 2026-03-26 17:46:22,671 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 17:46:22,683 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd732d0e93e2314723e2b5 HTTP/1.1" 200 None 2026-03-26 17:46:22,685 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 17:46:22,731 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2cde36778148209fd627e HTTP/1.1" 200 None 2026-03-26 17:46:22,737 - lava_callback - ERROR - Error processing callback for node 69c2cde36778148209fd627e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:46094 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37420 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60530 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51876 - "GET / HTTP/1.0" 200 OK 2026-03-26 17:47:02,512 - lava_callback - INFO - Received callback for node 69c2ce0c6778148209fd6289 INFO: 172.21.0.1:43230 - "POST /node/69c2ce0c6778148209fd6289 HTTP/1.0" 202 Accepted 2026-03-26 17:47:02,517 - lava_callback - INFO - Starting async job submit for node 69c2ce0c6778148209fd6289 2026-03-26 17:47:02,518 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 17:47:02,544 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2ce0c6778148209fd6289 HTTP/1.1" 200 None 2026-03-26 17:47:02,545 - lava_callback - DEBUG - Processing node 69c2ce0c6778148209fd6289: baseline-arm-clabbe 2026-03-26 17:47:02,545 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2ce0c6778148209fd6289 2026-03-26 17:47:02,546 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 17:47:02,559 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2ce0c6778148209fd6289 HTTP/1.1" 200 None 2026-03-26 17:47:02,560 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 17:47:02,574 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd732d0e93e2314723e2b5 HTTP/1.1" 200 None 2026-03-26 17:47:02,575 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 17:47:02,596 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2ce0c6778148209fd6289 HTTP/1.1" 200 None 2026-03-26 17:47:02,607 - lava_callback - ERROR - Error processing callback for node 69c2ce0c6778148209fd6289: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:43244 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43250 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39808 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39812 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51548 - "GET / HTTP/1.0" 200 OK 2026-03-26 17:47:28,255 - lava_callback - INFO - Received callback for node 69c56321abf66a8f4d9bbf0c INFO: 172.21.0.1:51560 - "POST /node/69c56321abf66a8f4d9bbf0c HTTP/1.0" 202 Accepted 2026-03-26 17:47:28,261 - lava_callback - INFO - Starting async job submit for node 69c56321abf66a8f4d9bbf0c 2026-03-26 17:47:28,286 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 17:47:28,300 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c56321abf66a8f4d9bbf0c HTTP/1.1" 200 None 2026-03-26 17:47:28,301 - lava_callback - DEBUG - Processing node 69c56321abf66a8f4d9bbf0c: ltp-syscalls-ipc 2026-03-26 17:47:28,667 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 17:47:28,976 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 17:47:28,978 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-syscalls-ipc-69c56321abf66a8f4d9bbf0c/log.txt.gz 2026-03-26 17:47:29,192 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 17:47:29,431 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 17:47:29,433 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-syscalls-ipc-69c56321abf66a8f4d9bbf0c/lava-logs.yaml 2026-03-26 17:47:29,442 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 17:47:29,657 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 17:47:29,659 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-syscalls-ipc-69c56321abf66a8f4d9bbf0c/lava_callback.json.gz 2026-03-26 17:47:29,684 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 17:47:29,699 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c56321abf66a8f4d9bbf0c HTTP/1.1" 200 None 2026-03-26 17:47:29,701 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 17:47:29,724 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c55c70abf66a8f4d9b9655 HTTP/1.1" 200 None 2026-03-26 17:47:29,725 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 17:47:29,750 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c56321abf66a8f4d9bbf0c HTTP/1.1" 200 None 2026-03-26 17:47:29,751 - lava_callback - INFO - Completed processing callback for node 69c56321abf66a8f4d9bbf0c 2026-03-26 17:47:32,485 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 17:47:32,501 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:33388 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43840 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39526 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43282 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43288 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38008 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38016 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36896 - "GET / HTTP/1.0" 200 OK 2026-03-26 17:48:28,241 - lava_callback - INFO - Received callback for node 69c56323abf66a8f4d9bbf10 INFO: 172.21.0.1:36900 - "POST /node/69c56323abf66a8f4d9bbf10 HTTP/1.0" 202 Accepted 2026-03-26 17:48:28,249 - lava_callback - INFO - Starting async job submit for node 69c56323abf66a8f4d9bbf10 2026-03-26 17:48:28,274 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 17:48:28,312 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c56323abf66a8f4d9bbf10 HTTP/1.1" 200 None 2026-03-26 17:48:28,313 - lava_callback - DEBUG - Processing node 69c56323abf66a8f4d9bbf10: ltp-timers 2026-03-26 17:48:28,667 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 17:48:28,805 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 17:48:28,807 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-timers-69c56323abf66a8f4d9bbf10/log.txt.gz 2026-03-26 17:48:29,021 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 17:48:29,231 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 17:48:29,233 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-timers-69c56323abf66a8f4d9bbf10/lava-logs.yaml 2026-03-26 17:48:29,241 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 17:48:29,380 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 17:48:29,382 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-timers-69c56323abf66a8f4d9bbf10/lava_callback.json.gz 2026-03-26 17:48:29,407 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 17:48:29,425 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c56323abf66a8f4d9bbf10 HTTP/1.1" 200 None 2026-03-26 17:48:29,426 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 17:48:29,440 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c55c70abf66a8f4d9b9655 HTTP/1.1" 200 None 2026-03-26 17:48:29,442 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 17:48:29,467 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c56323abf66a8f4d9bbf10 HTTP/1.1" 200 None 2026-03-26 17:48:29,473 - lava_callback - INFO - Completed processing callback for node 69c56323abf66a8f4d9bbf10 2026-03-26 17:48:32,503 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 17:48:32,519 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:41274 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36072 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34040 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59322 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59332 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53130 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53140 - "GET / HTTP/1.0" 200 OK 2026-03-26 17:49:17,869 - lava_callback - INFO - Received callback for node 69c56325abf66a8f4d9bbf32 INFO: 172.21.0.1:53156 - "POST /node/69c56325abf66a8f4d9bbf32 HTTP/1.0" 202 Accepted 2026-03-26 17:49:17,875 - lava_callback - INFO - Starting async job submit for node 69c56325abf66a8f4d9bbf32 2026-03-26 17:49:17,900 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 17:49:17,914 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c56325abf66a8f4d9bbf32 HTTP/1.1" 200 None 2026-03-26 17:49:17,915 - lava_callback - DEBUG - Processing node 69c56325abf66a8f4d9bbf32: ltp-watchqueue 2026-03-26 17:49:18,289 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 17:49:18,425 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 17:49:18,427 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-watchqueue-69c56325abf66a8f4d9bbf32/log.txt.gz 2026-03-26 17:49:18,630 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 17:49:18,855 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 17:49:18,857 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-watchqueue-69c56325abf66a8f4d9bbf32/lava-logs.yaml 2026-03-26 17:49:18,865 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 17:49:19,019 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 17:49:19,021 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-watchqueue-69c56325abf66a8f4d9bbf32/lava_callback.json.gz 2026-03-26 17:49:19,054 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 17:49:19,068 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c56325abf66a8f4d9bbf32 HTTP/1.1" 200 None 2026-03-26 17:49:19,070 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 17:49:19,083 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c55c70abf66a8f4d9b9655 HTTP/1.1" 200 None 2026-03-26 17:49:19,085 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 17:49:19,128 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c56325abf66a8f4d9bbf32 HTTP/1.1" 200 None 2026-03-26 17:49:19,129 - lava_callback - INFO - Completed processing callback for node 69c56325abf66a8f4d9bbf32 INFO: 172.21.0.1:38236 - "GET / HTTP/1.0" 200 OK 2026-03-26 17:49:32,522 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 17:49:32,538 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:36586 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38998 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45838 - "GET / HTTP/1.0" 200 OK 2026-03-26 17:50:05,116 - lava_callback - INFO - Received callback for node 69c2ceb86778148209fd62ae INFO: 172.21.0.1:42772 - "POST /node/69c2ceb86778148209fd62ae HTTP/1.0" 202 Accepted 2026-03-26 17:50:05,122 - lava_callback - INFO - Starting async job submit for node 69c2ceb86778148209fd62ae 2026-03-26 17:50:05,122 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 17:50:05,137 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2ceb86778148209fd62ae HTTP/1.1" 200 None 2026-03-26 17:50:05,138 - lava_callback - DEBUG - Processing node 69c2ceb86778148209fd62ae: baseline-arm64-baylibre 2026-03-26 17:50:05,138 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2ceb86778148209fd62ae 2026-03-26 17:50:05,138 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 17:50:05,152 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2ceb86778148209fd62ae HTTP/1.1" 200 None 2026-03-26 17:50:05,154 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 17:50:05,169 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad0a721a77312e2220ef1 HTTP/1.1" 200 None 2026-03-26 17:50:05,172 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 17:50:05,195 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2ceb86778148209fd62ae HTTP/1.1" 200 None 2026-03-26 17:50:05,200 - lava_callback - ERROR - Error processing callback for node 69c2ceb86778148209fd62ae: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:42778 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42784 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60384 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60400 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41358 - "GET / HTTP/1.0" 200 OK 2026-03-26 17:50:29,875 - lava_callback - INFO - Received callback for node 69c56dafabf66a8f4d9bc2a8 INFO: 172.21.0.1:52890 - "POST /node/69c56dafabf66a8f4d9bc2a8 HTTP/1.0" 202 Accepted 2026-03-26 17:50:29,897 - lava_callback - INFO - Starting async job submit for node 69c56dafabf66a8f4d9bc2a8 2026-03-26 17:50:29,962 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 17:50:29,976 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c56dafabf66a8f4d9bc2a8 HTTP/1.1" 200 None 2026-03-26 17:50:29,977 - lava_callback - DEBUG - Processing node 69c56dafabf66a8f4d9bc2a8: baseline-arm 2026-03-26 17:50:30,546 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 17:50:30,679 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 17:50:30,681 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69c56dafabf66a8f4d9bc2a8/log.txt.gz 2026-03-26 17:50:31,043 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 17:50:31,473 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 17:50:31,475 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c56dafabf66a8f4d9bc2a8/lava-logs.yaml 2026-03-26 17:50:31,489 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 17:50:31,660 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 17:50:31,662 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c56dafabf66a8f4d9bc2a8/lava_callback.json.gz DEBUG: Setup failed for 69c56dafabf66a8f4d9bc2a8. Transitting job node result: incomplete -> fail 2026-03-26 17:50:31,769 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 17:50:31,784 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c56dafabf66a8f4d9bc2a8 HTTP/1.1" 200 None 2026-03-26 17:50:31,785 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 17:50:31,799 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c55c70abf66a8f4d9b9655 HTTP/1.1" 200 None 2026-03-26 17:50:31,801 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 17:50:31,831 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c56dafabf66a8f4d9bc2a8 HTTP/1.1" 200 None 2026-03-26 17:50:31,939 - lava_callback - INFO - Completed processing callback for node 69c56dafabf66a8f4d9bc2a8 2026-03-26 17:50:32,540 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 17:50:32,561 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:52896 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55626 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46622 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40026 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40040 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55814 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55818 - "GET / HTTP/1.0" 200 OK 2026-03-26 17:51:25,053 - lava_callback - INFO - Received callback for node 69c2cf0b6778148209fd62c3 INFO: 172.21.0.1:40106 - "POST /node/69c2cf0b6778148209fd62c3 HTTP/1.0" 202 Accepted 2026-03-26 17:51:25,058 - lava_callback - INFO - Starting async job submit for node 69c2cf0b6778148209fd62c3 2026-03-26 17:51:25,059 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 17:51:25,077 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2cf0b6778148209fd62c3 HTTP/1.1" 200 None 2026-03-26 17:51:25,078 - lava_callback - DEBUG - Processing node 69c2cf0b6778148209fd62c3: baseline-arm64-baylibre 2026-03-26 17:51:25,078 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2cf0b6778148209fd62c3 2026-03-26 17:51:25,079 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 17:51:25,092 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2cf0b6778148209fd62c3 HTTP/1.1" 200 None 2026-03-26 17:51:25,093 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 17:51:25,107 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad0a721a77312e2220ef1 HTTP/1.1" 200 None 2026-03-26 17:51:25,108 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 17:51:25,128 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2cf0b6778148209fd62c3 HTTP/1.1" 200 None 2026-03-26 17:51:25,130 - lava_callback - ERROR - Error processing callback for node 69c2cf0b6778148209fd62c3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:40112 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48608 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50364 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55592 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58212 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58226 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48202 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48206 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33260 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33024 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56190 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53440 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41620 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41624 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43884 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43888 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47064 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33752 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37676 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58136 - "GET / HTTP/1.0" 200 OK 2026-03-26 17:54:05,204 - lava_callback - INFO - Received callback for node 69c2cfb36778148209fd62ec INFO: 172.21.0.1:46024 - "POST /node/69c2cfb36778148209fd62ec HTTP/1.0" 202 Accepted 2026-03-26 17:54:05,209 - lava_callback - INFO - Starting async job submit for node 69c2cfb36778148209fd62ec 2026-03-26 17:54:05,210 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 17:54:05,246 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2cfb36778148209fd62ec HTTP/1.1" 200 None 2026-03-26 17:54:05,247 - lava_callback - DEBUG - Processing node 69c2cfb36778148209fd62ec: baseline-arm64-baylibre 2026-03-26 17:54:05,247 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2cfb36778148209fd62ec 2026-03-26 17:54:05,248 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 17:54:05,261 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2cfb36778148209fd62ec HTTP/1.1" 200 None 2026-03-26 17:54:05,262 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 17:54:05,299 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad0a721a77312e2220ef1 HTTP/1.1" 200 None 2026-03-26 17:54:05,300 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 17:54:05,321 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2cfb36778148209fd62ec HTTP/1.1" 200 None 2026-03-26 17:54:05,328 - lava_callback - ERROR - Error processing callback for node 69c2cfb36778148209fd62ec: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:46034 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46046 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42350 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42364 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53060 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57402 - "GET /node/6991f6f801ff1e8e0859d227%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.21.0.1:57412 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46510 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34820 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39800 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39804 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54946 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54954 - "GET / HTTP/1.0" 200 OK 2026-03-26 17:55:25,253 - lava_callback - INFO - Received callback for node 69c2d0046778148209fd62f8 INFO: 172.21.0.1:54438 - "POST /node/69c2d0046778148209fd62f8 HTTP/1.0" 202 Accepted 2026-03-26 17:55:25,258 - lava_callback - INFO - Starting async job submit for node 69c2d0046778148209fd62f8 2026-03-26 17:55:25,259 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 17:55:25,299 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2d0046778148209fd62f8 HTTP/1.1" 200 None 2026-03-26 17:55:25,300 - lava_callback - DEBUG - Processing node 69c2d0046778148209fd62f8: baseline-arm64-baylibre 2026-03-26 17:55:25,300 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2d0046778148209fd62f8 2026-03-26 17:55:25,301 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 17:55:25,328 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2d0046778148209fd62f8 HTTP/1.1" 200 None 2026-03-26 17:55:25,330 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 17:55:25,367 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad0a721a77312e2220ef1 HTTP/1.1" 200 None 2026-03-26 17:55:25,369 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 17:55:25,437 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2d0046778148209fd62f8 HTTP/1.1" 200 None 2026-03-26 17:55:25,444 - lava_callback - ERROR - Error processing callback for node 69c2d0046778148209fd62f8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:54450 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53674 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43928 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58676 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56848 - "GET /node/69870e966e077e4126a7ed08%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.21.0.1:56854 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56858 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47604 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47618 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34560 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43200 - "GET / HTTP/1.0" 200 OK 2026-03-26 17:56:45,338 - lava_callback - INFO - Received callback for node 69c2d0566778148209fd630a INFO: 172.21.0.1:53842 - "POST /node/69c2d0566778148209fd630a HTTP/1.0" 202 Accepted 2026-03-26 17:56:45,344 - lava_callback - INFO - Starting async job submit for node 69c2d0566778148209fd630a 2026-03-26 17:56:45,345 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 17:56:45,360 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2d0566778148209fd630a HTTP/1.1" 200 None 2026-03-26 17:56:45,361 - lava_callback - DEBUG - Processing node 69c2d0566778148209fd630a: baseline-arm64-baylibre 2026-03-26 17:56:45,361 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2d0566778148209fd630a 2026-03-26 17:56:45,361 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 17:56:45,404 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2d0566778148209fd630a HTTP/1.1" 200 None 2026-03-26 17:56:45,405 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 17:56:45,419 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0163fc3d73288b0d05306 HTTP/1.1" 200 None 2026-03-26 17:56:45,421 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 17:56:45,446 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2d0566778148209fd630a HTTP/1.1" 200 None 2026-03-26 17:56:45,447 - lava_callback - ERROR - Error processing callback for node 69c2d0566778148209fd630a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:53846 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52500 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52464 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52466 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50968 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50972 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38424 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47670 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60740 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57916 - "GET / HTTP/1.0" 200 OK 2026-03-26 17:58:05,400 - lava_callback - INFO - Received callback for node 69c2d0aa6778148209fd631d INFO: 172.21.0.1:52544 - "POST /node/69c2d0aa6778148209fd631d HTTP/1.0" 202 Accepted 2026-03-26 17:58:05,405 - lava_callback - INFO - Starting async job submit for node 69c2d0aa6778148209fd631d 2026-03-26 17:58:05,406 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 17:58:05,434 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2d0aa6778148209fd631d HTTP/1.1" 200 None 2026-03-26 17:58:05,435 - lava_callback - DEBUG - Processing node 69c2d0aa6778148209fd631d: baseline-arm64-baylibre 2026-03-26 17:58:05,435 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2d0aa6778148209fd631d 2026-03-26 17:58:05,436 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 17:58:05,450 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2d0aa6778148209fd631d HTTP/1.1" 200 None 2026-03-26 17:58:05,451 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 17:58:05,466 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0163fc3d73288b0d05306 HTTP/1.1" 200 None 2026-03-26 17:58:05,467 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 17:58:05,489 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2d0aa6778148209fd631d HTTP/1.1" 200 None 2026-03-26 17:58:05,491 - lava_callback - ERROR - Error processing callback for node 69c2d0aa6778148209fd631d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:52556 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52562 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55980 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55986 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37828 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45318 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33418 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39980 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55082 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55096 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36710 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36720 - "GET / HTTP/1.0" 200 OK 2026-03-26 17:59:25,568 - lava_callback - INFO - Received callback for node 69c2d0fb6778148209fd6330 INFO: 172.21.0.1:48386 - "POST /node/69c2d0fb6778148209fd6330 HTTP/1.0" 202 Accepted 2026-03-26 17:59:25,573 - lava_callback - INFO - Starting async job submit for node 69c2d0fb6778148209fd6330 2026-03-26 17:59:25,574 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 17:59:25,589 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2d0fb6778148209fd6330 HTTP/1.1" 200 None 2026-03-26 17:59:25,590 - lava_callback - DEBUG - Processing node 69c2d0fb6778148209fd6330: baseline-arm64-baylibre 2026-03-26 17:59:25,590 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2d0fb6778148209fd6330 2026-03-26 17:59:25,591 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 17:59:25,604 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2d0fb6778148209fd6330 HTTP/1.1" 200 None 2026-03-26 17:59:25,605 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 17:59:25,622 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0163fc3d73288b0d05306 HTTP/1.1" 200 None 2026-03-26 17:59:25,623 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 17:59:25,645 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2d0fb6778148209fd6330 HTTP/1.1" 200 None 2026-03-26 17:59:25,647 - lava_callback - ERROR - Error processing callback for node 69c2d0fb6778148209fd6330: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:48388 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49262 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59316 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52824 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48930 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48938 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50530 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50544 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32910 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57014 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41848 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50400 - "GET / HTTP/1.0" 200 OK 2026-03-26 18:01:05,665 - lava_callback - INFO - Received callback for node 69c2d1506778148209fd633c INFO: 172.21.0.1:52644 - "POST /node/69c2d1506778148209fd633c HTTP/1.0" 202 Accepted 2026-03-26 18:01:05,674 - lava_callback - INFO - Starting async job submit for node 69c2d1506778148209fd633c 2026-03-26 18:01:05,675 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 18:01:05,692 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2d1506778148209fd633c HTTP/1.1" 200 None 2026-03-26 18:01:05,693 - lava_callback - DEBUG - Processing node 69c2d1506778148209fd633c: baseline-arm64-baylibre 2026-03-26 18:01:05,693 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2d1506778148209fd633c 2026-03-26 18:01:05,693 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 18:01:05,720 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2d1506778148209fd633c HTTP/1.1" 200 None 2026-03-26 18:01:05,722 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 18:01:05,736 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0163fc3d73288b0d05306 HTTP/1.1" 200 None 2026-03-26 18:01:05,737 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 18:01:05,765 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2d1506778148209fd633c HTTP/1.1" 200 None 2026-03-26 18:01:05,773 - lava_callback - ERROR - Error processing callback for node 69c2d1506778148209fd633c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:52650 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52666 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41186 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41194 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51230 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34112 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45172 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41438 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41020 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41032 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57626 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57634 - "GET / HTTP/1.0" 200 OK 2026-03-26 18:02:25,631 - lava_callback - INFO - Received callback for node 69c2d1a76778148209fd6366 INFO: 172.21.0.1:38754 - "POST /node/69c2d1a76778148209fd6366 HTTP/1.0" 202 Accepted 2026-03-26 18:02:25,636 - lava_callback - INFO - Starting async job submit for node 69c2d1a76778148209fd6366 2026-03-26 18:02:25,637 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 18:02:25,652 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2d1a76778148209fd6366 HTTP/1.1" 200 None 2026-03-26 18:02:25,653 - lava_callback - DEBUG - Processing node 69c2d1a76778148209fd6366: baseline-arm64-baylibre 2026-03-26 18:02:25,653 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2d1a76778148209fd6366 2026-03-26 18:02:25,653 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 18:02:25,667 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2d1a76778148209fd6366 HTTP/1.1" 200 None 2026-03-26 18:02:25,669 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 18:02:25,683 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0163fc3d73288b0d05306 HTTP/1.1" 200 None 2026-03-26 18:02:25,684 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 18:02:25,706 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2d1a76778148209fd6366 HTTP/1.1" 200 None 2026-03-26 18:02:25,713 - lava_callback - ERROR - Error processing callback for node 69c2d1a76778148209fd6366: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:38768 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39566 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50330 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48532 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49990 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50004 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60676 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60692 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40060 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36628 - "GET / HTTP/1.0" 200 OK 2026-03-26 18:03:45,681 - lava_callback - INFO - Received callback for node 69c2d1f56778148209fd6374 INFO: 172.21.0.1:38008 - "POST /node/69c2d1f56778148209fd6374 HTTP/1.0" 202 Accepted 2026-03-26 18:03:45,687 - lava_callback - INFO - Starting async job submit for node 69c2d1f56778148209fd6374 2026-03-26 18:03:45,688 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 18:03:45,703 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2d1f56778148209fd6374 HTTP/1.1" 200 None 2026-03-26 18:03:45,704 - lava_callback - DEBUG - Processing node 69c2d1f56778148209fd6374: baseline-arm64-baylibre 2026-03-26 18:03:45,705 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2d1f56778148209fd6374 2026-03-26 18:03:45,705 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 18:03:45,720 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2d1f56778148209fd6374 HTTP/1.1" 200 None 2026-03-26 18:03:45,721 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 18:03:45,735 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0163fc3d73288b0d05306 HTTP/1.1" 200 None 2026-03-26 18:03:45,737 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 18:03:45,762 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2d1f56778148209fd6374 HTTP/1.1" 200 None 2026-03-26 18:03:45,773 - lava_callback - ERROR - Error processing callback for node 69c2d1f56778148209fd6374: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:38024 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44010 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35606 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35616 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57240 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57250 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60868 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55436 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54800 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48340 - "GET / HTTP/1.0" 200 OK 2026-03-26 18:05:03,412 - lava_callback - INFO - Received callback for node 69c2d24a6778148209fd637f INFO: 172.21.0.1:46084 - "POST /node/69c2d24a6778148209fd637f HTTP/1.0" 202 Accepted 2026-03-26 18:05:03,417 - lava_callback - INFO - Starting async job submit for node 69c2d24a6778148209fd637f 2026-03-26 18:05:03,418 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 18:05:03,436 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2d24a6778148209fd637f HTTP/1.1" 200 None 2026-03-26 18:05:03,437 - lava_callback - DEBUG - Processing node 69c2d24a6778148209fd637f: baseline-arm64-clabbe 2026-03-26 18:05:03,437 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2d24a6778148209fd637f 2026-03-26 18:05:03,438 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 18:05:03,451 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2d24a6778148209fd637f HTTP/1.1" 200 None 2026-03-26 18:05:03,452 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 18:05:03,466 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0163fc3d73288b0d05306 HTTP/1.1" 200 None 2026-03-26 18:05:03,467 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 18:05:03,488 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2d24a6778148209fd637f HTTP/1.1" 200 None 2026-03-26 18:05:03,489 - lava_callback - ERROR - Error processing callback for node 69c2d24a6778148209fd637f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:46092 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46094 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36198 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36206 - "GET / HTTP/1.0" 200 OK 2026-03-26 18:05:20,269 - lava_callback - INFO - Received callback for node 69c57129abf66a8f4d9bc38c INFO: 172.21.0.1:44324 - "POST /node/69c57129abf66a8f4d9bc38c HTTP/1.0" 202 Accepted 2026-03-26 18:05:20,276 - lava_callback - INFO - Starting async job submit for node 69c57129abf66a8f4d9bc38c 2026-03-26 18:05:20,334 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 18:05:20,348 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c57129abf66a8f4d9bc38c HTTP/1.1" 200 None 2026-03-26 18:05:20,349 - lava_callback - DEBUG - Processing node 69c57129abf66a8f4d9bc38c: ltp-smoke 2026-03-26 18:05:20,818 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 18:05:20,956 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 18:05:20,958 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoke-69c57129abf66a8f4d9bc38c/log.txt.gz 2026-03-26 18:05:21,255 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 18:05:21,483 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 18:05:21,485 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-smoke-69c57129abf66a8f4d9bc38c/lava-logs.yaml 2026-03-26 18:05:21,496 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 18:05:21,820 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 18:05:21,821 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoke-69c57129abf66a8f4d9bc38c/lava_callback.json.gz 2026-03-26 18:05:21,939 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 18:05:21,953 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c57129abf66a8f4d9bc38c HTTP/1.1" 200 None 2026-03-26 18:05:21,955 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 18:05:21,968 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c55c76abf66a8f4d9b965b HTTP/1.1" 200 None 2026-03-26 18:05:21,970 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 18:05:21,995 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c57129abf66a8f4d9bc38c HTTP/1.1" 200 None 2026-03-26 18:05:22,122 - lava_callback - INFO - Completed processing callback for node 69c57129abf66a8f4d9bc38c INFO: 172.21.0.1:44336 - "GET / HTTP/1.0" 200 OK 2026-03-26 18:05:32,566 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 18:05:32,581 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:46360 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53166 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48978 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44462 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44472 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51938 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51942 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56410 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50624 - "GET / HTTP/1.0" 200 OK 2026-03-26 18:06:43,351 - lava_callback - INFO - Received callback for node 69c2d2a16778148209fd639e INFO: 172.21.0.1:46150 - "POST /node/69c2d2a16778148209fd639e HTTP/1.0" 202 Accepted 2026-03-26 18:06:43,356 - lava_callback - INFO - Starting async job submit for node 69c2d2a16778148209fd639e 2026-03-26 18:06:43,357 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 18:06:43,377 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2d2a16778148209fd639e HTTP/1.1" 200 None 2026-03-26 18:06:43,378 - lava_callback - DEBUG - Processing node 69c2d2a16778148209fd639e: baseline-arm64-clabbe 2026-03-26 18:06:43,378 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2d2a16778148209fd639e 2026-03-26 18:06:43,378 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 18:06:43,392 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2d2a16778148209fd639e HTTP/1.1" 200 None 2026-03-26 18:06:43,393 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 18:06:43,409 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0163fc3d73288b0d05306 HTTP/1.1" 200 None 2026-03-26 18:06:43,411 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 18:06:43,431 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2d2a16778148209fd639e HTTP/1.1" 200 None 2026-03-26 18:06:43,433 - lava_callback - ERROR - Error processing callback for node 69c2d2a16778148209fd639e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:46158 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36404 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52278 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52284 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33172 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33188 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46642 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57336 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33140 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52262 - "GET / HTTP/1.0" 200 OK 2026-03-26 18:08:03,421 - lava_callback - INFO - Received callback for node 69c2d2f66778148209fd63ad INFO: 172.21.0.1:32878 - "POST /node/69c2d2f66778148209fd63ad HTTP/1.0" 202 Accepted 2026-03-26 18:08:03,427 - lava_callback - INFO - Starting async job submit for node 69c2d2f66778148209fd63ad 2026-03-26 18:08:03,428 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 18:08:03,450 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2d2f66778148209fd63ad HTTP/1.1" 200 None 2026-03-26 18:08:03,451 - lava_callback - DEBUG - Processing node 69c2d2f66778148209fd63ad: baseline-arm64-clabbe 2026-03-26 18:08:03,451 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2d2f66778148209fd63ad 2026-03-26 18:08:03,451 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 18:08:03,466 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2d2f66778148209fd63ad HTTP/1.1" 200 None 2026-03-26 18:08:03,467 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 18:08:03,481 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0163fc3d73288b0d05306 HTTP/1.1" 200 None 2026-03-26 18:08:03,482 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 18:08:03,502 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2d2f66778148209fd63ad HTTP/1.1" 200 None 2026-03-26 18:08:03,508 - lava_callback - ERROR - Error processing callback for node 69c2d2f66778148209fd63ad: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:32886 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32902 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51490 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51502 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37888 - "GET / HTTP/1.0" 200 OK 2026-03-26 18:08:33,879 - lava_callback - INFO - Received callback for node 69c571f3abf66a8f4d9bc3eb INFO: 172.21.0.1:33968 - "POST /node/69c571f3abf66a8f4d9bc3eb HTTP/1.0" 202 Accepted 2026-03-26 18:08:33,885 - lava_callback - INFO - Starting async job submit for node 69c571f3abf66a8f4d9bc3eb 2026-03-26 18:08:33,940 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 18:08:33,959 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c571f3abf66a8f4d9bc3eb HTTP/1.1" 200 None 2026-03-26 18:08:33,960 - lava_callback - DEBUG - Processing node 69c571f3abf66a8f4d9bc3eb: baseline-arm 2026-03-26 18:08:34,543 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 18:08:34,696 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 18:08:34,697 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69c571f3abf66a8f4d9bc3eb/log.txt.gz 2026-03-26 18:08:35,045 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 18:08:35,275 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 18:08:35,277 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c571f3abf66a8f4d9bc3eb/lava-logs.yaml 2026-03-26 18:08:35,292 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 18:08:35,474 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 18:08:35,476 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c571f3abf66a8f4d9bc3eb/lava_callback.json.gz DEBUG: Setup failed for 69c571f3abf66a8f4d9bc3eb. Transitting job node result: incomplete -> fail 2026-03-26 18:08:35,584 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 18:08:35,598 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c571f3abf66a8f4d9bc3eb HTTP/1.1" 200 None 2026-03-26 18:08:35,600 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 18:08:35,614 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c55c70abf66a8f4d9b9655 HTTP/1.1" 200 None 2026-03-26 18:08:35,616 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 18:08:35,644 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c571f3abf66a8f4d9bc3eb HTTP/1.1" 200 None 2026-03-26 18:08:35,759 - lava_callback - INFO - Completed processing callback for node 69c571f3abf66a8f4d9bc3eb INFO: 172.21.0.1:33978 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36714 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55436 - "GET / HTTP/1.0" 200 OK 2026-03-26 18:09:02,584 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 18:09:02,601 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:39268 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39270 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59934 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59946 - "GET / HTTP/1.0" 200 OK 2026-03-26 18:09:23,490 - lava_callback - INFO - Received callback for node 69c2d34a6778148209fd63b2 INFO: 172.21.0.1:50016 - "POST /node/69c2d34a6778148209fd63b2 HTTP/1.0" 202 Accepted 2026-03-26 18:09:23,499 - lava_callback - INFO - Starting async job submit for node 69c2d34a6778148209fd63b2 2026-03-26 18:09:23,501 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 18:09:23,521 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2d34a6778148209fd63b2 HTTP/1.1" 200 None 2026-03-26 18:09:23,522 - lava_callback - DEBUG - Processing node 69c2d34a6778148209fd63b2: baseline-arm64-clabbe 2026-03-26 18:09:23,522 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2d34a6778148209fd63b2 2026-03-26 18:09:23,523 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 18:09:23,541 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2d34a6778148209fd63b2 HTTP/1.1" 200 None 2026-03-26 18:09:23,543 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 18:09:23,574 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0163fc3d73288b0d05306 HTTP/1.1" 200 None 2026-03-26 18:09:23,576 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 18:09:23,605 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2d34a6778148209fd63b2 HTTP/1.1" 200 None 2026-03-26 18:09:23,606 - lava_callback - ERROR - Error processing callback for node 69c2d34a6778148209fd63b2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:50028 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33312 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40926 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43092 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38238 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38246 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40316 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40326 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36316 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55284 - "GET / HTTP/1.0" 200 OK 2026-03-26 18:10:43,546 - lava_callback - INFO - Received callback for node 69c2d39f6778148209fd63bc INFO: 172.21.0.1:44192 - "POST /node/69c2d39f6778148209fd63bc HTTP/1.0" 202 Accepted 2026-03-26 18:10:43,551 - lava_callback - INFO - Starting async job submit for node 69c2d39f6778148209fd63bc 2026-03-26 18:10:43,552 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 18:10:43,566 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2d39f6778148209fd63bc HTTP/1.1" 200 None 2026-03-26 18:10:43,567 - lava_callback - DEBUG - Processing node 69c2d39f6778148209fd63bc: baseline-arm64-clabbe 2026-03-26 18:10:43,567 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2d39f6778148209fd63bc 2026-03-26 18:10:43,568 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 18:10:43,605 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2d39f6778148209fd63bc HTTP/1.1" 200 None 2026-03-26 18:10:43,607 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 18:10:43,638 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0163fc3d73288b0d05306 HTTP/1.1" 200 None 2026-03-26 18:10:43,639 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 18:10:43,660 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2d39f6778148209fd63bc HTTP/1.1" 200 None 2026-03-26 18:10:43,664 - lava_callback - ERROR - Error processing callback for node 69c2d39f6778148209fd63bc: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:44204 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50132 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53780 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53796 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59542 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59544 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40268 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36854 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42842 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48746 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36484 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36500 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41970 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41986 - "GET / HTTP/1.0" 200 OK 2026-03-26 18:12:23,588 - lava_callback - INFO - Received callback for node 69c2d3f46778148209fd63c8 INFO: 172.21.0.1:39404 - "POST /node/69c2d3f46778148209fd63c8 HTTP/1.0" 202 Accepted 2026-03-26 18:12:23,593 - lava_callback - INFO - Starting async job submit for node 69c2d3f46778148209fd63c8 2026-03-26 18:12:23,594 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 18:12:23,617 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2d3f46778148209fd63c8 HTTP/1.1" 200 None 2026-03-26 18:12:23,618 - lava_callback - DEBUG - Processing node 69c2d3f46778148209fd63c8: baseline-arm64-clabbe 2026-03-26 18:12:23,618 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2d3f46778148209fd63c8 2026-03-26 18:12:23,619 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 18:12:23,632 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2d3f46778148209fd63c8 HTTP/1.1" 200 None 2026-03-26 18:12:23,634 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 18:12:23,647 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0163fc3d73288b0d05306 HTTP/1.1" 200 None 2026-03-26 18:12:23,648 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 18:12:23,669 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2d3f46778148209fd63c8 HTTP/1.1" 200 None 2026-03-26 18:12:23,670 - lava_callback - ERROR - Error processing callback for node 69c2d3f46778148209fd63c8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:39414 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39478 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39726 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55024 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51666 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51674 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51548 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51558 - "GET / HTTP/1.0" 200 OK 2026-03-26 18:13:26,284 - lava_callback - INFO - Received callback for node 69c2d4426778148209fd63d4 INFO: 172.21.0.1:56410 - "POST /node/69c2d4426778148209fd63d4 HTTP/1.0" 202 Accepted 2026-03-26 18:13:26,290 - lava_callback - INFO - Starting async job submit for node 69c2d4426778148209fd63d4 2026-03-26 18:13:26,291 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 18:13:26,305 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2d4426778148209fd63d4 HTTP/1.1" 200 None 2026-03-26 18:13:26,305 - lava_callback - DEBUG - Processing node 69c2d4426778148209fd63d4: baseline-arm64-baylibre 2026-03-26 18:13:26,306 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2d4426778148209fd63d4 2026-03-26 18:13:26,306 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 18:13:26,328 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2d4426778148209fd63d4 HTTP/1.1" 200 None 2026-03-26 18:13:26,329 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 18:13:26,343 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad0a721a77312e2220ef1 HTTP/1.1" 200 None 2026-03-26 18:13:26,344 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 18:13:26,379 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2d4426778148209fd63d4 HTTP/1.1" 200 None 2026-03-26 18:13:26,380 - lava_callback - ERROR - Error processing callback for node 69c2d4426778148209fd63d4: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:56418 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49318 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35606 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46522 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60098 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60110 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58148 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58150 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54638 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47946 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51102 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48544 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51076 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51080 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35434 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35442 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53714 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52078 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54232 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39578 - "GET /node/693e5d72407f32424eb64b77%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.21.0.1:39590 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45548 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45560 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43308 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43316 - "GET / HTTP/1.0" 200 OK 2026-03-26 18:16:26,392 - lava_callback - INFO - Received callback for node 69c2d4ee6778148209fd63e8 INFO: 172.21.0.1:60110 - "POST /node/69c2d4ee6778148209fd63e8 HTTP/1.0" 202 Accepted 2026-03-26 18:16:26,397 - lava_callback - INFO - Starting async job submit for node 69c2d4ee6778148209fd63e8 2026-03-26 18:16:26,398 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 18:16:26,442 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2d4ee6778148209fd63e8 HTTP/1.1" 200 None 2026-03-26 18:16:26,443 - lava_callback - DEBUG - Processing node 69c2d4ee6778148209fd63e8: baseline-arm64-baylibre 2026-03-26 18:16:26,443 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2d4ee6778148209fd63e8 2026-03-26 18:16:26,444 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 18:16:26,485 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2d4ee6778148209fd63e8 HTTP/1.1" 200 None 2026-03-26 18:16:26,486 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 18:16:26,521 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad0a721a77312e2220ef1 HTTP/1.1" 200 None 2026-03-26 18:16:26,522 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 18:16:26,589 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2d4ee6778148209fd63e8 HTTP/1.1" 200 None 2026-03-26 18:16:26,591 - lava_callback - ERROR - Error processing callback for node 69c2d4ee6778148209fd63e8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:60126 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47122 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33184 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50456 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45094 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45104 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48520 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48526 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60706 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54654 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57448 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57822 - "GET / HTTP/1.0" 200 OK 2026-03-26 18:18:03,854 - lava_callback - INFO - Received callback for node 69c2d5496778148209fd63f0 INFO: 172.21.0.1:37468 - "POST /node/69c2d5496778148209fd63f0 HTTP/1.0" 202 Accepted 2026-03-26 18:18:03,859 - lava_callback - INFO - Starting async job submit for node 69c2d5496778148209fd63f0 2026-03-26 18:18:03,860 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 18:18:03,875 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2d5496778148209fd63f0 HTTP/1.1" 200 None 2026-03-26 18:18:03,875 - lava_callback - DEBUG - Processing node 69c2d5496778148209fd63f0: baseline-arm64-clabbe 2026-03-26 18:18:03,876 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2d5496778148209fd63f0 2026-03-26 18:18:03,876 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 18:18:03,889 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2d5496778148209fd63f0 HTTP/1.1" 200 None 2026-03-26 18:18:03,891 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 18:18:03,904 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad0a721a77312e2220ef1 HTTP/1.1" 200 None 2026-03-26 18:18:03,905 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 18:18:03,926 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2d5496778148209fd63f0 HTTP/1.1" 200 None 2026-03-26 18:18:03,930 - lava_callback - ERROR - Error processing callback for node 69c2d5496778148209fd63f0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:37484 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37488 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43620 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43622 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44594 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47734 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37868 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42856 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54894 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54896 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34200 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34204 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49160 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42774 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52366 - "GET / HTTP/1.0" 200 OK 2026-03-26 18:19:47,352 - lava_callback - INFO - Received callback for node 69c4f4dff63ce05331e2aed1 INFO: 172.21.0.1:52378 - "POST /node/69c4f4dff63ce05331e2aed1 HTTP/1.0" 202 Accepted 2026-03-26 18:19:47,359 - lava_callback - INFO - Starting async job submit for node 69c4f4dff63ce05331e2aed1 2026-03-26 18:19:47,384 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 18:19:47,399 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f4dff63ce05331e2aed1 HTTP/1.1" 200 None 2026-03-26 18:19:47,400 - lava_callback - DEBUG - Processing node 69c4f4dff63ce05331e2aed1: baseline-nfs-arm64 2026-03-26 18:19:47,925 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 18:19:48,219 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 18:19:48,221 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c4f4dff63ce05331e2aed1/log.txt.gz 2026-03-26 18:19:48,477 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 18:19:48,695 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 18:19:48,697 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c4f4dff63ce05331e2aed1/lava-logs.yaml 2026-03-26 18:19:48,709 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 18:19:48,919 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 18:19:48,920 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c4f4dff63ce05331e2aed1/lava_callback.json.gz 2026-03-26 18:19:48,947 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 18:19:48,964 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f4dff63ce05331e2aed1 HTTP/1.1" 200 None 2026-03-26 18:19:48,966 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 18:19:48,979 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebd8f63ce05331e2a492 HTTP/1.1" 200 None 2026-03-26 18:19:48,981 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 18:19:49,001 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4f4dff63ce05331e2aed1 HTTP/1.1" 200 None 2026-03-26 18:19:49,059 - lava_callback - INFO - Completed processing callback for node 69c4f4dff63ce05331e2aed1 INFO: 172.21.0.1:43584 - "GET / HTTP/1.0" 200 OK 2026-03-26 18:20:02,605 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 18:20:02,621 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:57772 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57788 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52976 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52978 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56432 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55566 - "GET / HTTP/1.0" 200 OK 2026-03-26 18:20:43,980 - lava_callback - INFO - Received callback for node 69c2d5f16778148209fd6409 INFO: 172.21.0.1:32902 - "POST /node/69c2d5f16778148209fd6409 HTTP/1.0" 202 Accepted 2026-03-26 18:20:43,986 - lava_callback - INFO - Starting async job submit for node 69c2d5f16778148209fd6409 2026-03-26 18:20:43,987 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 18:20:44,002 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2d5f16778148209fd6409 HTTP/1.1" 200 None 2026-03-26 18:20:44,003 - lava_callback - DEBUG - Processing node 69c2d5f16778148209fd6409: baseline-arm64-clabbe 2026-03-26 18:20:44,003 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2d5f16778148209fd6409 2026-03-26 18:20:44,004 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 18:20:44,031 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2d5f16778148209fd6409 HTTP/1.1" 200 None 2026-03-26 18:20:44,032 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 18:20:44,046 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad0a721a77312e2220ef1 HTTP/1.1" 200 None 2026-03-26 18:20:44,047 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 18:20:44,068 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2d5f16778148209fd6409 HTTP/1.1" 200 None 2026-03-26 18:20:44,074 - lava_callback - ERROR - Error processing callback for node 69c2d5f16778148209fd6409: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:32906 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42834 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58854 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58868 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58542 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58550 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58596 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52020 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46582 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51712 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56752 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56766 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42072 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42086 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47684 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40998 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53648 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57448 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52600 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52616 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57420 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57434 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52190 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44272 - "GET / HTTP/1.0" 200 OK 2026-03-26 18:23:44,121 - lava_callback - INFO - Received callback for node 69c2d69c6778148209fd6423 INFO: 172.21.0.1:56220 - "POST /node/69c2d69c6778148209fd6423 HTTP/1.0" 202 Accepted 2026-03-26 18:23:44,127 - lava_callback - INFO - Starting async job submit for node 69c2d69c6778148209fd6423 2026-03-26 18:23:44,128 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 18:23:44,142 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2d69c6778148209fd6423 HTTP/1.1" 200 None 2026-03-26 18:23:44,143 - lava_callback - DEBUG - Processing node 69c2d69c6778148209fd6423: baseline-arm64-clabbe 2026-03-26 18:23:44,143 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2d69c6778148209fd6423 2026-03-26 18:23:44,144 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 18:23:44,162 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2d69c6778148209fd6423 HTTP/1.1" 200 None 2026-03-26 18:23:44,164 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 18:23:44,181 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad0a721a77312e2220ef1 HTTP/1.1" 200 None 2026-03-26 18:23:44,183 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 18:23:44,204 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2d69c6778148209fd6423 HTTP/1.1" 200 None 2026-03-26 18:23:44,210 - lava_callback - ERROR - Error processing callback for node 69c2d69c6778148209fd6423: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:56224 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45836 - "GET / HTTP/1.0" 200 OK 2026-03-26 18:23:59,668 - lava_callback - INFO - Received callback for node 69c575a1abf66a8f4d9bc455 INFO: 172.21.0.1:45844 - "POST /node/69c575a1abf66a8f4d9bc455 HTTP/1.0" 202 Accepted 2026-03-26 18:23:59,675 - lava_callback - INFO - Starting async job submit for node 69c575a1abf66a8f4d9bc455 2026-03-26 18:23:59,731 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 18:23:59,747 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c575a1abf66a8f4d9bc455 HTTP/1.1" 200 None 2026-03-26 18:23:59,748 - lava_callback - DEBUG - Processing node 69c575a1abf66a8f4d9bc455: ltp-smoke 2026-03-26 18:24:00,250 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 18:24:00,386 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 18:24:00,387 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoke-69c575a1abf66a8f4d9bc455/log.txt.gz 2026-03-26 18:24:00,667 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 18:24:00,925 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 18:24:00,927 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-smoke-69c575a1abf66a8f4d9bc455/lava-logs.yaml 2026-03-26 18:24:00,939 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 18:24:01,080 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 18:24:01,082 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoke-69c575a1abf66a8f4d9bc455/lava_callback.json.gz 2026-03-26 18:24:01,200 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 18:24:01,215 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c575a1abf66a8f4d9bc455 HTTP/1.1" 200 None 2026-03-26 18:24:01,216 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 18:24:01,237 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c55c76abf66a8f4d9b965b HTTP/1.1" 200 None 2026-03-26 18:24:01,239 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 18:24:01,263 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c575a1abf66a8f4d9bc455 HTTP/1.1" 200 None 2026-03-26 18:24:01,369 - lava_callback - INFO - Completed processing callback for node 69c575a1abf66a8f4d9bc455 2026-03-26 18:24:02,625 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 18:24:02,646 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:33292 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33304 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33852 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33868 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35464 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51380 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47564 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53456 - "GET / HTTP/1.0" 200 OK 2026-03-26 18:25:04,316 - lava_callback - INFO - Received callback for node 69c2d6f16778148209fd6428 INFO: 172.21.0.1:56822 - "POST /node/69c2d6f16778148209fd6428 HTTP/1.0" 202 Accepted 2026-03-26 18:25:04,322 - lava_callback - INFO - Starting async job submit for node 69c2d6f16778148209fd6428 2026-03-26 18:25:04,323 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 18:25:04,338 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2d6f16778148209fd6428 HTTP/1.1" 200 None 2026-03-26 18:25:04,339 - lava_callback - DEBUG - Processing node 69c2d6f16778148209fd6428: baseline-arm64-clabbe 2026-03-26 18:25:04,339 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2d6f16778148209fd6428 2026-03-26 18:25:04,339 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 18:25:04,361 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2d6f16778148209fd6428 HTTP/1.1" 200 None 2026-03-26 18:25:04,362 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 18:25:04,376 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad0a721a77312e2220ef1 HTTP/1.1" 200 None 2026-03-26 18:25:04,378 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 18:25:04,398 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2d6f16778148209fd6428 HTTP/1.1" 200 None 2026-03-26 18:25:04,399 - lava_callback - ERROR - Error processing callback for node 69c2d6f16778148209fd6428: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:56824 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56838 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51756 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51762 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35014 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59060 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51070 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54336 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55298 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55314 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54906 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54910 - "GET / HTTP/1.0" 200 OK 2026-03-26 18:26:24,243 - lava_callback - INFO - Received callback for node 69c2d7466778148209fd6433 INFO: 172.21.0.1:41816 - "POST /node/69c2d7466778148209fd6433 HTTP/1.0" 202 Accepted 2026-03-26 18:26:24,249 - lava_callback - INFO - Starting async job submit for node 69c2d7466778148209fd6433 2026-03-26 18:26:24,250 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 18:26:24,282 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2d7466778148209fd6433 HTTP/1.1" 200 None 2026-03-26 18:26:24,283 - lava_callback - DEBUG - Processing node 69c2d7466778148209fd6433: baseline-arm64-clabbe 2026-03-26 18:26:24,284 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2d7466778148209fd6433 2026-03-26 18:26:24,284 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 18:26:24,305 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2d7466778148209fd6433 HTTP/1.1" 200 None 2026-03-26 18:26:24,307 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 18:26:24,346 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad0a721a77312e2220ef1 HTTP/1.1" 200 None 2026-03-26 18:26:24,348 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 18:26:24,368 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2d7466778148209fd6433 HTTP/1.1" 200 None 2026-03-26 18:26:24,376 - lava_callback - ERROR - Error processing callback for node 69c2d7466778148209fd6433: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:41826 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44750 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38268 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60864 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59642 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59644 - "GET / HTTP/1.0" 200 OK 2026-03-26 18:27:14,316 - lava_callback - INFO - Received callback for node 69c57653abf66a8f4d9bc479 INFO: 172.21.0.1:59918 - "POST /node/69c57653abf66a8f4d9bc479 HTTP/1.0" 202 Accepted 2026-03-26 18:27:14,322 - lava_callback - INFO - Starting async job submit for node 69c57653abf66a8f4d9bc479 2026-03-26 18:27:14,375 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 18:27:14,390 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c57653abf66a8f4d9bc479 HTTP/1.1" 200 None 2026-03-26 18:27:14,391 - lava_callback - DEBUG - Processing node 69c57653abf66a8f4d9bc479: baseline-arm 2026-03-26 18:27:14,976 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.21.0.1:59924 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-26 18:27:15,205 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 18:27:15,207 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69c57653abf66a8f4d9bc479/log.txt.gz 2026-03-26 18:27:15,546 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 18:27:15,793 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 18:27:15,795 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c57653abf66a8f4d9bc479/lava-logs.yaml 2026-03-26 18:27:15,809 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 18:27:15,962 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 18:27:15,963 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c57653abf66a8f4d9bc479/lava_callback.json.gz DEBUG: Setup failed for 69c57653abf66a8f4d9bc479. Transitting job node result: incomplete -> fail 2026-03-26 18:27:16,073 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 18:27:16,092 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c57653abf66a8f4d9bc479 HTTP/1.1" 200 None 2026-03-26 18:27:16,093 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 18:27:16,108 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c55c70abf66a8f4d9b9655 HTTP/1.1" 200 None 2026-03-26 18:27:16,109 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 18:27:16,134 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c57653abf66a8f4d9bc479 HTTP/1.1" 200 None 2026-03-26 18:27:16,261 - lava_callback - INFO - Completed processing callback for node 69c57653abf66a8f4d9bc479 INFO: 172.21.0.1:59926 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45036 - "GET / HTTP/1.0" 200 OK 2026-03-26 18:27:32,649 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 18:27:32,683 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:59294 - "GET / HTTP/1.0" 200 OK 2026-03-26 18:27:44,314 - lava_callback - INFO - Received callback for node 69c2d7986778148209fd644a INFO: 172.21.0.1:46300 - "POST /node/69c2d7986778148209fd644a HTTP/1.0" 202 Accepted 2026-03-26 18:27:44,319 - lava_callback - INFO - Starting async job submit for node 69c2d7986778148209fd644a 2026-03-26 18:27:44,320 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 18:27:44,343 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2d7986778148209fd644a HTTP/1.1" 200 None 2026-03-26 18:27:44,344 - lava_callback - DEBUG - Processing node 69c2d7986778148209fd644a: baseline-arm64-clabbe 2026-03-26 18:27:44,344 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2d7986778148209fd644a 2026-03-26 18:27:44,345 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 18:27:44,358 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2d7986778148209fd644a HTTP/1.1" 200 None 2026-03-26 18:27:44,359 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 18:27:44,371 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad0a721a77312e2220ef1 HTTP/1.1" 200 None 2026-03-26 18:27:44,373 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 18:27:44,393 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2d7986778148209fd644a HTTP/1.1" 200 None 2026-03-26 18:27:44,400 - lava_callback - ERROR - Error processing callback for node 69c2d7986778148209fd644a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:46314 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40946 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41906 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41912 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32778 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:32792 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51690 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36472 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41864 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34888 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35434 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35442 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43116 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43124 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50522 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39554 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46978 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33432 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37070 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37072 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41504 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41520 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43272 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60366 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54602 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43898 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47596 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47600 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58174 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58188 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32910 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34686 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56024 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40070 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40996 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41008 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55806 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55822 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60984 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51682 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51348 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48576 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35478 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35480 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57648 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57650 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39682 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52892 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39582 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42498 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47228 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47234 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36762 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36776 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52604 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46516 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36642 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57380 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49200 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49216 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48802 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48806 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41694 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45390 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34150 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39356 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49542 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49556 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42840 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42856 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58430 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60508 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59434 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52996 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35388 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35402 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47910 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47920 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46652 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50164 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50180 - "GET /node/699371ffcebe99dd8e069721%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.21.0.1:55712 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48392 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38882 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38886 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33298 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33304 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60238 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58116 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47620 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52268 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44514 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44528 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60758 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60768 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37648 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52912 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41848 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45284 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52556 - "GET /node/699190dd10b45da7d7d11519%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.21.0.1:52562 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52568 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45040 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45050 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50426 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39712 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39540 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57520 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36410 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36416 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47132 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47148 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50518 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50610 - "GET / HTTP/1.0" 200 OK 2026-03-26 18:41:40,221 - lava_callback - INFO - Received callback for node 69c579c2abf66a8f4d9bc4c2 INFO: 172.21.0.1:48344 - "POST /node/69c579c2abf66a8f4d9bc4c2 HTTP/1.0" 202 Accepted 2026-03-26 18:41:40,228 - lava_callback - INFO - Starting async job submit for node 69c579c2abf66a8f4d9bc4c2 2026-03-26 18:41:40,286 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 18:41:40,300 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c579c2abf66a8f4d9bc4c2 HTTP/1.1" 200 None 2026-03-26 18:41:40,301 - lava_callback - DEBUG - Processing node 69c579c2abf66a8f4d9bc4c2: ltp-smoke 2026-03-26 18:41:40,794 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 18:41:41,111 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 18:41:41,113 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoke-69c579c2abf66a8f4d9bc4c2/log.txt.gz 2026-03-26 18:41:41,389 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 18:41:41,597 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 18:41:41,599 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-smoke-69c579c2abf66a8f4d9bc4c2/lava-logs.yaml 2026-03-26 18:41:41,611 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 18:41:41,790 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 18:41:41,792 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoke-69c579c2abf66a8f4d9bc4c2/lava_callback.json.gz 2026-03-26 18:41:41,910 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 18:41:41,961 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c579c2abf66a8f4d9bc4c2 HTTP/1.1" 200 None 2026-03-26 18:41:41,963 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 18:41:41,976 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c55c76abf66a8f4d9b965b HTTP/1.1" 200 None 2026-03-26 18:41:41,978 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 18:41:42,040 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c579c2abf66a8f4d9bc4c2 HTTP/1.1" 200 None 2026-03-26 18:41:42,159 - lava_callback - INFO - Completed processing callback for node 69c579c2abf66a8f4d9bc4c2 INFO: 172.21.0.1:48356 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54094 - "GET / HTTP/1.0" 200 OK 2026-03-26 18:42:02,687 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 18:42:02,880 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:55262 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55272 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54746 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54762 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55278 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40876 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37886 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50444 - "GET /node/693f83a50d8ce4018da96aab%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.21.0.1:50448 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37498 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37502 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54774 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54778 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44522 - "GET / HTTP/1.0" 200 OK 2026-03-26 18:43:28,309 - lava_callback - INFO - Received callback for node 69c56de2abf66a8f4d9bc2b9 INFO: 172.21.0.1:44526 - "POST /node/69c56de2abf66a8f4d9bc2b9 HTTP/1.0" 202 Accepted 2026-03-26 18:43:28,357 - lava_callback - INFO - Starting async job submit for node 69c56de2abf66a8f4d9bc2b9 2026-03-26 18:43:28,381 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 18:43:28,407 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c56de2abf66a8f4d9bc2b9 HTTP/1.1" 200 None 2026-03-26 18:43:28,408 - lava_callback - DEBUG - Processing node 69c56de2abf66a8f4d9bc2b9: baseline-arm-pengutronix 2026-03-26 18:43:28,514 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 18:43:28,848 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 18:43:28,850 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c56de2abf66a8f4d9bc2b9/log.txt.gz 2026-03-26 18:43:28,909 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 18:43:29,088 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 18:43:29,090 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c56de2abf66a8f4d9bc2b9/lava-logs.yaml 2026-03-26 18:43:29,093 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 18:43:29,230 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 18:43:29,232 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c56de2abf66a8f4d9bc2b9/lava_callback.json.gz DEBUG: Setup failed for 69c56de2abf66a8f4d9bc2b9. Transitting job node result: incomplete -> fail 2026-03-26 18:43:29,296 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 18:43:29,311 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c56de2abf66a8f4d9bc2b9 HTTP/1.1" 200 None 2026-03-26 18:43:29,312 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 18:43:29,326 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c55c70abf66a8f4d9b9655 HTTP/1.1" 200 None 2026-03-26 18:43:29,328 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 18:43:29,351 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c56de2abf66a8f4d9bc2b9 HTTP/1.1" 200 None 2026-03-26 18:43:29,400 - lava_callback - INFO - Completed processing callback for node 69c56de2abf66a8f4d9bc2b9 2026-03-26 18:43:32,882 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 18:43:32,898 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:42078 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44580 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43822 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39338 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39348 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47002 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47006 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60696 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52066 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39762 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40988 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58256 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58258 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54216 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54224 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53708 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60242 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60198 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51450 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37306 - "GET /node/69936463cebe99dd8e0644cb%5Cn HTTP/1.0" 405 Method Not Allowed 2026-03-26 18:46:05,186 - lava_callback - INFO - Received callback for node 69c2dbd76778148209fd64d5 INFO: 172.21.0.1:37322 - "POST /node/69c2dbd76778148209fd64d5 HTTP/1.0" 202 Accepted 2026-03-26 18:46:05,191 - lava_callback - INFO - Starting async job submit for node 69c2dbd76778148209fd64d5 2026-03-26 18:46:05,192 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 18:46:05,228 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2dbd76778148209fd64d5 HTTP/1.1" 200 None 2026-03-26 18:46:05,229 - lava_callback - DEBUG - Processing node 69c2dbd76778148209fd64d5: baseline-arm-clabbe 2026-03-26 18:46:05,229 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2dbd76778148209fd64d5 2026-03-26 18:46:05,230 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 18:46:05,243 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2dbd76778148209fd64d5 HTTP/1.1" 200 None 2026-03-26 18:46:05,244 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 18:46:05,270 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d82b0efaf21d93103543 HTTP/1.1" 200 None 2026-03-26 18:46:05,271 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 18:46:05,292 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2dbd76778148209fd64d5 HTTP/1.1" 200 None 2026-03-26 18:46:05,293 - lava_callback - ERROR - Error processing callback for node 69c2dbd76778148209fd64d5: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:37334 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37346 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58330 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58336 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57758 - "GET /node/6991f8b601ff1e8e0859db43%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.21.0.1:57760 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42268 - "GET / HTTP/1.0" 200 OK 2026-03-26 18:46:45,125 - lava_callback - INFO - Received callback for node 69c2dc026778148209fd64e3 INFO: 172.21.0.1:49084 - "POST /node/69c2dc026778148209fd64e3 HTTP/1.0" 202 Accepted 2026-03-26 18:46:45,135 - lava_callback - INFO - Starting async job submit for node 69c2dc026778148209fd64e3 2026-03-26 18:46:45,137 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 18:46:45,197 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2dc026778148209fd64e3 HTTP/1.1" 200 None 2026-03-26 18:46:45,199 - lava_callback - DEBUG - Processing node 69c2dc026778148209fd64e3: baseline-arm-clabbe 2026-03-26 18:46:45,199 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2dc026778148209fd64e3 2026-03-26 18:46:45,200 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 18:46:45,217 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2dc026778148209fd64e3 HTTP/1.1" 200 None 2026-03-26 18:46:45,219 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 18:46:45,234 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d82b0efaf21d93103543 HTTP/1.1" 200 None 2026-03-26 18:46:45,235 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 18:46:45,279 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2dc026778148209fd64e3 HTTP/1.1" 200 None 2026-03-26 18:46:45,280 - lava_callback - ERROR - Error processing callback for node 69c2dc026778148209fd64e3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:49096 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58512 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36502 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36508 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48614 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48622 - "GET / HTTP/1.0" 200 OK 2026-03-26 18:47:25,156 - lava_callback - INFO - Received callback for node 69c2dc2d6778148209fd64ed INFO: 172.21.0.1:37532 - "POST /node/69c2dc2d6778148209fd64ed HTTP/1.0" 202 Accepted 2026-03-26 18:47:25,161 - lava_callback - INFO - Starting async job submit for node 69c2dc2d6778148209fd64ed 2026-03-26 18:47:25,162 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 18:47:25,186 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2dc2d6778148209fd64ed HTTP/1.1" 200 None 2026-03-26 18:47:25,187 - lava_callback - DEBUG - Processing node 69c2dc2d6778148209fd64ed: baseline-arm-clabbe 2026-03-26 18:47:25,187 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2dc2d6778148209fd64ed 2026-03-26 18:47:25,188 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 18:47:25,211 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2dc2d6778148209fd64ed HTTP/1.1" 200 None 2026-03-26 18:47:25,213 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 18:47:25,226 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d82b0efaf21d93103543 HTTP/1.1" 200 None 2026-03-26 18:47:25,228 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 18:47:25,291 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2dc2d6778148209fd64ed HTTP/1.1" 200 None 2026-03-26 18:47:25,292 - lava_callback - ERROR - Error processing callback for node 69c2dc2d6778148209fd64ed: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:37536 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50486 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57574 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42786 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43860 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43874 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33290 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33306 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39356 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53532 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39538 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47388 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39870 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39884 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57114 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57120 - "GET / HTTP/1.0" 200 OK 2026-03-26 18:49:25,270 - lava_callback - INFO - Received callback for node 69c2dcaa6778148209fd650f INFO: 172.21.0.1:44114 - "POST /node/69c2dcaa6778148209fd650f HTTP/1.0" 202 Accepted 2026-03-26 18:49:25,275 - lava_callback - INFO - Starting async job submit for node 69c2dcaa6778148209fd650f 2026-03-26 18:49:25,276 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 18:49:25,291 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2dcaa6778148209fd650f HTTP/1.1" 200 None 2026-03-26 18:49:25,292 - lava_callback - DEBUG - Processing node 69c2dcaa6778148209fd650f: baseline-arm-clabbe 2026-03-26 18:49:25,292 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2dcaa6778148209fd650f 2026-03-26 18:49:25,292 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 18:49:25,309 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2dcaa6778148209fd650f HTTP/1.1" 200 None 2026-03-26 18:49:25,310 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 18:49:25,325 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d82b0efaf21d93103543 HTTP/1.1" 200 None 2026-03-26 18:49:25,326 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 18:49:25,347 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2dcaa6778148209fd650f HTTP/1.1" 200 None 2026-03-26 18:49:25,353 - lava_callback - ERROR - Error processing callback for node 69c2dcaa6778148209fd650f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:44128 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45756 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47984 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52344 - "GET / HTTP/1.0" 200 OK 2026-03-26 18:50:05,451 - lava_callback - INFO - Received callback for node 69c2dcd46778148209fd651c INFO: 172.21.0.1:50868 - "POST /node/69c2dcd46778148209fd651c HTTP/1.0" 202 Accepted 2026-03-26 18:50:05,457 - lava_callback - INFO - Starting async job submit for node 69c2dcd46778148209fd651c 2026-03-26 18:50:05,458 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 18:50:05,472 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2dcd46778148209fd651c HTTP/1.1" 200 None 2026-03-26 18:50:05,473 - lava_callback - DEBUG - Processing node 69c2dcd46778148209fd651c: baseline-arm-clabbe 2026-03-26 18:50:05,473 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2dcd46778148209fd651c 2026-03-26 18:50:05,474 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 18:50:05,523 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2dcd46778148209fd651c HTTP/1.1" 200 None 2026-03-26 18:50:05,525 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 18:50:05,541 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d82b0efaf21d93103543 HTTP/1.1" 200 None 2026-03-26 18:50:05,543 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 18:50:05,563 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2dcd46778148209fd651c HTTP/1.1" 200 None 2026-03-26 18:50:05,564 - lava_callback - ERROR - Error processing callback for node 69c2dcd46778148209fd651c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:50884 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50892 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55100 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55116 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57916 - "GET /node/69bca2a3b4744551b27faa35%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.21.0.1:57926 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34432 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59038 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52524 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56568 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56570 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45804 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45820 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38938 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38440 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41036 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42026 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47092 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47102 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46886 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46890 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45294 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44010 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41350 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41296 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41802 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41816 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60384 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60394 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33506 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59932 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37974 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33048 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47308 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47324 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40074 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40088 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44542 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56846 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60752 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41894 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49066 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49080 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37862 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37872 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54660 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41898 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37306 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59822 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40076 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40092 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39606 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39616 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49484 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39820 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40386 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44568 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56686 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56694 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51172 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51188 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42978 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57934 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45586 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45248 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43394 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43396 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35542 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35558 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40458 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34266 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41782 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39560 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36352 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36358 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55186 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55198 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40916 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45162 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43266 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57338 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35376 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35380 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55118 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55124 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46528 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55076 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58226 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58070 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37268 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37282 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49702 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49718 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44836 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39534 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60398 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33662 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45064 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45070 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40568 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40572 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46140 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48290 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41360 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37470 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49434 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49448 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44092 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44100 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55506 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46496 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36432 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60484 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52656 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52664 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55358 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55374 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51120 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47276 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42894 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53846 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37988 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38002 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51674 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51680 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60492 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38050 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46406 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53614 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41332 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41340 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42702 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42710 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36736 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38714 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40894 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54018 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57350 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57358 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40678 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40684 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39686 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35176 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60660 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51644 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51210 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51218 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34538 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34542 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58836 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47276 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47282 - "GET /node/69876b99e31b58f834f19841%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.21.0.1:38900 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51754 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33170 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33176 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43846 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43852 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41164 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36504 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33534 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53522 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46100 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46110 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36724 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36730 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38684 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45532 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33882 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50736 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59122 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59126 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51684 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51696 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52610 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47122 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35102 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44736 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59548 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59550 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53510 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53512 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34482 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35100 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55820 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39352 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49216 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49224 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60628 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60630 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33258 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48224 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46632 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55250 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34010 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34012 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55178 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55180 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54876 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43412 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60434 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40796 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40610 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40612 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41894 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41904 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49998 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43056 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41214 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56852 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56442 - "GET /node/69a4e11f7caa0b8e1b59dcfc%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.21.0.1:56458 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56468 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38026 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38028 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36554 - "GET / HTTP/1.0" 200 OK 2026-03-26 19:16:30,037 - lava_callback - INFO - Received callback for node 69c2e3036778148209fd65eb INFO: 172.21.0.1:58046 - "POST /node/69c2e3036778148209fd65eb HTTP/1.0" 202 Accepted 2026-03-26 19:16:30,042 - lava_callback - INFO - Starting async job submit for node 69c2e3036778148209fd65eb 2026-03-26 19:16:30,043 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 19:16:30,062 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2e3036778148209fd65eb HTTP/1.1" 200 None 2026-03-26 19:16:30,063 - lava_callback - DEBUG - Processing node 69c2e3036778148209fd65eb: baseline-arm64-baylibre 2026-03-26 19:16:30,063 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2e3036778148209fd65eb 2026-03-26 19:16:30,064 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 19:16:30,081 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2e3036778148209fd65eb HTTP/1.1" 200 None 2026-03-26 19:16:30,083 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 19:16:30,098 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-26 19:16:30,100 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 19:16:30,122 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2e3036778148209fd65eb HTTP/1.1" 200 None 2026-03-26 19:16:30,129 - lava_callback - ERROR - Error processing callback for node 69c2e3036778148209fd65eb: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:58050 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54418 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52016 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49762 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49776 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38622 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38634 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54924 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33092 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39210 - "GET / HTTP/1.0" 200 OK 2026-03-26 19:17:49,700 - lava_callback - INFO - Received callback for node 69c2e3566778148209fd65f7 INFO: 172.21.0.1:33848 - "POST /node/69c2e3566778148209fd65f7 HTTP/1.0" 202 Accepted 2026-03-26 19:17:49,705 - lava_callback - INFO - Starting async job submit for node 69c2e3566778148209fd65f7 2026-03-26 19:17:49,706 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 19:17:49,731 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2e3566778148209fd65f7 HTTP/1.1" 200 None 2026-03-26 19:17:49,732 - lava_callback - DEBUG - Processing node 69c2e3566778148209fd65f7: baseline-arm64-baylibre 2026-03-26 19:17:49,732 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2e3566778148209fd65f7 2026-03-26 19:17:49,732 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 19:17:49,748 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2e3566778148209fd65f7 HTTP/1.1" 200 None 2026-03-26 19:17:49,750 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 19:17:49,767 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-26 19:17:49,768 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 19:17:49,789 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2e3566778148209fd65f7 HTTP/1.1" 200 None 2026-03-26 19:17:49,794 - lava_callback - ERROR - Error processing callback for node 69c2e3566778148209fd65f7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:33854 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48112 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48124 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52806 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52810 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49718 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57518 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57174 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38626 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56110 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56114 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35746 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35750 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47070 - "GET / HTTP/1.0" 200 OK 2026-03-26 19:19:29,915 - lava_callback - INFO - Received callback for node 69c2e3b86778148209fd6602 INFO: 172.21.0.1:40774 - "POST /node/69c2e3b86778148209fd6602 HTTP/1.0" 202 Accepted 2026-03-26 19:19:29,921 - lava_callback - INFO - Starting async job submit for node 69c2e3b86778148209fd6602 2026-03-26 19:19:29,922 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 19:19:29,937 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2e3b86778148209fd6602 HTTP/1.1" 200 None 2026-03-26 19:19:29,938 - lava_callback - DEBUG - Processing node 69c2e3b86778148209fd6602: baseline-arm64-baylibre 2026-03-26 19:19:29,938 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2e3b86778148209fd6602 2026-03-26 19:19:29,939 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 19:19:29,953 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2e3b86778148209fd6602 HTTP/1.1" 200 None 2026-03-26 19:19:29,954 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 19:19:29,967 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-26 19:19:29,969 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 19:19:29,990 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2e3b86778148209fd6602 HTTP/1.1" 200 None 2026-03-26 19:19:29,991 - lava_callback - ERROR - Error processing callback for node 69c2e3b86778148209fd6602: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:40784 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33596 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49774 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58814 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58820 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35064 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35068 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47556 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47474 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54168 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40146 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46282 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46296 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57272 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57280 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52862 - "GET / HTTP/1.0" 200 OK 2026-03-26 19:21:29,988 - lava_callback - INFO - Received callback for node 69c2e4296778148209fd662c INFO: 172.21.0.1:57158 - "POST /node/69c2e4296778148209fd662c HTTP/1.0" 202 Accepted 2026-03-26 19:21:29,994 - lava_callback - INFO - Starting async job submit for node 69c2e4296778148209fd662c 2026-03-26 19:21:29,995 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 19:21:30,012 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2e4296778148209fd662c HTTP/1.1" 200 None 2026-03-26 19:21:30,013 - lava_callback - DEBUG - Processing node 69c2e4296778148209fd662c: baseline-arm64-baylibre 2026-03-26 19:21:30,013 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2e4296778148209fd662c 2026-03-26 19:21:30,013 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 19:21:30,036 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2e4296778148209fd662c HTTP/1.1" 200 None 2026-03-26 19:21:30,037 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 19:21:30,050 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-26 19:21:30,052 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 19:21:30,072 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2e4296778148209fd662c HTTP/1.1" 200 None 2026-03-26 19:21:30,075 - lava_callback - ERROR - Error processing callback for node 69c2e4296778148209fd662c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:57162 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51352 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43908 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38732 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38748 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36916 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36928 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50680 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33104 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43396 - "GET / HTTP/1.0" 200 OK 2026-03-26 19:22:49,557 - lava_callback - INFO - Received callback for node 69c57e5fabf66a8f4d9bc4e6 INFO: 172.21.0.1:43402 - "POST /node/69c57e5fabf66a8f4d9bc4e6 HTTP/1.0" 202 Accepted 2026-03-26 19:22:49,605 - lava_callback - INFO - Starting async job submit for node 69c57e5fabf66a8f4d9bc4e6 2026-03-26 19:22:49,629 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 19:22:49,643 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c57e5fabf66a8f4d9bc4e6 HTTP/1.1" 200 None 2026-03-26 19:22:49,644 - lava_callback - DEBUG - Processing node 69c57e5fabf66a8f4d9bc4e6: baseline-arm-pengutronix 2026-03-26 19:22:49,753 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 19:22:49,902 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 19:22:49,904 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c57e5fabf66a8f4d9bc4e6/log.txt.gz 2026-03-26 19:22:49,963 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 19:22:50,116 - lava_callback - INFO - Received callback for node 69c2e47c6778148209fd6635 INFO: 172.21.0.1:58160 - "POST /node/69c2e47c6778148209fd6635 HTTP/1.0" 202 Accepted 2026-03-26 19:22:50,124 - lava_callback - INFO - Starting async job submit for node 69c2e47c6778148209fd6635 2026-03-26 19:22:50,125 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 19:22:50,139 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2e47c6778148209fd6635 HTTP/1.1" 200 None 2026-03-26 19:22:50,140 - lava_callback - DEBUG - Processing node 69c2e47c6778148209fd6635: baseline-arm64-baylibre 2026-03-26 19:22:50,140 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2e47c6778148209fd6635 2026-03-26 19:22:50,141 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 19:22:50,155 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2e47c6778148209fd6635 HTTP/1.1" 200 None 2026-03-26 19:22:50,157 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 19:22:50,176 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-26 19:22:50,178 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 19:22:50,211 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2e47c6778148209fd6635 HTTP/1.1" 200 None 2026-03-26 19:22:50,212 - lava_callback - ERROR - Error processing callback for node 69c2e47c6778148209fd6635: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-26 19:22:50,364 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 19:22:50,366 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c57e5fabf66a8f4d9bc4e6/lava-logs.yaml 2026-03-26 19:22:50,369 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 19:22:50,508 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 19:22:50,509 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c57e5fabf66a8f4d9bc4e6/lava_callback.json.gz DEBUG: Setup failed for 69c57e5fabf66a8f4d9bc4e6. Transitting job node result: incomplete -> fail 2026-03-26 19:22:50,566 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 19:22:50,579 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c57e5fabf66a8f4d9bc4e6 HTTP/1.1" 200 None 2026-03-26 19:22:50,581 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 19:22:50,594 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c55c70abf66a8f4d9b9655 HTTP/1.1" 200 None 2026-03-26 19:22:50,596 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 19:22:50,624 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c57e5fabf66a8f4d9bc4e6 HTTP/1.1" 200 None 2026-03-26 19:22:50,676 - lava_callback - INFO - Completed processing callback for node 69c57e5fabf66a8f4d9bc4e6 INFO: 172.21.0.1:58174 - "GET / HTTP/1.0" 200 OK 2026-03-26 19:23:02,908 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 19:23:02,925 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:50108 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50124 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50544 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50546 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47216 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50782 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51460 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40762 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50348 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50362 - "GET / HTTP/1.0" 200 OK 2026-03-26 19:24:10,041 - lava_callback - INFO - Received callback for node 69c2e4cf6778148209fd6642 INFO: 172.21.0.1:44032 - "POST /node/69c2e4cf6778148209fd6642 HTTP/1.0" 202 Accepted 2026-03-26 19:24:10,046 - lava_callback - INFO - Starting async job submit for node 69c2e4cf6778148209fd6642 2026-03-26 19:24:10,047 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 19:24:10,062 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2e4cf6778148209fd6642 HTTP/1.1" 200 None 2026-03-26 19:24:10,063 - lava_callback - DEBUG - Processing node 69c2e4cf6778148209fd6642: baseline-arm64-baylibre 2026-03-26 19:24:10,063 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2e4cf6778148209fd6642 2026-03-26 19:24:10,063 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 19:24:10,077 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2e4cf6778148209fd6642 HTTP/1.1" 200 None 2026-03-26 19:24:10,079 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 19:24:10,092 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-26 19:24:10,093 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 19:24:10,142 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2e4cf6778148209fd6642 HTTP/1.1" 200 None 2026-03-26 19:24:10,143 - lava_callback - ERROR - Error processing callback for node 69c2e4cf6778148209fd6642: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:44034 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44042 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60138 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51438 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43594 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41318 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37824 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37832 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60776 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60780 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34940 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50076 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47256 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52678 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57518 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57532 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56382 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56386 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39486 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38594 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60444 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34148 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43916 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43930 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33264 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33280 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33454 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55462 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55186 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48898 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39842 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39850 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43242 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43244 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51476 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59270 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37902 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43766 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34498 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34502 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43498 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43508 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43744 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47054 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42134 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46892 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58204 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58206 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38882 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38896 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34252 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56450 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47588 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50774 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52490 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52494 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50568 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50584 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59758 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35890 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37544 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58822 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33296 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33298 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51092 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51104 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58794 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46922 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34258 - "GET / HTTP/1.0" 200 OK 2026-03-26 19:32:47,265 - lava_callback - INFO - Received callback for node 69c2e6cf6778148209fd666a INFO: 172.21.0.1:34270 - "POST /node/69c2e6cf6778148209fd666a HTTP/1.0" 202 Accepted 2026-03-26 19:32:47,275 - lava_callback - INFO - Starting async job submit for node 69c2e6cf6778148209fd666a 2026-03-26 19:32:47,277 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 19:32:47,299 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2e6cf6778148209fd666a HTTP/1.1" 200 None 2026-03-26 19:32:47,300 - lava_callback - DEBUG - Processing node 69c2e6cf6778148209fd666a: baseline-arm64-clabbe 2026-03-26 19:32:47,301 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2e6cf6778148209fd666a 2026-03-26 19:32:47,302 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 19:32:47,321 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2e6cf6778148209fd666a HTTP/1.1" 200 None 2026-03-26 19:32:47,322 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 19:32:47,343 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-26 19:32:47,345 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 19:32:47,367 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2e6cf6778148209fd666a HTTP/1.1" 200 None 2026-03-26 19:32:47,373 - lava_callback - ERROR - Error processing callback for node 69c2e6cf6778148209fd666a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:59344 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60658 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60662 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54736 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54744 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51534 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37044 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33972 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47952 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34952 - "GET / HTTP/1.0" 200 OK 2026-03-26 19:34:07,261 - lava_callback - INFO - Received callback for node 69c2e7236778148209fd6678 INFO: 172.21.0.1:34960 - "POST /node/69c2e7236778148209fd6678 HTTP/1.0" 202 Accepted 2026-03-26 19:34:07,267 - lava_callback - INFO - Starting async job submit for node 69c2e7236778148209fd6678 2026-03-26 19:34:07,268 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 19:34:07,301 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2e7236778148209fd6678 HTTP/1.1" 200 None 2026-03-26 19:34:07,302 - lava_callback - DEBUG - Processing node 69c2e7236778148209fd6678: baseline-arm64-clabbe 2026-03-26 19:34:07,302 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2e7236778148209fd6678 2026-03-26 19:34:07,302 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 19:34:07,315 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2e7236778148209fd6678 HTTP/1.1" 200 None 2026-03-26 19:34:07,317 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 19:34:07,333 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-26 19:34:07,335 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 19:34:07,362 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2e7236778148209fd6678 HTTP/1.1" 200 None 2026-03-26 19:34:07,367 - lava_callback - ERROR - Error processing callback for node 69c2e7236778148209fd6678: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:34974 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38816 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38824 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33500 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49060 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38074 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55130 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47394 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47406 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60528 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60542 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37034 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33530 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41684 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50742 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56322 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56336 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60288 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60292 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39818 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46466 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48038 - "GET / HTTP/1.0" 200 OK 2026-03-26 19:36:47,374 - lava_callback - INFO - Received callback for node 69c2e7be6778148209fd6686 INFO: 172.21.0.1:48050 - "POST /node/69c2e7be6778148209fd6686 HTTP/1.0" 202 Accepted 2026-03-26 19:36:47,380 - lava_callback - INFO - Starting async job submit for node 69c2e7be6778148209fd6686 2026-03-26 19:36:47,381 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 19:36:47,396 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2e7be6778148209fd6686 HTTP/1.1" 200 None 2026-03-26 19:36:47,396 - lava_callback - DEBUG - Processing node 69c2e7be6778148209fd6686: baseline-arm64-clabbe 2026-03-26 19:36:47,397 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2e7be6778148209fd6686 2026-03-26 19:36:47,397 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 19:36:47,436 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2e7be6778148209fd6686 HTTP/1.1" 200 None 2026-03-26 19:36:47,438 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 19:36:47,460 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-26 19:36:47,462 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 19:36:47,494 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2e7be6778148209fd6686 HTTP/1.1" 200 None 2026-03-26 19:36:47,501 - lava_callback - ERROR - Error processing callback for node 69c2e7be6778148209fd6686: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:60062 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58564 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58576 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35888 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35896 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54354 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42104 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56714 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34772 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38222 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38230 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58716 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58730 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48816 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38818 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58518 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46238 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56638 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56646 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48184 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48188 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46536 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53556 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51420 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59614 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46064 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46066 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51824 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51830 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59696 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52780 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52780 - "GET / HTTP/1.0" 200 OK 2026-03-26 19:40:47,547 - lava_callback - INFO - Received callback for node 69c2e8b16778148209fd6696 INFO: 172.21.0.1:52788 - "POST /node/69c2e8b16778148209fd6696 HTTP/1.0" 202 Accepted 2026-03-26 19:40:47,552 - lava_callback - INFO - Starting async job submit for node 69c2e8b16778148209fd6696 2026-03-26 19:40:47,553 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 19:40:47,568 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2e8b16778148209fd6696 HTTP/1.1" 200 None 2026-03-26 19:40:47,569 - lava_callback - DEBUG - Processing node 69c2e8b16778148209fd6696: baseline-arm64-clabbe 2026-03-26 19:40:47,569 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2e8b16778148209fd6696 2026-03-26 19:40:47,569 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 19:40:47,583 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2e8b16778148209fd6696 HTTP/1.1" 200 None 2026-03-26 19:40:47,584 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 19:40:47,597 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-26 19:40:47,598 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 19:40:47,624 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2e8b16778148209fd6696 HTTP/1.1" 200 None 2026-03-26 19:40:47,625 - lava_callback - ERROR - Error processing callback for node 69c2e8b16778148209fd6696: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:48682 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47242 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47256 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46476 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46488 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33218 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32772 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57364 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59642 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45570 - "GET / HTTP/1.0" 200 OK 2026-03-26 19:42:07,646 - lava_callback - INFO - Received callback for node 69c2e9046778148209fd66a3 INFO: 172.21.0.1:45580 - "POST /node/69c2e9046778148209fd66a3 HTTP/1.0" 202 Accepted 2026-03-26 19:42:07,652 - lava_callback - INFO - Starting async job submit for node 69c2e9046778148209fd66a3 2026-03-26 19:42:07,653 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 19:42:07,667 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2e9046778148209fd66a3 HTTP/1.1" 200 None 2026-03-26 19:42:07,668 - lava_callback - DEBUG - Processing node 69c2e9046778148209fd66a3: baseline-arm64-clabbe 2026-03-26 19:42:07,668 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2e9046778148209fd66a3 2026-03-26 19:42:07,669 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 19:42:07,708 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2e9046778148209fd66a3 HTTP/1.1" 200 None 2026-03-26 19:42:07,709 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 19:42:07,723 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-26 19:42:07,724 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 19:42:07,758 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2e9046778148209fd66a3 HTTP/1.1" 200 None 2026-03-26 19:42:07,764 - lava_callback - ERROR - Error processing callback for node 69c2e9046778148209fd66a3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:45594 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40044 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40058 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32914 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53734 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55378 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33816 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37356 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37360 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36698 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36714 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55928 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41774 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39264 - "GET / HTTP/1.0" 200 OK 2026-03-26 19:43:47,665 - lava_callback - INFO - Received callback for node 69c2e9636778148209fd66ae INFO: 172.21.0.1:39270 - "POST /node/69c2e9636778148209fd66ae HTTP/1.0" 202 Accepted 2026-03-26 19:43:47,670 - lava_callback - INFO - Starting async job submit for node 69c2e9636778148209fd66ae 2026-03-26 19:43:47,671 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 19:43:47,686 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2e9636778148209fd66ae HTTP/1.1" 200 None 2026-03-26 19:43:47,687 - lava_callback - DEBUG - Processing node 69c2e9636778148209fd66ae: baseline-arm64-clabbe 2026-03-26 19:43:47,687 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2e9636778148209fd66ae 2026-03-26 19:43:47,687 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 19:43:47,701 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2e9636778148209fd66ae HTTP/1.1" 200 None 2026-03-26 19:43:47,702 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 19:43:47,720 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-26 19:43:47,722 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 19:43:47,744 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2e9636778148209fd66ae HTTP/1.1" 200 None 2026-03-26 19:43:47,751 - lava_callback - ERROR - Error processing callback for node 69c2e9636778148209fd66ae: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:41294 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34032 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34044 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59290 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59296 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44438 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49110 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48176 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36254 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50642 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50658 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60938 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60948 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54248 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56924 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48078 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53548 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59530 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59542 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38318 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38322 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43018 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35410 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44556 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51114 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33486 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33492 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56654 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56658 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38978 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39078 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40198 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38552 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34008 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34020 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48988 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48994 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49006 - "GET /node/6983eea0969566dcb0d2a10c%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.21.0.1:44310 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58652 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58088 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35670 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53716 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53718 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46826 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46828 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43292 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36308 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43470 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53894 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33246 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33250 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48606 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48622 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54244 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53810 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37520 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36420 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33120 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33126 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48166 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48182 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59422 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43932 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56554 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51850 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33670 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33672 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57444 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57458 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42452 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51402 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34930 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39424 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33444 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33450 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48562 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48576 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35504 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34222 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34606 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39372 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51490 - "GET /node/6992d9fa31166481c8dd4240%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.21.0.1:51502 - "GET /node/694a54f183c67c43d1ed9690%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.21.0.1:51510 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51526 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41772 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41778 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59478 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35806 - "GET /node/693e48f9407f32424eb5f4ef%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.21.0.1:35818 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60728 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60744 - "GET /node/694aacd783c67c43d1ede4ef%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.21.0.1:53024 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46844 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46846 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45094 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45104 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55582 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49810 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44050 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35354 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53666 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53668 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44484 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44490 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37974 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48738 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36870 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37818 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34876 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34890 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49062 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49068 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55944 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40564 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56034 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34154 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47550 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47560 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54356 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54360 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45286 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60186 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57024 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44924 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52822 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52834 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37896 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37910 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56708 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37268 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42836 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34282 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38302 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38314 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35018 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35022 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58686 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44920 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55496 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59902 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35278 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35288 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41082 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41086 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60732 - "GET / HTTP/1.0" 200 OK 2026-03-26 20:01:31,617 - lava_callback - INFO - Received callback for node 69c587ababf66a8f4d9bc5d1 INFO: 172.21.0.1:43720 - "POST /node/69c587ababf66a8f4d9bc5d1 HTTP/1.0" 202 Accepted 2026-03-26 20:01:31,665 - lava_callback - INFO - Starting async job submit for node 69c587ababf66a8f4d9bc5d1 2026-03-26 20:01:31,689 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 20:01:31,709 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c587ababf66a8f4d9bc5d1 HTTP/1.1" 200 None 2026-03-26 20:01:31,709 - lava_callback - DEBUG - Processing node 69c587ababf66a8f4d9bc5d1: baseline-arm-pengutronix 2026-03-26 20:01:31,820 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 20:01:32,000 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 20:01:32,002 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c587ababf66a8f4d9bc5d1/log.txt.gz 2026-03-26 20:01:32,060 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 20:01:32,239 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 20:01:32,241 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c587ababf66a8f4d9bc5d1/lava-logs.yaml 2026-03-26 20:01:32,244 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 20:01:32,414 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 20:01:32,416 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c587ababf66a8f4d9bc5d1/lava_callback.json.gz DEBUG: Setup failed for 69c587ababf66a8f4d9bc5d1. Transitting job node result: incomplete -> fail 2026-03-26 20:01:32,464 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 20:01:32,479 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c587ababf66a8f4d9bc5d1 HTTP/1.1" 200 None 2026-03-26 20:01:32,481 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 20:01:32,494 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c55c70abf66a8f4d9b9655 HTTP/1.1" 200 None 2026-03-26 20:01:32,496 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 20:01:32,521 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c587ababf66a8f4d9bc5d1 HTTP/1.1" 200 None 2026-03-26 20:01:32,590 - lava_callback - INFO - Completed processing callback for node 69c587ababf66a8f4d9bc5d1 2026-03-26 20:01:32,934 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 20:01:32,957 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:43728 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50496 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53266 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48076 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48084 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35624 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35638 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53178 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56536 - "GET /node/6985bbff71989fb1414974e9%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.21.0.1:56540 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33048 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46326 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50090 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50104 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33360 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33368 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58502 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35674 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58894 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40588 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33280 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33292 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36060 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36068 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56440 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51922 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44726 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40810 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58304 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58314 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56350 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56356 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51790 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60202 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39340 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41594 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52406 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52416 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50978 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50988 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49994 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46984 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40786 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49506 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59808 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59812 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44834 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44848 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35702 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59110 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37358 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45940 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54264 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54280 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54612 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54628 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43616 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50852 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49910 - "GET /node/684cc01257045812aa5286b4%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.21.0.1:49920 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46262 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54210 - "GET /node/698fcbb576d4b6342e9d7562%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.21.0.1:54226 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54240 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52444 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52454 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45760 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43416 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54684 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41178 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56414 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56428 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37332 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37340 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49352 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35678 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36650 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33622 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45170 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45176 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60940 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60948 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50892 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48956 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54736 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39978 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54002 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54016 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51678 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51682 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53720 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50904 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41548 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46934 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46776 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46786 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53136 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53152 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36078 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42726 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51392 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51544 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53656 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53668 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54812 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54818 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42372 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38114 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33754 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55964 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43788 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43798 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55808 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55818 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34268 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49488 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60090 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60470 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46728 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46736 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55438 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55452 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60396 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58546 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45472 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47718 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44918 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44928 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58146 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58162 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44466 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48586 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39280 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53696 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51850 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51860 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52950 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52960 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34866 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55924 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44322 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42228 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60684 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60700 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55994 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56000 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60558 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46604 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54296 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51296 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33550 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33554 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60412 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60418 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54476 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58986 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41758 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54500 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44326 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44334 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56738 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56752 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48308 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59382 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51282 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50398 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42616 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42626 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60544 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60546 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37164 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46032 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42800 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42250 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49166 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49180 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54066 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54068 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46424 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33904 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49802 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33544 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53624 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53638 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53620 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53628 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57670 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40852 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58282 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37686 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45146 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45156 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39750 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39756 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51936 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39222 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51444 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43794 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57472 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57474 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59518 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59534 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53344 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59222 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58902 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50298 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43226 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43232 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48224 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48236 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53728 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53848 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49626 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60836 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42466 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42470 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41190 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41198 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41176 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39052 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53640 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40524 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53026 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53042 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46788 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46800 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39104 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57842 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47608 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58650 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47202 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47218 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44750 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44754 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38676 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51082 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50086 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47324 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43618 - "GET /node/69973cc28babaef6576764b7%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.21.0.1:43624 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43630 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32924 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:32938 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35930 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59330 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53242 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49578 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39428 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39434 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44970 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44986 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42478 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58904 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37660 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52282 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58992 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58994 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56564 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56574 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60192 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36200 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54856 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53500 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47548 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47550 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38156 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38162 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57330 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33922 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50586 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53574 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40178 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40190 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60530 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60544 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41870 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57046 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43556 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56880 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56588 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56600 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42610 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42618 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43672 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48330 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59912 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52146 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34468 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34482 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45676 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45684 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45990 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47338 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47002 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47094 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34304 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34306 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52266 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52274 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35304 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33858 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35458 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43630 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38968 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38980 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39768 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39776 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54866 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41902 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48464 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57476 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34336 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34342 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60162 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60172 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39200 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36172 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47228 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52100 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60742 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54266 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54270 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54284 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37366 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42332 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47610 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44980 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32970 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41244 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41260 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41272 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43666 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32796 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56698 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48598 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41666 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37018 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37028 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37036 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45316 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45716 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44268 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44324 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55432 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40232 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40236 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40250 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48994 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40508 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33900 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57154 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48636 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44710 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44726 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44734 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43224 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34722 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60548 - "GET / HTTP/1.0" 200 OK 2026-03-26 20:45:54,909 - lava_callback - INFO - Received callback for node 69c2f7f06778148209fd6728 INFO: 172.21.0.1:60360 - "POST /node/69c2f7f06778148209fd6728 HTTP/1.0" 202 Accepted 2026-03-26 20:45:54,915 - lava_callback - INFO - Starting async job submit for node 69c2f7f06778148209fd6728 2026-03-26 20:45:54,916 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 20:45:54,930 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2f7f06778148209fd6728 HTTP/1.1" 200 None 2026-03-26 20:45:54,931 - lava_callback - DEBUG - Processing node 69c2f7f06778148209fd6728: baseline-arm-baylibre 2026-03-26 20:45:54,931 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2f7f06778148209fd6728 2026-03-26 20:45:54,932 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 20:45:54,945 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2f7f06778148209fd6728 HTTP/1.1" 200 None 2026-03-26 20:45:54,946 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 20:45:54,960 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248d74a2c5046eb87f890 HTTP/1.1" 200 None 2026-03-26 20:45:54,961 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 20:45:54,990 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2f7f06778148209fd6728 HTTP/1.1" 200 None 2026-03-26 20:45:54,992 - lava_callback - ERROR - Error processing callback for node 69c2f7f06778148209fd6728: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:60374 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43910 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47128 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47134 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47144 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54726 - "GET / HTTP/1.0" 200 OK 2026-03-26 20:46:34,583 - lava_callback - INFO - Received callback for node 69c2f8186778148209fd6738 INFO: 172.21.0.1:56090 - "POST /node/69c2f8186778148209fd6738 HTTP/1.0" 202 Accepted 2026-03-26 20:46:34,589 - lava_callback - INFO - Starting async job submit for node 69c2f8186778148209fd6738 2026-03-26 20:46:34,590 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 20:46:34,618 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2f8186778148209fd6738 HTTP/1.1" 200 None 2026-03-26 20:46:34,619 - lava_callback - DEBUG - Processing node 69c2f8186778148209fd6738: baseline-arm-baylibre 2026-03-26 20:46:34,619 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2f8186778148209fd6738 2026-03-26 20:46:34,620 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 20:46:34,633 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2f8186778148209fd6738 HTTP/1.1" 200 None 2026-03-26 20:46:34,635 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 20:46:34,650 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248d74a2c5046eb87f890 HTTP/1.1" 200 None 2026-03-26 20:46:34,652 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 20:46:34,673 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2f8186778148209fd6738 HTTP/1.1" 200 None 2026-03-26 20:46:34,678 - lava_callback - ERROR - Error processing callback for node 69c2f8186778148209fd6738: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:56094 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38240 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41456 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51090 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57838 - "GET / HTTP/1.0" 200 OK 2026-03-26 20:47:15,025 - lava_callback - INFO - Received callback for node 69c2f8416778148209fd6741 INFO: 172.21.0.1:57840 - "POST /node/69c2f8416778148209fd6741 HTTP/1.0" 202 Accepted 2026-03-26 20:47:15,031 - lava_callback - INFO - Starting async job submit for node 69c2f8416778148209fd6741 2026-03-26 20:47:15,032 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 20:47:15,047 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2f8416778148209fd6741 HTTP/1.1" 200 None 2026-03-26 20:47:15,048 - lava_callback - DEBUG - Processing node 69c2f8416778148209fd6741: baseline-arm-baylibre 2026-03-26 20:47:15,048 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2f8416778148209fd6741 2026-03-26 20:47:15,049 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 20:47:15,061 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2f8416778148209fd6741 HTTP/1.1" 200 None 2026-03-26 20:47:15,063 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 20:47:15,099 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248d74a2c5046eb87f890 HTTP/1.1" 200 None 2026-03-26 20:47:15,101 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 20:47:15,122 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2f8416778148209fd6741 HTTP/1.1" 200 None 2026-03-26 20:47:15,128 - lava_callback - ERROR - Error processing callback for node 69c2f8416778148209fd6741: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:57848 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57864 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35604 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41874 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42140 - "GET / HTTP/1.0" 200 OK 2026-03-26 20:47:54,676 - lava_callback - INFO - Received callback for node 69c2f8696778148209fd6752 INFO: 172.21.0.1:60418 - "POST /node/69c2f8696778148209fd6752 HTTP/1.0" 202 Accepted 2026-03-26 20:47:54,682 - lava_callback - INFO - Starting async job submit for node 69c2f8696778148209fd6752 2026-03-26 20:47:54,683 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 20:47:54,697 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2f8696778148209fd6752 HTTP/1.1" 200 None 2026-03-26 20:47:54,698 - lava_callback - DEBUG - Processing node 69c2f8696778148209fd6752: baseline-arm-baylibre 2026-03-26 20:47:54,698 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2f8696778148209fd6752 2026-03-26 20:47:54,699 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 20:47:54,721 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2f8696778148209fd6752 HTTP/1.1" 200 None 2026-03-26 20:47:54,722 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 20:47:54,736 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248d74a2c5046eb87f890 HTTP/1.1" 200 None 2026-03-26 20:47:54,738 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 20:47:54,759 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2f8696778148209fd6752 HTTP/1.1" 200 None 2026-03-26 20:47:54,765 - lava_callback - ERROR - Error processing callback for node 69c2f8696778148209fd6752: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:60432 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44018 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41408 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41410 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41414 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43740 - "GET / HTTP/1.0" 200 OK 2026-03-26 20:48:34,745 - lava_callback - INFO - Received callback for node 69c2f8936778148209fd675e INFO: 172.21.0.1:60330 - "POST /node/69c2f8936778148209fd675e HTTP/1.0" 202 Accepted 2026-03-26 20:48:34,750 - lava_callback - INFO - Starting async job submit for node 69c2f8936778148209fd675e 2026-03-26 20:48:34,751 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 20:48:34,766 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2f8936778148209fd675e HTTP/1.1" 200 None 2026-03-26 20:48:34,767 - lava_callback - DEBUG - Processing node 69c2f8936778148209fd675e: baseline-arm-baylibre 2026-03-26 20:48:34,767 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2f8936778148209fd675e 2026-03-26 20:48:34,768 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 20:48:34,781 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2f8936778148209fd675e HTTP/1.1" 200 None 2026-03-26 20:48:34,782 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 20:48:34,795 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248d74a2c5046eb87f890 HTTP/1.1" 200 None 2026-03-26 20:48:34,796 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 20:48:34,818 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2f8936778148209fd675e HTTP/1.1" 200 None 2026-03-26 20:48:34,824 - lava_callback - ERROR - Error processing callback for node 69c2f8936778148209fd675e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:60336 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42262 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42332 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47476 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52914 - "GET / HTTP/1.0" 200 OK 2026-03-26 20:49:14,752 - lava_callback - INFO - Received callback for node 69c2f8bb6778148209fd6767 INFO: 172.21.0.1:52920 - "POST /node/69c2f8bb6778148209fd6767 HTTP/1.0" 202 Accepted 2026-03-26 20:49:14,758 - lava_callback - INFO - Starting async job submit for node 69c2f8bb6778148209fd6767 2026-03-26 20:49:14,759 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 20:49:14,773 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2f8bb6778148209fd6767 HTTP/1.1" 200 None 2026-03-26 20:49:14,774 - lava_callback - DEBUG - Processing node 69c2f8bb6778148209fd6767: baseline-arm-baylibre 2026-03-26 20:49:14,774 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2f8bb6778148209fd6767 2026-03-26 20:49:14,774 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 20:49:14,791 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2f8bb6778148209fd6767 HTTP/1.1" 200 None 2026-03-26 20:49:14,793 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 20:49:14,810 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248d74a2c5046eb87f890 HTTP/1.1" 200 None 2026-03-26 20:49:14,811 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 20:49:14,833 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2f8bb6778148209fd6767 HTTP/1.1" 200 None 2026-03-26 20:49:14,839 - lava_callback - ERROR - Error processing callback for node 69c2f8bb6778148209fd6767: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:52934 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52940 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41314 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53132 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54518 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43046 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47934 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48550 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48562 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48570 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51898 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49174 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43874 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48932 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41662 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39192 - "GET / HTTP/1.0" 200 OK 2026-03-26 20:51:10,472 - lava_callback - INFO - Received callback for node 69c2f9356778148209fd6791 INFO: 172.21.0.1:39196 - "POST /node/69c2f9356778148209fd6791 HTTP/1.0" 202 Accepted 2026-03-26 20:51:10,477 - lava_callback - INFO - Starting async job submit for node 69c2f9356778148209fd6791 2026-03-26 20:51:10,478 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 20:51:10,507 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2f9356778148209fd6791 HTTP/1.1" 200 None 2026-03-26 20:51:10,507 - lava_callback - DEBUG - Processing node 69c2f9356778148209fd6791: baseline-arm-clabbe 2026-03-26 20:51:10,508 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2f9356778148209fd6791 2026-03-26 20:51:10,508 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 20:51:10,522 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2f9356778148209fd6791 HTTP/1.1" 200 None 2026-03-26 20:51:10,523 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 20:51:10,536 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248d74a2c5046eb87f890 HTTP/1.1" 200 None 2026-03-26 20:51:10,538 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 20:51:10,558 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2f9356778148209fd6791 HTTP/1.1" 200 None 2026-03-26 20:51:10,560 - lava_callback - ERROR - Error processing callback for node 69c2f9356778148209fd6791: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:39208 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39212 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54180 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39582 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39594 - "GET /node/693b0474fee8383e92b6f85f%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.21.0.1:44374 - "GET / HTTP/1.0" 200 OK 2026-03-26 20:51:50,480 - lava_callback - INFO - Received callback for node 69c2f95e6778148209fd679d INFO: 172.21.0.1:52634 - "POST /node/69c2f95e6778148209fd679d HTTP/1.0" 202 Accepted 2026-03-26 20:51:50,485 - lava_callback - INFO - Starting async job submit for node 69c2f95e6778148209fd679d 2026-03-26 20:51:50,486 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 20:51:50,500 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2f95e6778148209fd679d HTTP/1.1" 200 None 2026-03-26 20:51:50,501 - lava_callback - DEBUG - Processing node 69c2f95e6778148209fd679d: baseline-arm-clabbe 2026-03-26 20:51:50,501 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2f95e6778148209fd679d 2026-03-26 20:51:50,502 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 20:51:50,515 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2f95e6778148209fd679d HTTP/1.1" 200 None 2026-03-26 20:51:50,517 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 20:51:50,533 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248d74a2c5046eb87f890 HTTP/1.1" 200 None 2026-03-26 20:51:50,535 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 20:51:50,557 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2f95e6778148209fd679d HTTP/1.1" 200 None 2026-03-26 20:51:50,563 - lava_callback - ERROR - Error processing callback for node 69c2f95e6778148209fd679d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:52640 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58984 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36356 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36362 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36374 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45900 - "GET / HTTP/1.0" 200 OK 2026-03-26 20:52:30,555 - lava_callback - INFO - Received callback for node 69c2f9876778148209fd67ad INFO: 172.21.0.1:38970 - "POST /node/69c2f9876778148209fd67ad HTTP/1.0" 202 Accepted 2026-03-26 20:52:30,560 - lava_callback - INFO - Starting async job submit for node 69c2f9876778148209fd67ad 2026-03-26 20:52:30,561 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 20:52:30,576 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2f9876778148209fd67ad HTTP/1.1" 200 None 2026-03-26 20:52:30,577 - lava_callback - DEBUG - Processing node 69c2f9876778148209fd67ad: baseline-arm-clabbe 2026-03-26 20:52:30,577 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2f9876778148209fd67ad 2026-03-26 20:52:30,578 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 20:52:30,591 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2f9876778148209fd67ad HTTP/1.1" 200 None 2026-03-26 20:52:30,593 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 20:52:30,630 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248d74a2c5046eb87f890 HTTP/1.1" 200 None 2026-03-26 20:52:30,631 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 20:52:30,652 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2f9876778148209fd67ad HTTP/1.1" 200 None 2026-03-26 20:52:30,654 - lava_callback - ERROR - Error processing callback for node 69c2f9876778148209fd67ad: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:38978 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57406 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49948 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51648 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45068 - "GET / HTTP/1.0" 200 OK 2026-03-26 20:53:10,495 - lava_callback - INFO - Received callback for node 69c2f9b16778148209fd67b6 INFO: 172.21.0.1:45074 - "POST /node/69c2f9b16778148209fd67b6 HTTP/1.0" 202 Accepted 2026-03-26 20:53:10,501 - lava_callback - INFO - Starting async job submit for node 69c2f9b16778148209fd67b6 2026-03-26 20:53:10,501 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 20:53:10,515 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2f9b16778148209fd67b6 HTTP/1.1" 200 None 2026-03-26 20:53:10,516 - lava_callback - DEBUG - Processing node 69c2f9b16778148209fd67b6: baseline-arm-clabbe 2026-03-26 20:53:10,517 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2f9b16778148209fd67b6 2026-03-26 20:53:10,517 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 20:53:10,534 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2f9b16778148209fd67b6 HTTP/1.1" 200 None 2026-03-26 20:53:10,536 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 20:53:10,549 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248d74a2c5046eb87f890 HTTP/1.1" 200 None 2026-03-26 20:53:10,550 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 20:53:10,743 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2f9b16778148209fd67b6 HTTP/1.1" 200 None 2026-03-26 20:53:10,748 - lava_callback - ERROR - Error processing callback for node 69c2f9b16778148209fd67b6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:45078 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45080 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42416 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50636 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44784 - "GET / HTTP/1.0" 200 OK 2026-03-26 20:53:50,583 - lava_callback - INFO - Received callback for node 69c2f9d96778148209fd67c4 INFO: 172.21.0.1:52312 - "POST /node/69c2f9d96778148209fd67c4 HTTP/1.0" 202 Accepted 2026-03-26 20:53:50,589 - lava_callback - INFO - Starting async job submit for node 69c2f9d96778148209fd67c4 2026-03-26 20:53:50,589 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 20:53:50,603 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2f9d96778148209fd67c4 HTTP/1.1" 200 None 2026-03-26 20:53:50,604 - lava_callback - DEBUG - Processing node 69c2f9d96778148209fd67c4: baseline-arm-clabbe 2026-03-26 20:53:50,604 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2f9d96778148209fd67c4 2026-03-26 20:53:50,605 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 20:53:50,617 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2f9d96778148209fd67c4 HTTP/1.1" 200 None 2026-03-26 20:53:50,618 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 20:53:50,636 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248d74a2c5046eb87f890 HTTP/1.1" 200 None 2026-03-26 20:53:50,638 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 20:53:50,662 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2f9d96778148209fd67c4 HTTP/1.1" 200 None 2026-03-26 20:53:50,668 - lava_callback - ERROR - Error processing callback for node 69c2f9d96778148209fd67c4: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:52314 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42080 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40864 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40880 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40896 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40432 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49206 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41928 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44242 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49574 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34432 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34436 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34440 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36900 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38840 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45518 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40900 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53282 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46296 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46310 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46320 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55424 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59108 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47226 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48564 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46770 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51010 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51016 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51020 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57126 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34706 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43224 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57360 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43458 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36082 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36094 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36108 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48280 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58136 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57306 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40016 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36510 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54402 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54408 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54416 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37882 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59964 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58658 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47682 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47696 - "GET /favicon.ico HTTP/1.0" 404 Not Found INFO: 172.21.0.1:47704 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60318 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50244 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50260 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50274 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48822 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46552 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48108 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34020 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54768 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49504 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49512 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49524 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36398 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37986 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53836 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40196 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46282 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56646 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56660 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56662 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34824 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49700 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57396 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46996 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34548 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48346 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48352 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48354 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50570 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45274 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43014 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53806 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59438 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58746 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58762 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58778 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48826 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55836 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51552 - "GET / HTTP/1.0" 200 OK 2026-03-26 21:04:55,898 - lava_callback - INFO - Received callback for node 69c2fc746778148209fd6821 INFO: 172.21.0.1:47380 - "POST /node/69c2fc746778148209fd6821 HTTP/1.0" 202 Accepted 2026-03-26 21:04:55,903 - lava_callback - INFO - Starting async job submit for node 69c2fc746778148209fd6821 2026-03-26 21:04:55,904 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 21:04:55,919 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2fc746778148209fd6821 HTTP/1.1" 200 None 2026-03-26 21:04:55,919 - lava_callback - DEBUG - Processing node 69c2fc746778148209fd6821: baseline-arm64-baylibre 2026-03-26 21:04:55,920 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2fc746778148209fd6821 2026-03-26 21:04:55,920 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 21:04:55,933 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2fc746778148209fd6821 HTTP/1.1" 200 None 2026-03-26 21:04:55,934 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 21:04:55,948 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5827544ab5cd4a7d9dd HTTP/1.1" 200 None 2026-03-26 21:04:55,949 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 21:04:55,972 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2fc746778148209fd6821 HTTP/1.1" 200 None 2026-03-26 21:04:55,983 - lava_callback - ERROR - Error processing callback for node 69c2fc746778148209fd6821: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:47396 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41374 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35118 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35128 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35138 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58806 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43156 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46270 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35566 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47290 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36568 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36578 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36584 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58402 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40430 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41998 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53586 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50642 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45418 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45434 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45438 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56322 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51582 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37122 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53664 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59926 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44164 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44176 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44190 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56678 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53682 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46990 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53780 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48818 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59310 - "GET / HTTP/1.0" 200 OK 2026-03-26 21:09:11,379 - lava_callback - INFO - Received callback for node 69c2fd6b6778148209fd6857 INFO: 172.21.0.1:59316 - "POST /node/69c2fd6b6778148209fd6857 HTTP/1.0" 202 Accepted 2026-03-26 21:09:11,385 - lava_callback - INFO - Starting async job submit for node 69c2fd6b6778148209fd6857 2026-03-26 21:09:11,386 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 21:09:11,400 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2fd6b6778148209fd6857 HTTP/1.1" 200 None 2026-03-26 21:09:11,401 - lava_callback - DEBUG - Processing node 69c2fd6b6778148209fd6857: baseline-arm64-clabbe 2026-03-26 21:09:11,401 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2fd6b6778148209fd6857 2026-03-26 21:09:11,402 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 21:09:11,422 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2fd6b6778148209fd6857 HTTP/1.1" 200 None 2026-03-26 21:09:11,424 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 21:09:11,437 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5827544ab5cd4a7d9dd HTTP/1.1" 200 None 2026-03-26 21:09:11,439 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 21:09:11,459 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2fd6b6778148209fd6857 HTTP/1.1" 200 None 2026-03-26 21:09:11,460 - lava_callback - ERROR - Error processing callback for node 69c2fd6b6778148209fd6857: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:59330 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59332 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60826 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57922 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35554 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35916 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58010 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58050 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58060 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58064 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38070 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46226 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59010 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58758 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41906 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37136 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37138 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37154 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45452 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42050 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54950 - "GET / HTTP/1.0" 200 OK 2026-03-26 21:11:51,356 - lava_callback - INFO - Received callback for node 69c2fe0c6778148209fd687e INFO: 172.21.0.1:50980 - "POST /node/69c2fe0c6778148209fd687e HTTP/1.0" 202 Accepted 2026-03-26 21:11:51,362 - lava_callback - INFO - Starting async job submit for node 69c2fe0c6778148209fd687e 2026-03-26 21:11:51,362 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 21:11:51,384 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2fe0c6778148209fd687e HTTP/1.1" 200 None 2026-03-26 21:11:51,385 - lava_callback - DEBUG - Processing node 69c2fe0c6778148209fd687e: baseline-arm64-clabbe 2026-03-26 21:11:51,386 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2fe0c6778148209fd687e 2026-03-26 21:11:51,386 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 21:11:51,400 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2fe0c6778148209fd687e HTTP/1.1" 200 None 2026-03-26 21:11:51,401 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 21:11:51,414 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5827544ab5cd4a7d9dd HTTP/1.1" 200 None 2026-03-26 21:11:51,416 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 21:11:51,436 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2fe0c6778148209fd687e HTTP/1.1" 200 None 2026-03-26 21:11:51,446 - lava_callback - ERROR - Error processing callback for node 69c2fe0c6778148209fd687e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:50984 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35526 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58068 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58070 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58072 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58784 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47558 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56634 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35662 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53146 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33442 - "GET / HTTP/1.0" 200 OK 2026-03-26 21:13:11,387 - lava_callback - INFO - Received callback for node 69c2fe5f6778148209fd688e INFO: 172.21.0.1:33456 - "POST /node/69c2fe5f6778148209fd688e HTTP/1.0" 202 Accepted 2026-03-26 21:13:11,392 - lava_callback - INFO - Starting async job submit for node 69c2fe5f6778148209fd688e 2026-03-26 21:13:11,393 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 21:13:11,408 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2fe5f6778148209fd688e HTTP/1.1" 200 None 2026-03-26 21:13:11,409 - lava_callback - DEBUG - Processing node 69c2fe5f6778148209fd688e: baseline-arm64-clabbe 2026-03-26 21:13:11,409 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2fe5f6778148209fd688e 2026-03-26 21:13:11,410 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 21:13:11,423 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2fe5f6778148209fd688e HTTP/1.1" 200 None 2026-03-26 21:13:11,424 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 21:13:11,444 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5827544ab5cd4a7d9dd HTTP/1.1" 200 None 2026-03-26 21:13:11,445 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 21:13:11,465 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2fe5f6778148209fd688e HTTP/1.1" 200 None 2026-03-26 21:13:11,467 - lava_callback - ERROR - Error processing callback for node 69c2fe5f6778148209fd688e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:33462 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33470 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51576 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49458 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53986 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33502 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37226 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60340 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60350 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60360 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49632 - "GET / HTTP/1.0" 200 OK 2026-03-26 21:14:36,164 - lava_callback - INFO - Received callback for node 69c2fea76778148209fd6898 INFO: 172.21.0.1:43112 - "POST /node/69c2fea76778148209fd6898 HTTP/1.0" 202 Accepted 2026-03-26 21:14:36,170 - lava_callback - INFO - Starting async job submit for node 69c2fea76778148209fd6898 2026-03-26 21:14:36,171 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 21:14:36,185 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2fea76778148209fd6898 HTTP/1.1" 200 None 2026-03-26 21:14:36,186 - lava_callback - DEBUG - Processing node 69c2fea76778148209fd6898: baseline-arm64-baylibre 2026-03-26 21:14:36,186 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2fea76778148209fd6898 2026-03-26 21:14:36,187 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 21:14:36,200 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2fea76778148209fd6898 HTTP/1.1" 200 None 2026-03-26 21:14:36,201 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 21:14:36,214 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248e54a2c5046eb87f89f HTTP/1.1" 200 None 2026-03-26 21:14:36,216 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 21:14:36,243 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2fea76778148209fd6898 HTTP/1.1" 200 None 2026-03-26 21:14:36,245 - lava_callback - ERROR - Error processing callback for node 69c2fea76778148209fd6898: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:43124 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37660 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34510 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51396 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39548 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39554 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39568 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39168 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38096 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33940 - "GET / HTTP/1.0" 200 OK 2026-03-26 21:15:56,226 - lava_callback - INFO - Received callback for node 69c2fef86778148209fd68a6 INFO: 172.21.0.1:39476 - "POST /node/69c2fef86778148209fd68a6 HTTP/1.0" 202 Accepted 2026-03-26 21:15:56,231 - lava_callback - INFO - Starting async job submit for node 69c2fef86778148209fd68a6 2026-03-26 21:15:56,232 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 21:15:56,251 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2fef86778148209fd68a6 HTTP/1.1" 200 None 2026-03-26 21:15:56,251 - lava_callback - DEBUG - Processing node 69c2fef86778148209fd68a6: baseline-arm64-baylibre 2026-03-26 21:15:56,252 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2fef86778148209fd68a6 2026-03-26 21:15:56,252 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 21:15:56,266 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2fef86778148209fd68a6 HTTP/1.1" 200 None 2026-03-26 21:15:56,267 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 21:15:56,281 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248e54a2c5046eb87f89f HTTP/1.1" 200 None 2026-03-26 21:15:56,282 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 21:15:56,304 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2fef86778148209fd68a6 HTTP/1.1" 200 None 2026-03-26 21:15:56,311 - lava_callback - ERROR - Error processing callback for node 69c2fef86778148209fd68a6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:39484 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47372 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45010 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45018 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45030 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44400 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32816 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55960 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54094 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42376 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56272 - "GET / HTTP/1.0" 200 OK 2026-03-26 21:17:11,610 - lava_callback - INFO - Received callback for node 69c2ff526778148209fd68b7 INFO: 172.21.0.1:56280 - "POST /node/69c2ff526778148209fd68b7 HTTP/1.0" 202 Accepted 2026-03-26 21:17:11,616 - lava_callback - INFO - Starting async job submit for node 69c2ff526778148209fd68b7 2026-03-26 21:17:11,617 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 21:17:11,635 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2ff526778148209fd68b7 HTTP/1.1" 200 None 2026-03-26 21:17:11,636 - lava_callback - DEBUG - Processing node 69c2ff526778148209fd68b7: baseline-arm64-clabbe 2026-03-26 21:17:11,636 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2ff526778148209fd68b7 2026-03-26 21:17:11,637 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 21:17:11,651 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2ff526778148209fd68b7 HTTP/1.1" 200 None 2026-03-26 21:17:11,652 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 21:17:11,666 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5827544ab5cd4a7d9dd HTTP/1.1" 200 None 2026-03-26 21:17:11,668 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 21:17:11,695 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2ff526778148209fd68b7 HTTP/1.1" 200 None 2026-03-26 21:17:11,697 - lava_callback - ERROR - Error processing callback for node 69c2ff526778148209fd68b7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:56284 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56290 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38450 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43578 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47710 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50668 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55060 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33434 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33442 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33454 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33168 - "GET / HTTP/1.0" 200 OK 2026-03-26 21:18:31,721 - lava_callback - INFO - Received callback for node 69c2ffa36778148209fd68c4 INFO: 172.21.0.1:41180 - "POST /node/69c2ffa36778148209fd68c4 HTTP/1.0" 202 Accepted 2026-03-26 21:18:31,726 - lava_callback - INFO - Starting async job submit for node 69c2ffa36778148209fd68c4 2026-03-26 21:18:31,727 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 21:18:31,743 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2ffa36778148209fd68c4 HTTP/1.1" 200 None 2026-03-26 21:18:31,744 - lava_callback - DEBUG - Processing node 69c2ffa36778148209fd68c4: baseline-arm64-clabbe 2026-03-26 21:18:31,744 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2ffa36778148209fd68c4 2026-03-26 21:18:31,745 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 21:18:31,761 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2ffa36778148209fd68c4 HTTP/1.1" 200 None 2026-03-26 21:18:31,762 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 21:18:31,778 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5827544ab5cd4a7d9dd HTTP/1.1" 200 None 2026-03-26 21:18:31,780 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 21:18:31,800 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2ffa36778148209fd68c4 HTTP/1.1" 200 None 2026-03-26 21:18:31,802 - lava_callback - ERROR - Error processing callback for node 69c2ffa36778148209fd68c4: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:41188 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36492 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42622 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55708 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32984 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32996 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33004 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59392 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40710 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60404 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46334 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60488 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45876 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45884 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45890 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56986 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41024 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41104 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56398 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58374 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51116 - "GET / HTTP/1.0" 200 OK 2026-03-26 21:21:11,973 - lava_callback - INFO - Received callback for node 69c300436778148209fd68d8 INFO: 172.21.0.1:51120 - "POST /node/69c300436778148209fd68d8 HTTP/1.0" 202 Accepted 2026-03-26 21:21:11,979 - lava_callback - INFO - Starting async job submit for node 69c300436778148209fd68d8 2026-03-26 21:21:11,980 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 21:21:11,994 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c300436778148209fd68d8 HTTP/1.1" 200 None 2026-03-26 21:21:11,995 - lava_callback - DEBUG - Processing node 69c300436778148209fd68d8: baseline-arm64-clabbe 2026-03-26 21:21:11,995 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c300436778148209fd68d8 2026-03-26 21:21:11,996 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 21:21:12,015 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c300436778148209fd68d8 HTTP/1.1" 200 None 2026-03-26 21:21:12,017 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 21:21:12,033 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5827544ab5cd4a7d9dd HTTP/1.1" 200 None 2026-03-26 21:21:12,034 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 21:21:12,055 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c300436778148209fd68d8 HTTP/1.1" 200 None 2026-03-26 21:21:12,061 - lava_callback - ERROR - Error processing callback for node 69c300436778148209fd68d8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:51128 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51134 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60560 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47012 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45074 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55480 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45472 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43614 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43622 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43634 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39342 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48966 - "GET / HTTP/1.0" 200 OK 2026-03-26 21:22:36,946 - lava_callback - INFO - Received callback for node 69c3008d6778148209fd68e1 INFO: 172.21.0.1:48982 - "POST /node/69c3008d6778148209fd68e1 HTTP/1.0" 202 Accepted 2026-03-26 21:22:36,952 - lava_callback - INFO - Starting async job submit for node 69c3008d6778148209fd68e1 2026-03-26 21:22:36,953 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 21:22:36,967 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3008d6778148209fd68e1 HTTP/1.1" 200 None 2026-03-26 21:22:36,968 - lava_callback - DEBUG - Processing node 69c3008d6778148209fd68e1: baseline-arm64-baylibre 2026-03-26 21:22:36,969 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3008d6778148209fd68e1 2026-03-26 21:22:36,969 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 21:22:36,982 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3008d6778148209fd68e1 HTTP/1.1" 200 None 2026-03-26 21:22:36,983 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 21:22:36,997 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248e54a2c5046eb87f89f HTTP/1.1" 200 None 2026-03-26 21:22:36,999 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 21:22:37,031 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3008d6778148209fd68e1 HTTP/1.1" 200 None 2026-03-26 21:22:37,033 - lava_callback - ERROR - Error processing callback for node 69c3008d6778148209fd68e1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:54442 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40064 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39890 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57362 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57366 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57382 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50222 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52876 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36644 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46722 - "GET / HTTP/1.0" 200 OK 2026-03-26 21:23:56,700 - lava_callback - INFO - Received callback for node 69c300dd6778148209fd68f2 INFO: 172.21.0.1:46728 - "POST /node/69c300dd6778148209fd68f2 HTTP/1.0" 202 Accepted 2026-03-26 21:23:56,706 - lava_callback - INFO - Starting async job submit for node 69c300dd6778148209fd68f2 2026-03-26 21:23:56,707 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 21:23:56,732 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c300dd6778148209fd68f2 HTTP/1.1" 200 None 2026-03-26 21:23:56,733 - lava_callback - DEBUG - Processing node 69c300dd6778148209fd68f2: baseline-arm64-baylibre 2026-03-26 21:23:56,733 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c300dd6778148209fd68f2 2026-03-26 21:23:56,733 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 21:23:56,746 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c300dd6778148209fd68f2 HTTP/1.1" 200 None 2026-03-26 21:23:56,747 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 21:23:56,761 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248e54a2c5046eb87f89f HTTP/1.1" 200 None 2026-03-26 21:23:56,762 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 21:23:56,807 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c300dd6778148209fd68f2 HTTP/1.1" 200 None 2026-03-26 21:23:56,808 - lava_callback - ERROR - Error processing callback for node 69c300dd6778148209fd68f2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:35714 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50634 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50636 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50650 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51394 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60490 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53282 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52962 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40006 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58722 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58734 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58742 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35572 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42496 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46186 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47878 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37294 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35208 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35210 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35214 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52432 - "GET / HTTP/1.0" 200 OK 2026-03-26 21:26:32,009 - lava_callback - INFO - Received callback for node 69c301856778148209fd6914 INFO: 172.21.0.1:37782 - "POST /node/69c301856778148209fd6914 HTTP/1.0" 202 Accepted 2026-03-26 21:26:32,015 - lava_callback - INFO - Starting async job submit for node 69c301856778148209fd6914 2026-03-26 21:26:32,016 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 21:26:32,030 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c301856778148209fd6914 HTTP/1.1" 200 None 2026-03-26 21:26:32,030 - lava_callback - DEBUG - Processing node 69c301856778148209fd6914: baseline-arm64-clabbe 2026-03-26 21:26:32,031 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c301856778148209fd6914 2026-03-26 21:26:32,031 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 21:26:32,051 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c301856778148209fd6914 HTTP/1.1" 200 None 2026-03-26 21:26:32,053 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 21:26:32,068 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248e54a2c5046eb87f89f HTTP/1.1" 200 None 2026-03-26 21:26:32,069 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 21:26:32,098 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c301856778148209fd6914 HTTP/1.1" 200 None 2026-03-26 21:26:32,099 - lava_callback - ERROR - Error processing callback for node 69c301856778148209fd6914: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:37796 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39758 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47766 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36016 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33726 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33728 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33730 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44290 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33580 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49344 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44154 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57498 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36710 - "GET / HTTP/1.0" 200 OK 2026-03-26 21:28:12,075 - lava_callback - INFO - Received callback for node 69c301d76778148209fd691d INFO: 172.21.0.1:36722 - "POST /node/69c301d76778148209fd691d HTTP/1.0" 202 Accepted 2026-03-26 21:28:12,081 - lava_callback - INFO - Starting async job submit for node 69c301d76778148209fd691d 2026-03-26 21:28:12,082 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 21:28:12,097 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c301d76778148209fd691d HTTP/1.1" 200 None 2026-03-26 21:28:12,098 - lava_callback - DEBUG - Processing node 69c301d76778148209fd691d: baseline-arm64-clabbe 2026-03-26 21:28:12,098 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c301d76778148209fd691d 2026-03-26 21:28:12,098 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 21:28:12,112 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c301d76778148209fd691d HTTP/1.1" 200 None 2026-03-26 21:28:12,113 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 21:28:12,128 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248e54a2c5046eb87f89f HTTP/1.1" 200 None 2026-03-26 21:28:12,129 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 21:28:12,166 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c301d76778148209fd691d HTTP/1.1" 200 None 2026-03-26 21:28:12,168 - lava_callback - ERROR - Error processing callback for node 69c301d76778148209fd691d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:36736 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36746 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46764 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40676 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50812 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49564 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58770 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47042 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47058 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47072 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34962 - "GET / HTTP/1.0" 200 OK 2026-03-26 21:29:32,168 - lava_callback - INFO - Received callback for node 69c302286778148209fd692b INFO: 172.21.0.1:56606 - "POST /node/69c302286778148209fd692b HTTP/1.0" 202 Accepted 2026-03-26 21:29:32,173 - lava_callback - INFO - Starting async job submit for node 69c302286778148209fd692b 2026-03-26 21:29:32,174 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 21:29:32,189 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c302286778148209fd692b HTTP/1.1" 200 None 2026-03-26 21:29:32,189 - lava_callback - DEBUG - Processing node 69c302286778148209fd692b: baseline-arm64-clabbe 2026-03-26 21:29:32,190 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c302286778148209fd692b 2026-03-26 21:29:32,190 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 21:29:32,203 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c302286778148209fd692b HTTP/1.1" 200 None 2026-03-26 21:29:32,205 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 21:29:32,218 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248e54a2c5046eb87f89f HTTP/1.1" 200 None 2026-03-26 21:29:32,219 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 21:29:32,245 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c302286778148209fd692b HTTP/1.1" 200 None 2026-03-26 21:29:32,250 - lava_callback - ERROR - Error processing callback for node 69c302286778148209fd692b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:56620 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48448 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46048 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58928 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52674 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52684 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52698 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40042 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42324 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58180 - "GET / HTTP/1.0" 200 OK 2026-03-26 21:30:52,180 - lava_callback - INFO - Received callback for node 69c302796778148209fd693e INFO: 172.21.0.1:58996 - "POST /node/69c302796778148209fd693e HTTP/1.0" 202 Accepted 2026-03-26 21:30:52,185 - lava_callback - INFO - Starting async job submit for node 69c302796778148209fd693e 2026-03-26 21:30:52,186 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 21:30:52,200 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c302796778148209fd693e HTTP/1.1" 200 None 2026-03-26 21:30:52,201 - lava_callback - DEBUG - Processing node 69c302796778148209fd693e: baseline-arm64-clabbe 2026-03-26 21:30:52,201 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c302796778148209fd693e 2026-03-26 21:30:52,202 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 21:30:52,215 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c302796778148209fd693e HTTP/1.1" 200 None 2026-03-26 21:30:52,216 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 21:30:52,232 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248e54a2c5046eb87f89f HTTP/1.1" 200 None 2026-03-26 21:30:52,234 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 21:30:52,255 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c302796778148209fd693e HTTP/1.1" 200 None 2026-03-26 21:30:52,257 - lava_callback - ERROR - Error processing callback for node 69c302796778148209fd693e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:59002 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58934 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44428 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44440 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44450 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46558 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44646 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43988 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60926 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58790 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59730 - "GET / HTTP/1.0" 200 OK 2026-03-26 21:32:12,277 - lava_callback - INFO - Received callback for node 69c302cb6778148209fd694e INFO: 172.21.0.1:59736 - "POST /node/69c302cb6778148209fd694e HTTP/1.0" 202 Accepted 2026-03-26 21:32:12,283 - lava_callback - INFO - Starting async job submit for node 69c302cb6778148209fd694e 2026-03-26 21:32:12,284 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 21:32:12,298 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c302cb6778148209fd694e HTTP/1.1" 200 None 2026-03-26 21:32:12,299 - lava_callback - DEBUG - Processing node 69c302cb6778148209fd694e: baseline-arm64-clabbe 2026-03-26 21:32:12,299 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c302cb6778148209fd694e 2026-03-26 21:32:12,300 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 21:32:12,318 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c302cb6778148209fd694e HTTP/1.1" 200 None 2026-03-26 21:32:12,320 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 21:32:12,333 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248e54a2c5046eb87f89f HTTP/1.1" 200 None 2026-03-26 21:32:12,335 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 21:32:12,356 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c302cb6778148209fd694e HTTP/1.1" 200 None 2026-03-26 21:32:12,357 - lava_callback - ERROR - Error processing callback for node 69c302cb6778148209fd694e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:59746 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59752 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49428 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41064 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35290 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39212 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45788 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53450 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53456 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53468 - "GET / HTTP/1.0" 200 OK 2026-03-26 21:33:17,237 - lava_callback - INFO - Received callback for node 69c303086778148209fd6956 INFO: 172.21.0.1:53476 - "POST /node/69c303086778148209fd6956 HTTP/1.0" 202 Accepted 2026-03-26 21:33:17,243 - lava_callback - INFO - Starting async job submit for node 69c303086778148209fd6956 2026-03-26 21:33:17,244 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 21:33:17,259 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c303086778148209fd6956 HTTP/1.1" 200 None 2026-03-26 21:33:17,260 - lava_callback - DEBUG - Processing node 69c303086778148209fd6956: baseline-arm-baylibre 2026-03-26 21:33:17,260 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c303086778148209fd6956 2026-03-26 21:33:17,261 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 21:33:17,281 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c303086778148209fd6956 HTTP/1.1" 200 None 2026-03-26 21:33:17,283 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 21:33:17,296 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5737544ab5cd4a7d9ce HTTP/1.1" 200 None 2026-03-26 21:33:17,297 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 21:33:17,338 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c303086778148209fd6956 HTTP/1.1" 200 None 2026-03-26 21:33:17,345 - lava_callback - ERROR - Error processing callback for node 69c303086778148209fd6956: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:60570 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49690 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55146 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37116 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33060 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46496 - "GET / HTTP/1.0" 200 OK 2026-03-26 21:34:12,340 - lava_callback - INFO - Received callback for node 69c303426778148209fd6966 INFO: 172.21.0.1:46500 - "POST /node/69c303426778148209fd6966 HTTP/1.0" 202 Accepted 2026-03-26 21:34:12,346 - lava_callback - INFO - Starting async job submit for node 69c303426778148209fd6966 2026-03-26 21:34:12,347 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 21:34:12,367 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c303426778148209fd6966 HTTP/1.1" 200 None 2026-03-26 21:34:12,368 - lava_callback - DEBUG - Processing node 69c303426778148209fd6966: baseline-arm64-clabbe 2026-03-26 21:34:12,368 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c303426778148209fd6966 2026-03-26 21:34:12,369 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 21:34:12,382 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c303426778148209fd6966 HTTP/1.1" 200 None 2026-03-26 21:34:12,384 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 21:34:12,396 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248e54a2c5046eb87f89f HTTP/1.1" 200 None 2026-03-26 21:34:12,398 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 21:34:12,419 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c303426778148209fd6966 HTTP/1.1" 200 None 2026-03-26 21:34:12,420 - lava_callback - ERROR - Error processing callback for node 69c303426778148209fd6966: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:46512 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46518 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49752 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35084 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53378 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47592 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43588 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42948 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42954 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42956 - "GET / HTTP/1.0" 200 OK 2026-03-26 21:35:17,291 - lava_callback - INFO - Received callback for node 69c303826778148209fd6976 INFO: 172.21.0.1:42958 - "POST /node/69c303826778148209fd6976 HTTP/1.0" 202 Accepted 2026-03-26 21:35:17,297 - lava_callback - INFO - Starting async job submit for node 69c303826778148209fd6976 2026-03-26 21:35:17,298 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 21:35:17,312 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c303826778148209fd6976 HTTP/1.1" 200 None 2026-03-26 21:35:17,313 - lava_callback - DEBUG - Processing node 69c303826778148209fd6976: baseline-arm-baylibre 2026-03-26 21:35:17,313 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c303826778148209fd6976 2026-03-26 21:35:17,314 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 21:35:17,348 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c303826778148209fd6976 HTTP/1.1" 200 None 2026-03-26 21:35:17,350 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 21:35:17,367 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5737544ab5cd4a7d9ce HTTP/1.1" 200 None 2026-03-26 21:35:17,368 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 21:35:17,389 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c303826778148209fd6976 HTTP/1.1" 200 None 2026-03-26 21:35:17,398 - lava_callback - ERROR - Error processing callback for node 69c303826778148209fd6976: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:49648 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45902 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45760 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51286 - "GET / HTTP/1.0" 200 OK 2026-03-26 21:35:57,313 - lava_callback - INFO - Received callback for node 69c303a96778148209fd6980 INFO: 172.21.0.1:51290 - "POST /node/69c303a96778148209fd6980 HTTP/1.0" 202 Accepted 2026-03-26 21:35:57,319 - lava_callback - INFO - Starting async job submit for node 69c303a96778148209fd6980 2026-03-26 21:35:57,320 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 21:35:57,334 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c303a96778148209fd6980 HTTP/1.1" 200 None 2026-03-26 21:35:57,335 - lava_callback - DEBUG - Processing node 69c303a96778148209fd6980: baseline-arm-baylibre 2026-03-26 21:35:57,335 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c303a96778148209fd6980 2026-03-26 21:35:57,336 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 21:35:57,350 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c303a96778148209fd6980 HTTP/1.1" 200 None 2026-03-26 21:35:57,352 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 21:35:57,388 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5737544ab5cd4a7d9ce HTTP/1.1" 200 None 2026-03-26 21:35:57,390 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 21:35:57,416 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c303a96778148209fd6980 HTTP/1.1" 200 None 2026-03-26 21:35:57,423 - lava_callback - ERROR - Error processing callback for node 69c303a96778148209fd6980: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:52894 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56184 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56198 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56210 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36704 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35750 - "GET / HTTP/1.0" 200 OK 2026-03-26 21:36:37,356 - lava_callback - INFO - Received callback for node 69c303db6778148209fd698e INFO: 172.21.0.1:35754 - "POST /node/69c303db6778148209fd698e HTTP/1.0" 202 Accepted 2026-03-26 21:36:37,362 - lava_callback - INFO - Starting async job submit for node 69c303db6778148209fd698e 2026-03-26 21:36:37,363 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 21:36:37,377 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c303db6778148209fd698e HTTP/1.1" 200 None 2026-03-26 21:36:37,378 - lava_callback - DEBUG - Processing node 69c303db6778148209fd698e: baseline-arm64-baylibre 2026-03-26 21:36:37,379 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c303db6778148209fd698e 2026-03-26 21:36:37,379 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 21:36:37,392 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c303db6778148209fd698e HTTP/1.1" 200 None 2026-03-26 21:36:37,394 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 21:36:37,407 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5827544ab5cd4a7d9dd HTTP/1.1" 200 None 2026-03-26 21:36:37,409 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 21:36:37,429 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c303db6778148209fd698e HTTP/1.1" 200 None 2026-03-26 21:36:37,431 - lava_callback - ERROR - Error processing callback for node 69c303db6778148209fd698e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:39628 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52214 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41920 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42112 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42128 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42136 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43206 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48300 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48490 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44262 - "GET / HTTP/1.0" 200 OK 2026-03-26 21:37:57,835 - lava_callback - INFO - Received callback for node 69c3042b6778148209fd6998 INFO: 172.21.0.1:44266 - "POST /node/69c3042b6778148209fd6998 HTTP/1.0" 202 Accepted 2026-03-26 21:37:57,841 - lava_callback - INFO - Starting async job submit for node 69c3042b6778148209fd6998 2026-03-26 21:37:57,841 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 21:37:57,856 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3042b6778148209fd6998 HTTP/1.1" 200 None 2026-03-26 21:37:57,856 - lava_callback - DEBUG - Processing node 69c3042b6778148209fd6998: baseline-arm64-baylibre 2026-03-26 21:37:57,857 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3042b6778148209fd6998 2026-03-26 21:37:57,857 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 21:37:57,870 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3042b6778148209fd6998 HTTP/1.1" 200 None 2026-03-26 21:37:57,872 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 21:37:57,892 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5827544ab5cd4a7d9dd HTTP/1.1" 200 None 2026-03-26 21:37:57,894 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 21:37:57,914 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3042b6778148209fd6998 HTTP/1.1" 200 None 2026-03-26 21:37:57,924 - lava_callback - ERROR - Error processing callback for node 69c3042b6778148209fd6998: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:56542 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52036 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52050 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52062 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47398 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54790 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33250 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34148 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40172 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41684 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41688 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41702 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60160 - "GET / HTTP/1.0" 200 OK 2026-03-26 21:39:32,555 - lava_callback - INFO - Received callback for node 69c304846778148209fd69a6 INFO: 172.21.0.1:54800 - "POST /node/69c304846778148209fd69a6 HTTP/1.0" 202 Accepted 2026-03-26 21:39:32,560 - lava_callback - INFO - Starting async job submit for node 69c304846778148209fd69a6 2026-03-26 21:39:32,561 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 21:39:32,582 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c304846778148209fd69a6 HTTP/1.1" 200 None 2026-03-26 21:39:32,583 - lava_callback - DEBUG - Processing node 69c304846778148209fd69a6: baseline-arm64-clabbe 2026-03-26 21:39:32,584 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c304846778148209fd69a6 2026-03-26 21:39:32,584 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 21:39:32,597 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c304846778148209fd69a6 HTTP/1.1" 200 None 2026-03-26 21:39:32,599 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 21:39:32,620 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5827544ab5cd4a7d9dd HTTP/1.1" 200 None 2026-03-26 21:39:32,622 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 21:39:32,642 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c304846778148209fd69a6 HTTP/1.1" 200 None 2026-03-26 21:39:32,647 - lava_callback - ERROR - Error processing callback for node 69c304846778148209fd69a6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:54806 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41048 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49266 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44018 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53874 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53876 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53878 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51752 - "GET / HTTP/1.0" 200 OK 2026-03-26 21:40:32,611 - lava_callback - INFO - Received callback for node 69c304c76778148209fd69b0 INFO: 172.21.0.1:51140 - "POST /node/69c304c76778148209fd69b0 HTTP/1.0" 202 Accepted 2026-03-26 21:40:32,616 - lava_callback - INFO - Starting async job submit for node 69c304c76778148209fd69b0 2026-03-26 21:40:32,617 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 21:40:32,650 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c304c76778148209fd69b0 HTTP/1.1" 200 None 2026-03-26 21:40:32,651 - lava_callback - DEBUG - Processing node 69c304c76778148209fd69b0: baseline-arm-clabbe 2026-03-26 21:40:32,651 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c304c76778148209fd69b0 2026-03-26 21:40:32,652 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 21:40:32,666 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c304c76778148209fd69b0 HTTP/1.1" 200 None 2026-03-26 21:40:32,667 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 21:40:32,680 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5737544ab5cd4a7d9ce HTTP/1.1" 200 None 2026-03-26 21:40:32,681 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 21:40:32,702 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c304c76778148209fd69b0 HTTP/1.1" 200 None 2026-03-26 21:40:32,708 - lava_callback - ERROR - Error processing callback for node 69c304c76778148209fd69b0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:51156 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50844 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38434 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38222 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57718 - "GET / HTTP/1.0" 200 OK 2026-03-26 21:41:12,620 - lava_callback - INFO - Received callback for node 69c304ef6778148209fd69bc INFO: 172.21.0.1:57726 - "POST /node/69c304ef6778148209fd69bc HTTP/1.0" 202 Accepted 2026-03-26 21:41:12,626 - lava_callback - INFO - Starting async job submit for node 69c304ef6778148209fd69bc 2026-03-26 21:41:12,626 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 21:41:12,641 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c304ef6778148209fd69bc HTTP/1.1" 200 None 2026-03-26 21:41:12,642 - lava_callback - DEBUG - Processing node 69c304ef6778148209fd69bc: baseline-arm-clabbe 2026-03-26 21:41:12,642 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c304ef6778148209fd69bc 2026-03-26 21:41:12,643 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 21:41:12,666 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c304ef6778148209fd69bc HTTP/1.1" 200 None 2026-03-26 21:41:12,668 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 21:41:12,681 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5737544ab5cd4a7d9ce HTTP/1.1" 200 None 2026-03-26 21:41:12,682 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 21:41:12,705 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c304ef6778148209fd69bc HTTP/1.1" 200 None 2026-03-26 21:41:12,712 - lava_callback - ERROR - Error processing callback for node 69c304ef6778148209fd69bc: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:57728 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57744 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56118 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41712 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59018 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42754 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44544 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44406 - "GET / HTTP/1.0" 200 OK 2026-03-26 21:42:12,635 - lava_callback - INFO - Received callback for node 69c305246778148209fd69cc INFO: 172.21.0.1:44408 - "POST /node/69c305246778148209fd69cc HTTP/1.0" 202 Accepted 2026-03-26 21:42:12,641 - lava_callback - INFO - Starting async job submit for node 69c305246778148209fd69cc 2026-03-26 21:42:12,641 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 21:42:12,655 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c305246778148209fd69cc HTTP/1.1" 200 None 2026-03-26 21:42:12,656 - lava_callback - DEBUG - Processing node 69c305246778148209fd69cc: baseline-arm64-clabbe 2026-03-26 21:42:12,657 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c305246778148209fd69cc 2026-03-26 21:42:12,657 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 21:42:12,670 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c305246778148209fd69cc HTTP/1.1" 200 None 2026-03-26 21:42:12,672 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 21:42:12,688 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5827544ab5cd4a7d9dd HTTP/1.1" 200 None 2026-03-26 21:42:12,690 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 21:42:12,725 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c305246778148209fd69cc HTTP/1.1" 200 None 2026-03-26 21:42:12,726 - lava_callback - ERROR - Error processing callback for node 69c305246778148209fd69cc: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:44410 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44420 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36352 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43030 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57236 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41738 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55596 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60022 - "GET / HTTP/1.0" 200 OK 2026-03-26 21:43:12,678 - lava_callback - INFO - Received callback for node 69c305666778148209fd69d5 INFO: 172.21.0.1:60024 - "POST /node/69c305666778148209fd69d5 HTTP/1.0" 202 Accepted 2026-03-26 21:43:12,683 - lava_callback - INFO - Starting async job submit for node 69c305666778148209fd69d5 2026-03-26 21:43:12,684 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 21:43:12,699 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c305666778148209fd69d5 HTTP/1.1" 200 None 2026-03-26 21:43:12,700 - lava_callback - DEBUG - Processing node 69c305666778148209fd69d5: baseline-arm-clabbe 2026-03-26 21:43:12,700 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c305666778148209fd69d5 2026-03-26 21:43:12,701 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 21:43:12,717 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c305666778148209fd69d5 HTTP/1.1" 200 None 2026-03-26 21:43:12,718 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 21:43:12,731 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5737544ab5cd4a7d9ce HTTP/1.1" 200 None 2026-03-26 21:43:12,733 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 21:43:12,754 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c305666778148209fd69d5 HTTP/1.1" 200 None 2026-03-26 21:43:12,755 - lava_callback - ERROR - Error processing callback for node 69c305666778148209fd69d5: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:60036 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60042 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53140 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48594 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53374 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35376 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45848 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33664 - "GET / HTTP/1.0" 200 OK 2026-03-26 21:44:12,730 - lava_callback - INFO - Received callback for node 69c3059c6778148209fd69e3 INFO: 172.21.0.1:33670 - "POST /node/69c3059c6778148209fd69e3 HTTP/1.0" 202 Accepted 2026-03-26 21:44:12,736 - lava_callback - INFO - Starting async job submit for node 69c3059c6778148209fd69e3 2026-03-26 21:44:12,736 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 21:44:12,751 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3059c6778148209fd69e3 HTTP/1.1" 200 None 2026-03-26 21:44:12,751 - lava_callback - DEBUG - Processing node 69c3059c6778148209fd69e3: baseline-arm64-clabbe 2026-03-26 21:44:12,752 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3059c6778148209fd69e3 2026-03-26 21:44:12,752 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 21:44:12,766 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3059c6778148209fd69e3 HTTP/1.1" 200 None 2026-03-26 21:44:12,768 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 21:44:12,781 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5827544ab5cd4a7d9dd HTTP/1.1" 200 None 2026-03-26 21:44:12,782 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 21:44:12,809 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3059c6778148209fd69e3 HTTP/1.1" 200 None 2026-03-26 21:44:12,810 - lava_callback - ERROR - Error processing callback for node 69c3059c6778148209fd69e3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:33686 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33700 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48640 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37666 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43584 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41994 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33696 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42950 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42966 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42980 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43006 - "GET / HTTP/1.0" 200 OK 2026-03-26 21:45:32,759 - lava_callback - INFO - Received callback for node 69c305ef6778148209fd69ec INFO: 172.21.0.1:42082 - "POST /node/69c305ef6778148209fd69ec HTTP/1.0" 202 Accepted 2026-03-26 21:45:32,765 - lava_callback - INFO - Starting async job submit for node 69c305ef6778148209fd69ec 2026-03-26 21:45:32,766 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 21:45:32,788 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c305ef6778148209fd69ec HTTP/1.1" 200 None 2026-03-26 21:45:32,789 - lava_callback - DEBUG - Processing node 69c305ef6778148209fd69ec: baseline-arm64-clabbe 2026-03-26 21:45:32,789 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c305ef6778148209fd69ec 2026-03-26 21:45:32,789 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 21:45:32,803 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c305ef6778148209fd69ec HTTP/1.1" 200 None 2026-03-26 21:45:32,804 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 21:45:32,817 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5827544ab5cd4a7d9dd HTTP/1.1" 200 None 2026-03-26 21:45:32,818 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 21:45:32,839 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c305ef6778148209fd69ec HTTP/1.1" 200 None 2026-03-26 21:45:32,840 - lava_callback - ERROR - Error processing callback for node 69c305ef6778148209fd69ec: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:42098 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45384 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56728 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48590 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44138 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44154 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44158 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59968 - "GET / HTTP/1.0" 200 OK 2026-03-26 21:46:32,832 - lava_callback - INFO - Received callback for node 69c306336778148209fd69fa INFO: 172.21.0.1:42014 - "POST /node/69c306336778148209fd69fa HTTP/1.0" 202 Accepted 2026-03-26 21:46:32,837 - lava_callback - INFO - Starting async job submit for node 69c306336778148209fd69fa 2026-03-26 21:46:32,838 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 21:46:32,853 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c306336778148209fd69fa HTTP/1.1" 200 None 2026-03-26 21:46:32,854 - lava_callback - DEBUG - Processing node 69c306336778148209fd69fa: baseline-arm-clabbe 2026-03-26 21:46:32,855 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c306336778148209fd69fa 2026-03-26 21:46:32,855 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 21:46:32,868 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c306336778148209fd69fa HTTP/1.1" 200 None 2026-03-26 21:46:32,869 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 21:46:32,884 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5737544ab5cd4a7d9ce HTTP/1.1" 200 None 2026-03-26 21:46:32,886 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 21:46:32,906 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c306336778148209fd69fa HTTP/1.1" 200 None 2026-03-26 21:46:32,914 - lava_callback - ERROR - Error processing callback for node 69c306336778148209fd69fa: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:42018 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45568 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35320 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57810 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43250 - "GET / HTTP/1.0" 200 OK 2026-03-26 21:47:12,817 - lava_callback - INFO - Received callback for node 69c3065c6778148209fd6a02 INFO: 172.21.0.1:43252 - "POST /node/69c3065c6778148209fd6a02 HTTP/1.0" 202 Accepted 2026-03-26 21:47:12,822 - lava_callback - INFO - Starting async job submit for node 69c3065c6778148209fd6a02 2026-03-26 21:47:12,823 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 21:47:12,838 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3065c6778148209fd6a02 HTTP/1.1" 200 None 2026-03-26 21:47:12,839 - lava_callback - DEBUG - Processing node 69c3065c6778148209fd6a02: baseline-arm-clabbe 2026-03-26 21:47:12,839 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3065c6778148209fd6a02 2026-03-26 21:47:12,840 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 21:47:12,853 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3065c6778148209fd6a02 HTTP/1.1" 200 None 2026-03-26 21:47:12,854 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 21:47:12,872 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5737544ab5cd4a7d9ce HTTP/1.1" 200 None 2026-03-26 21:47:12,874 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 21:47:12,899 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3065c6778148209fd6a02 HTTP/1.1" 200 None 2026-03-26 21:47:12,900 - lava_callback - ERROR - Error processing callback for node 69c3065c6778148209fd6a02: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:43254 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43260 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37648 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45286 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54762 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37132 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37554 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41422 - "GET / HTTP/1.0" 200 OK 2026-03-26 21:48:12,900 - lava_callback - INFO - Received callback for node 69c306946778148209fd6a13 INFO: 172.21.0.1:41438 - "POST /node/69c306946778148209fd6a13 HTTP/1.0" 202 Accepted 2026-03-26 21:48:12,906 - lava_callback - INFO - Starting async job submit for node 69c306946778148209fd6a13 2026-03-26 21:48:12,907 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 21:48:12,920 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c306946778148209fd6a13 HTTP/1.1" 200 None 2026-03-26 21:48:12,921 - lava_callback - DEBUG - Processing node 69c306946778148209fd6a13: baseline-arm64-clabbe 2026-03-26 21:48:12,921 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c306946778148209fd6a13 2026-03-26 21:48:12,922 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 21:48:12,948 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c306946778148209fd6a13 HTTP/1.1" 200 None 2026-03-26 21:48:12,949 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 21:48:12,962 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5827544ab5cd4a7d9dd HTTP/1.1" 200 None 2026-03-26 21:48:12,964 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 21:48:12,984 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c306946778148209fd6a13 HTTP/1.1" 200 None 2026-03-26 21:48:12,991 - lava_callback - ERROR - Error processing callback for node 69c306946778148209fd6a13: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:41444 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41450 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35284 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33612 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50506 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58012 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43464 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51414 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51416 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51418 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49232 - "GET / HTTP/1.0" 200 OK 2026-03-26 21:49:32,968 - lava_callback - INFO - Received callback for node 69c306e86778148209fd6a20 INFO: 172.21.0.1:55206 - "POST /node/69c306e86778148209fd6a20 HTTP/1.0" 202 Accepted 2026-03-26 21:49:32,974 - lava_callback - INFO - Starting async job submit for node 69c306e86778148209fd6a20 2026-03-26 21:49:32,975 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 21:49:32,990 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c306e86778148209fd6a20 HTTP/1.1" 200 None 2026-03-26 21:49:32,991 - lava_callback - DEBUG - Processing node 69c306e86778148209fd6a20: baseline-arm64-clabbe 2026-03-26 21:49:32,991 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c306e86778148209fd6a20 2026-03-26 21:49:32,992 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 21:49:33,006 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c306e86778148209fd6a20 HTTP/1.1" 200 None 2026-03-26 21:49:33,007 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 21:49:33,021 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5827544ab5cd4a7d9dd HTTP/1.1" 200 None 2026-03-26 21:49:33,023 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 21:49:33,044 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c306e86778148209fd6a20 HTTP/1.1" 200 None 2026-03-26 21:49:33,049 - lava_callback - ERROR - Error processing callback for node 69c306e86778148209fd6a20: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:55220 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34670 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33962 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33966 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33970 - "GET /favicon.ico HTTP/1.0" 404 Not Found INFO: 172.21.0.1:58036 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57026 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57030 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57044 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46226 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49614 - "GET /.well-known/security.txt HTTP/1.0" 404 Not Found INFO: 172.21.0.1:49622 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33496 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38278 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58396 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56648 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56654 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56668 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57398 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32772 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39612 - "GET / HTTP/1.0" 200 OK 2026-03-26 21:51:53,020 - lava_callback - INFO - Received callback for node 69c307636778148209fd6a35 INFO: 172.21.0.1:57156 - "POST /node/69c307636778148209fd6a35 HTTP/1.0" 202 Accepted 2026-03-26 21:51:53,025 - lava_callback - INFO - Starting async job submit for node 69c307636778148209fd6a35 2026-03-26 21:51:53,026 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 21:51:53,062 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c307636778148209fd6a35 HTTP/1.1" 200 None 2026-03-26 21:51:53,063 - lava_callback - DEBUG - Processing node 69c307636778148209fd6a35: baseline-arm64-clabbe 2026-03-26 21:51:53,063 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c307636778148209fd6a35 2026-03-26 21:51:53,063 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 21:51:53,076 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c307636778148209fd6a35 HTTP/1.1" 200 None 2026-03-26 21:51:53,078 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 21:51:53,090 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5827544ab5cd4a7d9dd HTTP/1.1" 200 None 2026-03-26 21:51:53,092 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 21:51:53,111 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c307636778148209fd6a35 HTTP/1.1" 200 None 2026-03-26 21:51:53,112 - lava_callback - ERROR - Error processing callback for node 69c307636778148209fd6a35: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:57172 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50514 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55644 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55660 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55662 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60498 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60296 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59812 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44964 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36914 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40298 - "GET / HTTP/1.0" 200 OK 2026-03-26 21:53:13,032 - lava_callback - INFO - Received callback for node 69c307b76778148209fd6a41 INFO: 172.21.0.1:40304 - "POST /node/69c307b76778148209fd6a41 HTTP/1.0" 202 Accepted 2026-03-26 21:53:13,038 - lava_callback - INFO - Starting async job submit for node 69c307b76778148209fd6a41 2026-03-26 21:53:13,039 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 21:53:13,054 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c307b76778148209fd6a41 HTTP/1.1" 200 None 2026-03-26 21:53:13,055 - lava_callback - DEBUG - Processing node 69c307b76778148209fd6a41: baseline-arm64-clabbe 2026-03-26 21:53:13,055 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c307b76778148209fd6a41 2026-03-26 21:53:13,056 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 21:53:13,070 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c307b76778148209fd6a41 HTTP/1.1" 200 None 2026-03-26 21:53:13,072 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 21:53:13,092 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5827544ab5cd4a7d9dd HTTP/1.1" 200 None 2026-03-26 21:53:13,094 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 21:53:13,114 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c307b76778148209fd6a41 HTTP/1.1" 200 None 2026-03-26 21:53:13,116 - lava_callback - ERROR - Error processing callback for node 69c307b76778148209fd6a41: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:40310 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40314 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57064 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60682 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46878 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46894 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46900 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47652 - "GET /favicon.ico HTTP/1.0" 404 Not Found INFO: 172.21.0.1:47656 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46342 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52310 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52316 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52318 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43000 - "GET / HTTP/1.0" 200 OK 2026-03-26 21:54:33,131 - lava_callback - INFO - Received callback for node 69c308096778148209fd6a47 INFO: 172.21.0.1:40292 - "POST /node/69c308096778148209fd6a47 HTTP/1.0" 202 Accepted 2026-03-26 21:54:33,137 - lava_callback - INFO - Starting async job submit for node 69c308096778148209fd6a47 2026-03-26 21:54:33,137 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 21:54:33,152 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c308096778148209fd6a47 HTTP/1.1" 200 None 2026-03-26 21:54:33,152 - lava_callback - DEBUG - Processing node 69c308096778148209fd6a47: baseline-arm64-clabbe 2026-03-26 21:54:33,153 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c308096778148209fd6a47 2026-03-26 21:54:33,153 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 21:54:33,169 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c308096778148209fd6a47 HTTP/1.1" 200 None 2026-03-26 21:54:33,170 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 21:54:33,208 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5827544ab5cd4a7d9dd HTTP/1.1" 200 None 2026-03-26 21:54:33,209 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 21:54:33,252 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c308096778148209fd6a47 HTTP/1.1" 200 None 2026-03-26 21:54:33,254 - lava_callback - ERROR - Error processing callback for node 69c308096778148209fd6a47: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:40296 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52496 - "GET /sitemap.xml HTTP/1.0" 404 Not Found INFO: 172.21.0.1:52498 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47244 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57102 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36936 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36942 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36958 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47898 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45744 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43034 - "GET / HTTP/1.0" 200 OK 2026-03-26 21:55:53,283 - lava_callback - INFO - Received callback for node 69c3085e6778148209fd6a55 INFO: 172.21.0.1:45094 - "POST /node/69c3085e6778148209fd6a55 HTTP/1.0" 202 Accepted 2026-03-26 21:55:53,289 - lava_callback - INFO - Starting async job submit for node 69c3085e6778148209fd6a55 2026-03-26 21:55:53,290 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 21:55:53,319 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3085e6778148209fd6a55 HTTP/1.1" 200 None 2026-03-26 21:55:53,320 - lava_callback - DEBUG - Processing node 69c3085e6778148209fd6a55: baseline-arm64-clabbe 2026-03-26 21:55:53,320 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3085e6778148209fd6a55 2026-03-26 21:55:53,321 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 21:55:53,343 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3085e6778148209fd6a55 HTTP/1.1" 200 None 2026-03-26 21:55:53,345 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 21:55:53,376 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5827544ab5cd4a7d9dd HTTP/1.1" 200 None 2026-03-26 21:55:53,378 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 21:55:53,399 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3085e6778148209fd6a55 HTTP/1.1" 200 None 2026-03-26 21:55:53,405 - lava_callback - ERROR - Error processing callback for node 69c3085e6778148209fd6a55: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:45100 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55466 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54102 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54116 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54126 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58124 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56980 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40962 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47708 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42536 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52760 - "GET / HTTP/1.0" 200 OK 2026-03-26 21:57:13,383 - lava_callback - INFO - Received callback for node 69c308b06778148209fd6a63 INFO: 172.21.0.1:52776 - "POST /node/69c308b06778148209fd6a63 HTTP/1.0" 202 Accepted 2026-03-26 21:57:13,389 - lava_callback - INFO - Starting async job submit for node 69c308b06778148209fd6a63 2026-03-26 21:57:13,389 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 21:57:13,404 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c308b06778148209fd6a63 HTTP/1.1" 200 None 2026-03-26 21:57:13,405 - lava_callback - DEBUG - Processing node 69c308b06778148209fd6a63: baseline-arm64-clabbe 2026-03-26 21:57:13,405 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c308b06778148209fd6a63 2026-03-26 21:57:13,406 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 21:57:13,432 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c308b06778148209fd6a63 HTTP/1.1" 200 None 2026-03-26 21:57:13,433 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 21:57:13,446 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5827544ab5cd4a7d9dd HTTP/1.1" 200 None 2026-03-26 21:57:13,448 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 21:57:13,477 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c308b06778148209fd6a63 HTTP/1.1" 200 None 2026-03-26 21:57:13,479 - lava_callback - ERROR - Error processing callback for node 69c308b06778148209fd6a63: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:52778 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52788 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42258 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46078 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60802 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49200 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59958 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53410 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53416 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53422 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47478 - "GET / HTTP/1.0" 200 OK 2026-03-26 21:58:33,302 - lava_callback - INFO - Received callback for node 69c309026778148209fd6a75 INFO: 172.21.0.1:44386 - "POST /node/69c309026778148209fd6a75 HTTP/1.0" 202 Accepted 2026-03-26 21:58:33,308 - lava_callback - INFO - Starting async job submit for node 69c309026778148209fd6a75 2026-03-26 21:58:33,309 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 21:58:33,323 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c309026778148209fd6a75 HTTP/1.1" 200 None 2026-03-26 21:58:33,324 - lava_callback - DEBUG - Processing node 69c309026778148209fd6a75: baseline-arm64-clabbe 2026-03-26 21:58:33,324 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c309026778148209fd6a75 2026-03-26 21:58:33,325 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 21:58:33,353 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c309026778148209fd6a75 HTTP/1.1" 200 None 2026-03-26 21:58:33,354 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 21:58:33,368 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5827544ab5cd4a7d9dd HTTP/1.1" 200 None 2026-03-26 21:58:33,370 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 21:58:33,411 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c309026778148209fd6a75 HTTP/1.1" 200 None 2026-03-26 21:58:33,417 - lava_callback - ERROR - Error processing callback for node 69c309026778148209fd6a75: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:44396 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55934 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57150 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47064 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52694 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52710 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52724 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49320 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56870 - "GET / HTTP/1.0" 200 OK 2026-03-26 21:59:38,599 - lava_callback - INFO - Received callback for node 69c309416778148209fd6a7c INFO: 172.21.0.1:56886 - "POST /node/69c309416778148209fd6a7c HTTP/1.0" 202 Accepted 2026-03-26 21:59:38,604 - lava_callback - INFO - Starting async job submit for node 69c309416778148209fd6a7c 2026-03-26 21:59:38,605 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 21:59:38,623 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c309416778148209fd6a7c HTTP/1.1" 200 None 2026-03-26 21:59:38,624 - lava_callback - DEBUG - Processing node 69c309416778148209fd6a7c: baseline-arm-baylibre 2026-03-26 21:59:38,624 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c309416778148209fd6a7c 2026-03-26 21:59:38,625 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 21:59:38,638 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c309416778148209fd6a7c HTTP/1.1" 200 None 2026-03-26 21:59:38,639 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 21:59:38,655 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5737544ab5cd4a7d9ce HTTP/1.1" 200 None 2026-03-26 21:59:38,656 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 21:59:38,676 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c309416778148209fd6a7c HTTP/1.1" 200 None 2026-03-26 21:59:38,682 - lava_callback - ERROR - Error processing callback for node 69c309416778148209fd6a7c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:60718 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47092 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51776 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41992 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42004 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42008 - "GET / HTTP/1.0" 200 OK 2026-03-26 22:00:18,596 - lava_callback - INFO - Received callback for node 69c3096a6778148209fd6a86 INFO: 172.21.0.1:42014 - "POST /node/69c3096a6778148209fd6a86 HTTP/1.0" 202 Accepted 2026-03-26 22:00:18,601 - lava_callback - INFO - Starting async job submit for node 69c3096a6778148209fd6a86 2026-03-26 22:00:18,603 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:00:18,616 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3096a6778148209fd6a86 HTTP/1.1" 200 None 2026-03-26 22:00:18,617 - lava_callback - DEBUG - Processing node 69c3096a6778148209fd6a86: baseline-arm-baylibre 2026-03-26 22:00:18,618 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3096a6778148209fd6a86 2026-03-26 22:00:18,618 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:00:18,633 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3096a6778148209fd6a86 HTTP/1.1" 200 None 2026-03-26 22:00:18,634 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:00:18,647 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5737544ab5cd4a7d9ce HTTP/1.1" 200 None 2026-03-26 22:00:18,649 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:00:18,668 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3096a6778148209fd6a86 HTTP/1.1" 200 None 2026-03-26 22:00:18,673 - lava_callback - ERROR - Error processing callback for node 69c3096a6778148209fd6a86: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:38052 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52020 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41768 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42442 - "GET / HTTP/1.0" 200 OK 2026-03-26 22:00:58,634 - lava_callback - INFO - Received callback for node 69c309946778148209fd6a92 INFO: 172.21.0.1:42456 - "POST /node/69c309946778148209fd6a92 HTTP/1.0" 202 Accepted 2026-03-26 22:00:58,640 - lava_callback - INFO - Starting async job submit for node 69c309946778148209fd6a92 2026-03-26 22:00:58,640 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:00:58,654 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c309946778148209fd6a92 HTTP/1.1" 200 None 2026-03-26 22:00:58,655 - lava_callback - DEBUG - Processing node 69c309946778148209fd6a92: baseline-arm-baylibre 2026-03-26 22:00:58,655 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c309946778148209fd6a92 2026-03-26 22:00:58,656 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:00:58,669 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c309946778148209fd6a92 HTTP/1.1" 200 None 2026-03-26 22:00:58,670 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:00:58,683 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5737544ab5cd4a7d9ce HTTP/1.1" 200 None 2026-03-26 22:00:58,684 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:00:58,705 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c309946778148209fd6a92 HTTP/1.1" 200 None 2026-03-26 22:00:58,706 - lava_callback - ERROR - Error processing callback for node 69c309946778148209fd6a92: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:56632 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39698 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39706 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39722 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49778 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32788 - "GET / HTTP/1.0" 200 OK 2026-03-26 22:01:38,677 - lava_callback - INFO - Received callback for node 69c309bd6778148209fd6aa0 INFO: 172.21.0.1:32798 - "POST /node/69c309bd6778148209fd6aa0 HTTP/1.0" 202 Accepted 2026-03-26 22:01:38,684 - lava_callback - INFO - Starting async job submit for node 69c309bd6778148209fd6aa0 2026-03-26 22:01:38,685 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:01:38,700 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c309bd6778148209fd6aa0 HTTP/1.1" 200 None 2026-03-26 22:01:38,701 - lava_callback - DEBUG - Processing node 69c309bd6778148209fd6aa0: baseline-arm-baylibre 2026-03-26 22:01:38,702 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c309bd6778148209fd6aa0 2026-03-26 22:01:38,702 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:01:38,716 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c309bd6778148209fd6aa0 HTTP/1.1" 200 None 2026-03-26 22:01:38,717 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:01:38,731 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5737544ab5cd4a7d9ce HTTP/1.1" 200 None 2026-03-26 22:01:38,732 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:01:38,770 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c309bd6778148209fd6aa0 HTTP/1.1" 200 None 2026-03-26 22:01:38,771 - lava_callback - ERROR - Error processing callback for node 69c309bd6778148209fd6aa0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:46672 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60600 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51240 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60556 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60570 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60580 - "GET / HTTP/1.0" 200 OK 2026-03-26 22:02:18,762 - lava_callback - INFO - Received callback for node 69c309e46778148209fd6aac INFO: 172.21.0.1:60584 - "POST /node/69c309e46778148209fd6aac HTTP/1.0" 202 Accepted 2026-03-26 22:02:18,768 - lava_callback - INFO - Starting async job submit for node 69c309e46778148209fd6aac 2026-03-26 22:02:18,769 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:02:18,783 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c309e46778148209fd6aac HTTP/1.1" 200 None 2026-03-26 22:02:18,784 - lava_callback - DEBUG - Processing node 69c309e46778148209fd6aac: baseline-arm-baylibre 2026-03-26 22:02:18,784 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c309e46778148209fd6aac 2026-03-26 22:02:18,785 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:02:18,798 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c309e46778148209fd6aac HTTP/1.1" 200 None 2026-03-26 22:02:18,799 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:02:18,813 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5737544ab5cd4a7d9ce HTTP/1.1" 200 None 2026-03-26 22:02:18,814 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:02:18,836 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c309e46778148209fd6aac HTTP/1.1" 200 None 2026-03-26 22:02:18,841 - lava_callback - ERROR - Error processing callback for node 69c309e46778148209fd6aac: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:60796 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60080 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44322 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42624 - "GET / HTTP/1.0" 200 OK 2026-03-26 22:02:58,756 - lava_callback - INFO - Received callback for node 69c30a0d6778148209fd6ab8 INFO: 172.21.0.1:42632 - "POST /node/69c30a0d6778148209fd6ab8 HTTP/1.0" 202 Accepted 2026-03-26 22:02:58,762 - lava_callback - INFO - Starting async job submit for node 69c30a0d6778148209fd6ab8 2026-03-26 22:02:58,762 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:02:58,776 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c30a0d6778148209fd6ab8 HTTP/1.1" 200 None 2026-03-26 22:02:58,777 - lava_callback - DEBUG - Processing node 69c30a0d6778148209fd6ab8: baseline-arm-baylibre 2026-03-26 22:02:58,777 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c30a0d6778148209fd6ab8 2026-03-26 22:02:58,778 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:02:58,792 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c30a0d6778148209fd6ab8 HTTP/1.1" 200 None 2026-03-26 22:02:58,794 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:02:58,815 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5737544ab5cd4a7d9ce HTTP/1.1" 200 None 2026-03-26 22:02:58,816 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:02:58,837 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c30a0d6778148209fd6ab8 HTTP/1.1" 200 None 2026-03-26 22:02:58,843 - lava_callback - ERROR - Error processing callback for node 69c30a0d6778148209fd6ab8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:45212 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53954 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53958 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53974 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44440 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59892 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56814 - "GET / HTTP/1.0" 200 OK 2026-03-26 22:03:53,544 - lava_callback - INFO - Received callback for node 69c30a3b6778148209fd6ac5 INFO: 172.21.0.1:33286 - "POST /node/69c30a3b6778148209fd6ac5 HTTP/1.0" 202 Accepted 2026-03-26 22:03:53,550 - lava_callback - INFO - Starting async job submit for node 69c30a3b6778148209fd6ac5 2026-03-26 22:03:53,551 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:03:53,571 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c30a3b6778148209fd6ac5 HTTP/1.1" 200 None 2026-03-26 22:03:53,572 - lava_callback - DEBUG - Processing node 69c30a3b6778148209fd6ac5: baseline-arm-clabbe 2026-03-26 22:03:53,573 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c30a3b6778148209fd6ac5 2026-03-26 22:03:53,573 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:03:53,586 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c30a3b6778148209fd6ac5 HTTP/1.1" 200 None 2026-03-26 22:03:53,588 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:03:53,611 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5737544ab5cd4a7d9ce HTTP/1.1" 200 None 2026-03-26 22:03:53,612 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:03:53,633 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c30a3b6778148209fd6ac5 HTTP/1.1" 200 None 2026-03-26 22:03:53,634 - lava_callback - ERROR - Error processing callback for node 69c30a3b6778148209fd6ac5: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:33302 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56148 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60908 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60910 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60922 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54210 - "GET / HTTP/1.0" 200 OK 2026-03-26 22:04:33,506 - lava_callback - INFO - Received callback for node 69c30a656778148209fd6ad5 INFO: 172.21.0.1:52830 - "POST /node/69c30a656778148209fd6ad5 HTTP/1.0" 202 Accepted 2026-03-26 22:04:33,512 - lava_callback - INFO - Starting async job submit for node 69c30a656778148209fd6ad5 2026-03-26 22:04:33,512 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:04:33,527 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c30a656778148209fd6ad5 HTTP/1.1" 200 None 2026-03-26 22:04:33,528 - lava_callback - DEBUG - Processing node 69c30a656778148209fd6ad5: baseline-arm-clabbe 2026-03-26 22:04:33,528 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c30a656778148209fd6ad5 2026-03-26 22:04:33,528 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:04:33,543 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c30a656778148209fd6ad5 HTTP/1.1" 200 None 2026-03-26 22:04:33,544 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:04:33,559 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5737544ab5cd4a7d9ce HTTP/1.1" 200 None 2026-03-26 22:04:33,560 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:04:33,584 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c30a656778148209fd6ad5 HTTP/1.1" 200 None 2026-03-26 22:04:33,594 - lava_callback - ERROR - Error processing callback for node 69c30a656778148209fd6ad5: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:52842 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40554 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58456 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39912 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40334 - "GET / HTTP/1.0" 200 OK 2026-03-26 22:05:13,598 - lava_callback - INFO - Received callback for node 69c30a8e6778148209fd6ae3 INFO: 172.21.0.1:40346 - "POST /node/69c30a8e6778148209fd6ae3 HTTP/1.0" 202 Accepted 2026-03-26 22:05:13,604 - lava_callback - INFO - Starting async job submit for node 69c30a8e6778148209fd6ae3 2026-03-26 22:05:13,605 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:05:13,619 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c30a8e6778148209fd6ae3 HTTP/1.1" 200 None 2026-03-26 22:05:13,620 - lava_callback - DEBUG - Processing node 69c30a8e6778148209fd6ae3: baseline-arm-clabbe 2026-03-26 22:05:13,620 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c30a8e6778148209fd6ae3 2026-03-26 22:05:13,621 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:05:13,634 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c30a8e6778148209fd6ae3 HTTP/1.1" 200 None 2026-03-26 22:05:13,636 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:05:13,648 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5737544ab5cd4a7d9ce HTTP/1.1" 200 None 2026-03-26 22:05:13,650 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:05:13,688 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c30a8e6778148209fd6ae3 HTTP/1.1" 200 None 2026-03-26 22:05:13,700 - lava_callback - ERROR - Error processing callback for node 69c30a8e6778148209fd6ae3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:40354 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40356 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38922 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60638 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57326 - "GET / HTTP/1.0" 200 OK 2026-03-26 22:05:53,611 - lava_callback - INFO - Received callback for node 69c30ab76778148209fd6aef INFO: 172.21.0.1:53322 - "POST /node/69c30ab76778148209fd6aef HTTP/1.0" 202 Accepted 2026-03-26 22:05:53,617 - lava_callback - INFO - Starting async job submit for node 69c30ab76778148209fd6aef 2026-03-26 22:05:53,618 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:05:53,632 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c30ab76778148209fd6aef HTTP/1.1" 200 None 2026-03-26 22:05:53,633 - lava_callback - DEBUG - Processing node 69c30ab76778148209fd6aef: baseline-arm-clabbe 2026-03-26 22:05:53,633 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c30ab76778148209fd6aef 2026-03-26 22:05:53,633 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:05:53,658 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c30ab76778148209fd6aef HTTP/1.1" 200 None 2026-03-26 22:05:53,660 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:05:53,674 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5737544ab5cd4a7d9ce HTTP/1.1" 200 None 2026-03-26 22:05:53,675 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:05:53,717 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c30ab76778148209fd6aef HTTP/1.1" 200 None 2026-03-26 22:05:53,726 - lava_callback - ERROR - Error processing callback for node 69c30ab76778148209fd6aef: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:53330 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58274 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35448 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35450 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35452 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52674 - "GET / HTTP/1.0" 200 OK 2026-03-26 22:06:33,671 - lava_callback - INFO - Received callback for node 69c30adf6778148209fd6aff INFO: 172.21.0.1:57132 - "POST /node/69c30adf6778148209fd6aff HTTP/1.0" 202 Accepted 2026-03-26 22:06:33,677 - lava_callback - INFO - Starting async job submit for node 69c30adf6778148209fd6aff 2026-03-26 22:06:33,678 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:06:33,693 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c30adf6778148209fd6aff HTTP/1.1" 200 None 2026-03-26 22:06:33,694 - lava_callback - DEBUG - Processing node 69c30adf6778148209fd6aff: baseline-arm-clabbe 2026-03-26 22:06:33,694 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c30adf6778148209fd6aff 2026-03-26 22:06:33,695 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:06:33,712 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c30adf6778148209fd6aff HTTP/1.1" 200 None 2026-03-26 22:06:33,714 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:06:33,727 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5737544ab5cd4a7d9ce HTTP/1.1" 200 None 2026-03-26 22:06:33,729 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:06:33,750 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c30adf6778148209fd6aff HTTP/1.1" 200 None 2026-03-26 22:06:33,751 - lava_callback - ERROR - Error processing callback for node 69c30adf6778148209fd6aff: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:57138 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46008 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34774 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53722 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48322 - "GET / HTTP/1.0" 200 OK 2026-03-26 22:07:13,670 - lava_callback - INFO - Received callback for node 69c30b086778148209fd6b0c INFO: 172.21.0.1:48330 - "POST /node/69c30b086778148209fd6b0c HTTP/1.0" 202 Accepted 2026-03-26 22:07:13,676 - lava_callback - INFO - Starting async job submit for node 69c30b086778148209fd6b0c 2026-03-26 22:07:13,676 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:07:13,700 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c30b086778148209fd6b0c HTTP/1.1" 200 None 2026-03-26 22:07:13,701 - lava_callback - DEBUG - Processing node 69c30b086778148209fd6b0c: baseline-arm-clabbe 2026-03-26 22:07:13,701 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c30b086778148209fd6b0c 2026-03-26 22:07:13,702 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:07:13,737 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c30b086778148209fd6b0c HTTP/1.1" 200 None 2026-03-26 22:07:13,738 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:07:13,751 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5737544ab5cd4a7d9ce HTTP/1.1" 200 None 2026-03-26 22:07:13,753 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:07:13,774 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c30b086778148209fd6b0c HTTP/1.1" 200 None 2026-03-26 22:07:13,775 - lava_callback - ERROR - Error processing callback for node 69c30b086778148209fd6b0c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:48338 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48354 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36478 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52770 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41320 - "GET / HTTP/1.0" 200 OK 2026-03-26 22:07:53,718 - lava_callback - INFO - Received callback for node 69c30b306778148209fd6b19 INFO: 172.21.0.1:40582 - "POST /node/69c30b306778148209fd6b19 HTTP/1.0" 202 Accepted 2026-03-26 22:07:53,724 - lava_callback - INFO - Starting async job submit for node 69c30b306778148209fd6b19 2026-03-26 22:07:53,724 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:07:53,739 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c30b306778148209fd6b19 HTTP/1.1" 200 None 2026-03-26 22:07:53,740 - lava_callback - DEBUG - Processing node 69c30b306778148209fd6b19: baseline-arm-clabbe 2026-03-26 22:07:53,740 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c30b306778148209fd6b19 2026-03-26 22:07:53,741 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:07:53,754 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c30b306778148209fd6b19 HTTP/1.1" 200 None 2026-03-26 22:07:53,755 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:07:53,768 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5737544ab5cd4a7d9ce HTTP/1.1" 200 None 2026-03-26 22:07:53,769 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:07:53,794 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c30b306778148209fd6b19 HTTP/1.1" 200 None 2026-03-26 22:07:53,801 - lava_callback - ERROR - Error processing callback for node 69c30b306778148209fd6b19: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:40586 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35698 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55722 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55730 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55738 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34214 - "GET / HTTP/1.0" 200 OK 2026-03-26 22:08:33,685 - lava_callback - INFO - Received callback for node 69c30b586778148209fd6b29 INFO: 172.21.0.1:50792 - "POST /node/69c30b586778148209fd6b29 HTTP/1.0" 202 Accepted 2026-03-26 22:08:33,692 - lava_callback - INFO - Starting async job submit for node 69c30b586778148209fd6b29 2026-03-26 22:08:33,693 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:08:33,707 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c30b586778148209fd6b29 HTTP/1.1" 200 None 2026-03-26 22:08:33,708 - lava_callback - DEBUG - Processing node 69c30b586778148209fd6b29: baseline-arm-clabbe 2026-03-26 22:08:33,708 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c30b586778148209fd6b29 2026-03-26 22:08:33,709 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:08:33,722 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c30b586778148209fd6b29 HTTP/1.1" 200 None 2026-03-26 22:08:33,723 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:08:33,736 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5737544ab5cd4a7d9ce HTTP/1.1" 200 None 2026-03-26 22:08:33,738 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:08:33,757 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c30b586778148209fd6b29 HTTP/1.1" 200 None 2026-03-26 22:08:33,762 - lava_callback - ERROR - Error processing callback for node 69c30b586778148209fd6b29: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:50798 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38754 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50576 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52330 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38180 - "GET / HTTP/1.0" 200 OK 2026-03-26 22:09:13,736 - lava_callback - INFO - Received callback for node 69c30b806778148209fd6b36 INFO: 172.21.0.1:38182 - "POST /node/69c30b806778148209fd6b36 HTTP/1.0" 202 Accepted 2026-03-26 22:09:13,741 - lava_callback - INFO - Starting async job submit for node 69c30b806778148209fd6b36 2026-03-26 22:09:13,742 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:09:13,758 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c30b806778148209fd6b36 HTTP/1.1" 200 None 2026-03-26 22:09:13,759 - lava_callback - DEBUG - Processing node 69c30b806778148209fd6b36: baseline-arm-clabbe 2026-03-26 22:09:13,759 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c30b806778148209fd6b36 2026-03-26 22:09:13,760 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:09:13,772 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c30b806778148209fd6b36 HTTP/1.1" 200 None 2026-03-26 22:09:13,774 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:09:13,788 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5737544ab5cd4a7d9ce HTTP/1.1" 200 None 2026-03-26 22:09:13,790 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:09:13,811 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c30b806778148209fd6b36 HTTP/1.1" 200 None 2026-03-26 22:09:13,812 - lava_callback - ERROR - Error processing callback for node 69c30b806778148209fd6b36: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:38190 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38200 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43556 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59456 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58218 - "GET / HTTP/1.0" 200 OK 2026-03-26 22:09:53,765 - lava_callback - INFO - Received callback for node 69c30ba96778148209fd6b43 INFO: 172.21.0.1:53756 - "POST /node/69c30ba96778148209fd6b43 HTTP/1.0" 202 Accepted 2026-03-26 22:09:53,770 - lava_callback - INFO - Starting async job submit for node 69c30ba96778148209fd6b43 2026-03-26 22:09:53,771 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:09:53,786 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c30ba96778148209fd6b43 HTTP/1.1" 200 None 2026-03-26 22:09:53,787 - lava_callback - DEBUG - Processing node 69c30ba96778148209fd6b43: baseline-arm-clabbe 2026-03-26 22:09:53,787 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c30ba96778148209fd6b43 2026-03-26 22:09:53,788 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:09:53,804 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c30ba96778148209fd6b43 HTTP/1.1" 200 None 2026-03-26 22:09:53,805 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:09:53,819 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5737544ab5cd4a7d9ce HTTP/1.1" 200 None 2026-03-26 22:09:53,821 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:09:53,841 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c30ba96778148209fd6b43 HTTP/1.1" 200 None 2026-03-26 22:09:53,847 - lava_callback - ERROR - Error processing callback for node 69c30ba96778148209fd6b43: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:53770 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52322 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53132 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53148 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53164 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44720 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46058 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41210 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51888 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46826 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50568 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50572 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50582 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52316 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60658 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58820 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57766 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38252 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38964 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38974 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38980 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34344 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53836 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51570 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45714 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50366 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56290 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56302 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56312 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49380 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56602 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50688 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52870 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47460 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34106 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34114 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34126 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46936 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55240 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49584 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60620 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43504 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44316 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44332 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44334 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52412 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46418 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57674 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38736 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56272 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42134 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42146 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42154 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36066 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44464 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40190 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52224 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56026 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60446 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60452 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60460 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40356 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44078 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52522 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57928 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39860 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45008 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45014 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45026 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33522 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44060 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54646 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44942 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60456 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48116 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48128 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48136 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57738 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45274 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52270 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57914 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35788 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49416 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49420 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49422 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46626 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53328 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57970 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44388 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38782 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56884 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56892 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56896 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46338 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51576 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57822 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42480 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43552 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56406 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56416 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56418 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60916 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48572 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51572 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51774 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35608 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40858 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40862 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40876 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41082 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50344 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33042 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51500 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42516 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53602 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53616 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53630 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60018 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55258 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60094 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53492 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47354 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56306 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56316 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56328 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49560 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60714 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44838 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44496 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54106 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46166 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46174 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46188 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46910 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40140 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59338 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42454 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47834 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47136 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47140 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47144 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48384 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44488 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47198 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36588 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47140 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55968 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55976 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55986 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38208 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45452 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42534 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34808 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57478 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59292 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59308 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59320 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42752 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59484 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58998 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50798 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45542 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55724 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55732 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55734 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44122 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50986 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35602 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46192 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36622 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36616 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36628 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36630 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42208 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38910 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49568 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60104 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56694 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60960 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60968 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60982 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53550 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34888 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47294 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46018 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50964 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59502 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59510 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59524 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42492 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54608 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48664 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57644 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56762 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48806 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48812 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48826 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51034 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39644 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55644 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49404 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33930 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54768 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54778 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54782 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38272 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39354 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54892 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55896 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59064 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39970 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39972 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39982 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44382 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56032 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56492 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33528 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54218 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49338 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49350 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49362 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39598 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34404 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36200 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44322 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56440 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54398 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54408 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54418 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56842 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32920 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48340 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40342 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44226 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36226 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36238 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36242 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51406 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50944 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50412 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35952 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35644 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37926 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37928 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37944 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48382 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36192 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48782 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57910 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55988 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56832 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56834 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56836 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33948 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40148 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52118 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39992 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50870 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50230 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50242 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50254 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56930 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50784 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33190 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33264 - "GET / HTTP/1.0" 200 OK 2026-03-26 22:43:01,173 - lava_callback - INFO - Received callback for node 69c313646778148209fd6b9d INFO: 172.21.0.1:59846 - "POST /node/69c313646778148209fd6b9d HTTP/1.0" 202 Accepted 2026-03-26 22:43:01,178 - lava_callback - INFO - Starting async job submit for node 69c313646778148209fd6b9d 2026-03-26 22:43:01,179 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:43:01,197 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c313646778148209fd6b9d HTTP/1.1" 200 None 2026-03-26 22:43:01,197 - lava_callback - DEBUG - Processing node 69c313646778148209fd6b9d: baseline-arm-baylibre 2026-03-26 22:43:01,198 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c313646778148209fd6b9d 2026-03-26 22:43:01,198 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:43:01,212 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c313646778148209fd6b9d HTTP/1.1" 200 None 2026-03-26 22:43:01,214 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:43:01,271 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b94b6778148209fd336c HTTP/1.1" 200 None 2026-03-26 22:43:01,273 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:43:01,341 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c313646778148209fd6b9d HTTP/1.1" 200 None 2026-03-26 22:43:01,342 - lava_callback - ERROR - Error processing callback for node 69c313646778148209fd6b9d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:59858 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55198 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55208 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55220 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41056 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37016 - "GET / HTTP/1.0" 200 OK 2026-03-26 22:43:41,066 - lava_callback - INFO - Received callback for node 69c3138b6778148209fd6bb2 INFO: 172.21.0.1:34060 - "POST /node/69c3138b6778148209fd6bb2 HTTP/1.0" 202 Accepted 2026-03-26 22:43:41,072 - lava_callback - INFO - Starting async job submit for node 69c3138b6778148209fd6bb2 2026-03-26 22:43:41,073 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:43:41,087 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3138b6778148209fd6bb2 HTTP/1.1" 200 None 2026-03-26 22:43:41,088 - lava_callback - DEBUG - Processing node 69c3138b6778148209fd6bb2: baseline-arm-baylibre 2026-03-26 22:43:41,088 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3138b6778148209fd6bb2 2026-03-26 22:43:41,089 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:43:41,102 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3138b6778148209fd6bb2 HTTP/1.1" 200 None 2026-03-26 22:43:41,104 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:43:41,135 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b94b6778148209fd336c HTTP/1.1" 200 None 2026-03-26 22:43:41,137 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:43:41,158 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3138b6778148209fd6bb2 HTTP/1.1" 200 None 2026-03-26 22:43:41,163 - lava_callback - ERROR - Error processing callback for node 69c3138b6778148209fd6bb2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:34068 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47460 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60608 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56018 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56024 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56034 - "GET / HTTP/1.0" 200 OK 2026-03-26 22:44:21,249 - lava_callback - INFO - Received callback for node 69c313b26778148209fd6bbb INFO: 172.21.0.1:56572 - "POST /node/69c313b26778148209fd6bbb HTTP/1.0" 202 Accepted 2026-03-26 22:44:21,255 - lava_callback - INFO - Starting async job submit for node 69c313b26778148209fd6bbb 2026-03-26 22:44:21,256 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:44:21,278 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c313b26778148209fd6bbb HTTP/1.1" 200 None 2026-03-26 22:44:21,279 - lava_callback - DEBUG - Processing node 69c313b26778148209fd6bbb: baseline-arm-baylibre 2026-03-26 22:44:21,280 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c313b26778148209fd6bbb 2026-03-26 22:44:21,280 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:44:21,295 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c313b26778148209fd6bbb HTTP/1.1" 200 None 2026-03-26 22:44:21,297 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:44:21,311 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b94b6778148209fd336c HTTP/1.1" 200 None 2026-03-26 22:44:21,313 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:44:21,334 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c313b26778148209fd6bbb HTTP/1.1" 200 None 2026-03-26 22:44:21,336 - lava_callback - ERROR - Error processing callback for node 69c313b26778148209fd6bbb: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:56578 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44382 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47710 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48824 - "GET / HTTP/1.0" 200 OK 2026-03-26 22:45:01,150 - lava_callback - INFO - Received callback for node 69c313d96778148209fd6bc5 INFO: 172.21.0.1:45884 - "POST /node/69c313d96778148209fd6bc5 HTTP/1.0" 202 Accepted 2026-03-26 22:45:01,155 - lava_callback - INFO - Starting async job submit for node 69c313d96778148209fd6bc5 2026-03-26 22:45:01,156 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:45:01,170 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c313d96778148209fd6bc5 HTTP/1.1" 200 None 2026-03-26 22:45:01,171 - lava_callback - DEBUG - Processing node 69c313d96778148209fd6bc5: baseline-arm-baylibre 2026-03-26 22:45:01,171 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c313d96778148209fd6bc5 2026-03-26 22:45:01,172 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:45:01,185 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c313d96778148209fd6bc5 HTTP/1.1" 200 None 2026-03-26 22:45:01,186 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:45:01,199 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b94b6778148209fd336c HTTP/1.1" 200 None 2026-03-26 22:45:01,201 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:45:01,223 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c313d96778148209fd6bc5 HTTP/1.1" 200 None 2026-03-26 22:45:01,224 - lava_callback - ERROR - Error processing callback for node 69c313d96778148209fd6bc5: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:45896 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48290 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48304 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48314 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40390 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53290 - "GET / HTTP/1.0" 200 OK 2026-03-26 22:45:41,351 - lava_callback - INFO - Received callback for node 69c314006778148209fd6bcf INFO: 172.21.0.1:58128 - "POST /node/69c314006778148209fd6bcf HTTP/1.0" 202 Accepted 2026-03-26 22:45:41,356 - lava_callback - INFO - Starting async job submit for node 69c314006778148209fd6bcf 2026-03-26 22:45:41,357 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:45:41,371 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c314006778148209fd6bcf HTTP/1.1" 200 None 2026-03-26 22:45:41,372 - lava_callback - DEBUG - Processing node 69c314006778148209fd6bcf: baseline-arm-baylibre 2026-03-26 22:45:41,372 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c314006778148209fd6bcf 2026-03-26 22:45:41,373 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:45:41,389 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c314006778148209fd6bcf HTTP/1.1" 200 None 2026-03-26 22:45:41,391 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:45:41,409 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b94b6778148209fd336c HTTP/1.1" 200 None 2026-03-26 22:45:41,410 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:45:41,432 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c314006778148209fd6bcf HTTP/1.1" 200 None 2026-03-26 22:45:41,441 - lava_callback - ERROR - Error processing callback for node 69c314006778148209fd6bcf: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:58142 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59708 - "GET / HTTP/1.0" 200 OK 2026-03-26 22:46:01,189 - lava_callback - INFO - Received callback for node 69c314276778148209fd6be2 INFO: 172.21.0.1:56302 - "POST /node/69c314276778148209fd6be2 HTTP/1.0" 202 Accepted 2026-03-26 22:46:01,194 - lava_callback - INFO - Starting async job submit for node 69c314276778148209fd6be2 2026-03-26 22:46:01,195 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:46:01,210 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c314276778148209fd6be2 HTTP/1.1" 200 None 2026-03-26 22:46:01,211 - lava_callback - DEBUG - Processing node 69c314276778148209fd6be2: baseline-arm-baylibre 2026-03-26 22:46:01,211 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c314276778148209fd6be2 2026-03-26 22:46:01,211 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:46:01,224 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c314276778148209fd6be2 HTTP/1.1" 200 None 2026-03-26 22:46:01,226 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:46:01,240 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b94b6778148209fd336c HTTP/1.1" 200 None 2026-03-26 22:46:01,241 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:46:01,299 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c314276778148209fd6be2 HTTP/1.1" 200 None 2026-03-26 22:46:01,301 - lava_callback - ERROR - Error processing callback for node 69c314276778148209fd6be2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:56308 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55436 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55440 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55448 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38772 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42840 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41938 - "GET /node/69870f9d6e077e4126a7edc5%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.21.0.1:41948 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45006 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44836 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41314 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41324 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41334 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41576 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54482 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59592 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39648 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37914 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54006 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54022 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-26 22:48:15,789 - lava_callback - INFO - Received callback for node 69c3149d6778148209fd6c07 INFO: 172.21.0.1:54032 - "POST /node/69c3149d6778148209fd6c07 HTTP/1.0" 202 Accepted 2026-03-26 22:48:15,795 - lava_callback - INFO - Starting async job submit for node 69c3149d6778148209fd6c07 2026-03-26 22:48:15,796 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:48:15,822 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3149d6778148209fd6c07 HTTP/1.1" 200 None 2026-03-26 22:48:15,823 - lava_callback - DEBUG - Processing node 69c3149d6778148209fd6c07: baseline-arm-clabbe 2026-03-26 22:48:15,823 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3149d6778148209fd6c07 2026-03-26 22:48:15,824 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:48:15,837 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3149d6778148209fd6c07 HTTP/1.1" 200 None 2026-03-26 22:48:15,838 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:48:15,852 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b94b6778148209fd336c HTTP/1.1" 200 None 2026-03-26 22:48:15,854 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:48:15,876 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3149d6778148209fd6c07 HTTP/1.1" 200 None 2026-03-26 22:48:15,877 - lava_callback - ERROR - Error processing callback for node 69c3149d6778148209fd6c07: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:54044 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58574 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34656 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60346 - "GET / HTTP/1.0" 200 OK 2026-03-26 22:48:55,494 - lava_callback - INFO - Received callback for node 69c314c56778148209fd6c0e INFO: 172.21.0.1:51486 - "POST /node/69c314c56778148209fd6c0e HTTP/1.0" 202 Accepted 2026-03-26 22:48:55,500 - lava_callback - INFO - Starting async job submit for node 69c314c56778148209fd6c0e 2026-03-26 22:48:55,501 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:48:55,518 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c314c56778148209fd6c0e HTTP/1.1" 200 None 2026-03-26 22:48:55,519 - lava_callback - DEBUG - Processing node 69c314c56778148209fd6c0e: baseline-arm-clabbe 2026-03-26 22:48:55,519 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c314c56778148209fd6c0e 2026-03-26 22:48:55,520 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:48:55,533 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c314c56778148209fd6c0e HTTP/1.1" 200 None 2026-03-26 22:48:55,534 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:48:55,547 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b94b6778148209fd336c HTTP/1.1" 200 None 2026-03-26 22:48:55,549 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:48:55,574 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c314c56778148209fd6c0e HTTP/1.1" 200 None 2026-03-26 22:48:55,576 - lava_callback - ERROR - Error processing callback for node 69c314c56778148209fd6c0e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:51488 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55712 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37152 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37154 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37156 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37580 - "GET / HTTP/1.0" 200 OK 2026-03-26 22:49:35,564 - lava_callback - INFO - Received callback for node 69c314ed6778148209fd6c1b INFO: 172.21.0.1:36762 - "POST /node/69c314ed6778148209fd6c1b HTTP/1.0" 202 Accepted 2026-03-26 22:49:35,570 - lava_callback - INFO - Starting async job submit for node 69c314ed6778148209fd6c1b 2026-03-26 22:49:35,571 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:49:35,588 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c314ed6778148209fd6c1b HTTP/1.1" 200 None 2026-03-26 22:49:35,589 - lava_callback - DEBUG - Processing node 69c314ed6778148209fd6c1b: baseline-arm-clabbe 2026-03-26 22:49:35,589 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c314ed6778148209fd6c1b 2026-03-26 22:49:35,590 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:49:35,604 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c314ed6778148209fd6c1b HTTP/1.1" 200 None 2026-03-26 22:49:35,605 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:49:35,626 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b94b6778148209fd336c HTTP/1.1" 200 None 2026-03-26 22:49:35,627 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:49:35,648 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c314ed6778148209fd6c1b HTTP/1.1" 200 None 2026-03-26 22:49:35,650 - lava_callback - ERROR - Error processing callback for node 69c314ed6778148209fd6c1b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:36772 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50244 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41812 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40426 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40136 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40150 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-26 22:50:15,429 - lava_callback - INFO - Received callback for node 69c315146778148209fd6c2c INFO: 172.21.0.1:40164 - "POST /node/69c315146778148209fd6c2c HTTP/1.0" 202 Accepted 2026-03-26 22:50:15,438 - lava_callback - INFO - Starting async job submit for node 69c315146778148209fd6c2c 2026-03-26 22:50:15,440 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:50:15,469 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c315146778148209fd6c2c HTTP/1.1" 200 None 2026-03-26 22:50:15,470 - lava_callback - DEBUG - Processing node 69c315146778148209fd6c2c: baseline-arm-clabbe 2026-03-26 22:50:15,471 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c315146778148209fd6c2c 2026-03-26 22:50:15,471 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:50:15,496 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c315146778148209fd6c2c HTTP/1.1" 200 None 2026-03-26 22:50:15,499 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:50:15,527 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b94b6778148209fd336c HTTP/1.1" 200 None 2026-03-26 22:50:15,529 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:50:15,566 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c315146778148209fd6c2c HTTP/1.1" 200 None 2026-03-26 22:50:15,568 - lava_callback - ERROR - Error processing callback for node 69c315146778148209fd6c2c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:40168 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49434 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34910 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56684 - "GET / HTTP/1.0" 200 OK 2026-03-26 22:50:55,604 - lava_callback - INFO - Received callback for node 69c3153a6778148209fd6c38 INFO: 172.21.0.1:50426 - "POST /node/69c3153a6778148209fd6c38 HTTP/1.0" 202 Accepted 2026-03-26 22:50:55,612 - lava_callback - INFO - Starting async job submit for node 69c3153a6778148209fd6c38 2026-03-26 22:50:55,613 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:50:55,648 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3153a6778148209fd6c38 HTTP/1.1" 200 None 2026-03-26 22:50:55,650 - lava_callback - DEBUG - Processing node 69c3153a6778148209fd6c38: baseline-arm-clabbe 2026-03-26 22:50:55,650 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3153a6778148209fd6c38 2026-03-26 22:50:55,651 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:50:55,676 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3153a6778148209fd6c38 HTTP/1.1" 200 None 2026-03-26 22:50:55,678 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:50:55,706 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b94b6778148209fd336c HTTP/1.1" 200 None 2026-03-26 22:50:55,709 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:50:55,746 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3153a6778148209fd6c38 HTTP/1.1" 200 None 2026-03-26 22:50:55,747 - lava_callback - ERROR - Error processing callback for node 69c3153a6778148209fd6c38: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:50436 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51238 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53858 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53868 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53874 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38214 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53626 - "GET / HTTP/1.0" 200 OK 2026-03-26 22:51:46,452 - lava_callback - INFO - Received callback for node 69c5b864abf66a8f4d9bcc46 INFO: 172.21.0.1:52122 - "POST /node/69c5b864abf66a8f4d9bcc46 HTTP/1.0" 202 Accepted 2026-03-26 22:51:46,459 - lava_callback - INFO - Starting async job submit for node 69c5b864abf66a8f4d9bcc46 2026-03-26 22:51:46,478 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:51:46,496 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b864abf66a8f4d9bcc46 HTTP/1.1" 200 None 2026-03-26 22:51:46,496 - lava_callback - DEBUG - Processing node 69c5b864abf66a8f4d9bcc46: baseline-arm64 INFO: 172.21.0.1:52124 - "GET / HTTP/1.0" 200 OK 2026-03-26 22:51:46,647 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 22:51:46,769 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 22:51:46,771 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c5b864abf66a8f4d9bcc46/log.txt.gz 2026-03-26 22:51:46,910 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 22:51:47,084 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 22:51:47,086 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c5b864abf66a8f4d9bcc46/lava-logs.yaml 2026-03-26 22:51:47,091 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 22:51:47,232 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 22:51:47,234 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c5b864abf66a8f4d9bcc46/lava_callback.json.gz 2026-03-26 22:51:47,253 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:51:47,267 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b864abf66a8f4d9bcc46 HTTP/1.1" 200 None 2026-03-26 22:51:47,268 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:51:47,282 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-03-26 22:51:47,283 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:51:47,308 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5b864abf66a8f4d9bcc46 HTTP/1.1" 200 None 2026-03-26 22:51:47,309 - lava_callback - INFO - Completed processing callback for node 69c5b864abf66a8f4d9bcc46 2026-03-26 22:51:51,714 - lava_callback - INFO - Received callback for node 69c5b870abf66a8f4d9bcc69 INFO: 172.21.0.1:56028 - "POST /node/69c5b870abf66a8f4d9bcc69 HTTP/1.0" 202 Accepted 2026-03-26 22:51:51,722 - lava_callback - INFO - Starting async job submit for node 69c5b870abf66a8f4d9bcc69 2026-03-26 22:51:51,741 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:51:51,756 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b870abf66a8f4d9bcc69 HTTP/1.1" 200 None 2026-03-26 22:51:51,757 - lava_callback - DEBUG - Processing node 69c5b870abf66a8f4d9bcc69: baseline-arm64 2026-03-26 22:51:51,907 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 22:51:52,077 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 22:51:52,079 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c5b870abf66a8f4d9bcc69/log.txt.gz 2026-03-26 22:51:52,228 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 22:51:52,413 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 22:51:52,415 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c5b870abf66a8f4d9bcc69/lava-logs.yaml 2026-03-26 22:51:52,419 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 22:51:52,550 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 22:51:52,552 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c5b870abf66a8f4d9bcc69/lava_callback.json.gz 2026-03-26 22:51:52,570 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:51:52,585 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b870abf66a8f4d9bcc69 HTTP/1.1" 200 None 2026-03-26 22:51:52,587 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:51:52,600 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-03-26 22:51:52,602 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:51:52,626 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5b870abf66a8f4d9bcc69 HTTP/1.1" 200 None 2026-03-26 22:51:52,628 - lava_callback - INFO - Completed processing callback for node 69c5b870abf66a8f4d9bcc69 2026-03-26 22:51:56,299 - lava_callback - INFO - Received callback for node 69c5b881abf66a8f4d9bccb3 INFO: 172.21.0.1:56036 - "POST /node/69c5b881abf66a8f4d9bccb3 HTTP/1.0" 202 Accepted 2026-03-26 22:51:56,306 - lava_callback - INFO - Starting async job submit for node 69c5b881abf66a8f4d9bccb3 2026-03-26 22:51:56,308 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:51:56,322 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b881abf66a8f4d9bccb3 HTTP/1.1" 200 None 2026-03-26 22:51:56,323 - lava_callback - DEBUG - Processing node 69c5b881abf66a8f4d9bccb3: baseline-nfs-arm64 2026-03-26 22:51:56,325 - lava_callback - WARNING - Failed to upload log 2026-03-26 22:51:56,327 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 22:51:56,464 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 22:51:56,465 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c5b881abf66a8f4d9bccb3/lava-logs.yaml 2026-03-26 22:51:56,467 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.21.0.1:56050 - "GET / HTTP/1.0" 200 OK 2026-03-26 22:51:56,574 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 22:51:56,575 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c5b881abf66a8f4d9bccb3/lava_callback.json.gz 2026-03-26 22:51:56,577 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:51:56,591 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b881abf66a8f4d9bccb3 HTTP/1.1" 200 None 2026-03-26 22:51:56,592 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:51:56,606 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-03-26 22:51:56,607 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:51:56,633 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5b881abf66a8f4d9bccb3 HTTP/1.1" 200 None 2026-03-26 22:51:56,637 - lava_callback - INFO - Completed processing callback for node 69c5b881abf66a8f4d9bccb3 2026-03-26 22:52:02,992 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:52:03,011 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:41430 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53636 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53642 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53650 - "GET / HTTP/1.0" 200 OK 2026-03-26 22:52:21,572 - lava_callback - INFO - Received callback for node 69c315a16778148209fd6c52 INFO: 172.21.0.1:44210 - "POST /node/69c315a16778148209fd6c52 HTTP/1.0" 202 Accepted 2026-03-26 22:52:21,578 - lava_callback - INFO - Starting async job submit for node 69c315a16778148209fd6c52 2026-03-26 22:52:21,579 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:52:21,593 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c315a16778148209fd6c52 HTTP/1.1" 200 None 2026-03-26 22:52:21,594 - lava_callback - DEBUG - Processing node 69c315a16778148209fd6c52: baseline-arm-baylibre 2026-03-26 22:52:21,594 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c315a16778148209fd6c52 2026-03-26 22:52:21,595 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:52:21,608 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c315a16778148209fd6c52 HTTP/1.1" 200 None 2026-03-26 22:52:21,610 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:52:21,628 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01633c3d73288b0d052f0 HTTP/1.1" 200 None 2026-03-26 22:52:21,629 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:52:21,651 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c315a16778148209fd6c52 HTTP/1.1" 200 None 2026-03-26 22:52:21,652 - lava_callback - ERROR - Error processing callback for node 69c315a16778148209fd6c52: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:44216 - "GET / HTTP/1.0" 200 OK 2026-03-26 22:52:33,524 - lava_callback - INFO - Received callback for node 69c5b869abf66a8f4d9bcc55 INFO: 172.21.0.1:39678 - "POST /node/69c5b869abf66a8f4d9bcc55 HTTP/1.0" 202 Accepted 2026-03-26 22:52:33,533 - lava_callback - INFO - Starting async job submit for node 69c5b869abf66a8f4d9bcc55 2026-03-26 22:52:33,591 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:52:33,637 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b869abf66a8f4d9bcc55 HTTP/1.1" 200 None 2026-03-26 22:52:33,638 - lava_callback - DEBUG - Processing node 69c5b869abf66a8f4d9bcc55: baseline-arm64 2026-03-26 22:52:33,845 - lava_callback - WARNING - Failed to upload log 2026-03-26 22:52:33,921 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 22:52:34,098 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 22:52:34,101 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c5b869abf66a8f4d9bcc55/lava-logs.yaml 2026-03-26 22:52:34,104 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 22:52:34,111 - lava_callback - INFO - Received callback for node 69c5b866abf66a8f4d9bcc4c INFO: 172.21.0.1:39686 - "POST /node/69c5b866abf66a8f4d9bcc4c HTTP/1.0" 202 Accepted 2026-03-26 22:52:34,120 - lava_callback - INFO - Starting async job submit for node 69c5b866abf66a8f4d9bcc4c 2026-03-26 22:52:34,143 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:52:34,159 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b866abf66a8f4d9bcc4c HTTP/1.1" 200 None 2026-03-26 22:52:34,160 - lava_callback - DEBUG - Processing node 69c5b866abf66a8f4d9bcc4c: baseline-arm64 2026-03-26 22:52:34,257 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 22:52:34,300 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c5b869abf66a8f4d9bcc55/lava_callback.json.gz 2026-03-26 22:52:34,372 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:52:34,460 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b869abf66a8f4d9bcc55 HTTP/1.1" 200 None 2026-03-26 22:52:34,492 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:52:34,511 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 22:52:34,522 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-03-26 22:52:34,523 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:52:34,558 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5b869abf66a8f4d9bcc55 HTTP/1.1" 200 None 2026-03-26 22:52:34,627 - lava_callback - INFO - Completed processing callback for node 69c5b869abf66a8f4d9bcc55 2026-03-26 22:52:34,763 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 22:52:34,765 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c5b866abf66a8f4d9bcc4c/log.txt.gz 2026-03-26 22:52:34,959 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 22:52:35,149 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 22:52:35,151 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c5b866abf66a8f4d9bcc4c/lava-logs.yaml 2026-03-26 22:52:35,160 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 22:52:35,327 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 22:52:35,330 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c5b866abf66a8f4d9bcc4c/lava_callback.json.gz 2026-03-26 22:52:35,359 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:52:35,383 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b866abf66a8f4d9bcc4c HTTP/1.1" 200 None 2026-03-26 22:52:35,385 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:52:35,407 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-03-26 22:52:35,409 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:52:35,451 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5b866abf66a8f4d9bcc4c HTTP/1.1" 200 None 2026-03-26 22:52:35,453 - lava_callback - INFO - Completed processing callback for node 69c5b866abf66a8f4d9bcc4c INFO: 172.21.0.1:39692 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57714 - "GET / HTTP/1.0" 200 OK 2026-03-26 22:52:49,890 - lava_callback - INFO - Received callback for node 69c5b865abf66a8f4d9bcc49 INFO: 172.21.0.1:38484 - "POST /node/69c5b865abf66a8f4d9bcc49 HTTP/1.0" 202 Accepted 2026-03-26 22:52:49,897 - lava_callback - INFO - Starting async job submit for node 69c5b865abf66a8f4d9bcc49 2026-03-26 22:52:49,918 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:52:49,934 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b865abf66a8f4d9bcc49 HTTP/1.1" 200 None 2026-03-26 22:52:49,935 - lava_callback - DEBUG - Processing node 69c5b865abf66a8f4d9bcc49: baseline-arm64 2026-03-26 22:52:50,176 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 22:52:50,315 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 22:52:50,317 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c5b865abf66a8f4d9bcc49/log.txt.gz 2026-03-26 22:52:50,475 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 22:52:50,660 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 22:52:50,661 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c5b865abf66a8f4d9bcc49/lava-logs.yaml 2026-03-26 22:52:50,667 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 22:52:50,792 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 22:52:50,793 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c5b865abf66a8f4d9bcc49/lava_callback.json.gz 2026-03-26 22:52:50,815 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:52:50,829 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b865abf66a8f4d9bcc49 HTTP/1.1" 200 None 2026-03-26 22:52:50,831 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:52:50,844 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-03-26 22:52:50,845 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:52:51,014 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5b865abf66a8f4d9bcc49 HTTP/1.1" 200 None 2026-03-26 22:52:51,016 - lava_callback - INFO - Completed processing callback for node 69c5b865abf66a8f4d9bcc49 2026-03-26 22:52:51,730 - lava_callback - INFO - Received callback for node 69c5b871abf66a8f4d9bcc6c INFO: 172.21.0.1:38500 - "POST /node/69c5b871abf66a8f4d9bcc6c HTTP/1.0" 202 Accepted 2026-03-26 22:52:51,737 - lava_callback - INFO - Starting async job submit for node 69c5b871abf66a8f4d9bcc6c 2026-03-26 22:52:51,761 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:52:51,775 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b871abf66a8f4d9bcc6c HTTP/1.1" 200 None 2026-03-26 22:52:51,776 - lava_callback - DEBUG - Processing node 69c5b871abf66a8f4d9bcc6c: baseline-arm64 2026-03-26 22:52:52,002 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 22:52:52,135 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 22:52:52,137 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c5b871abf66a8f4d9bcc6c/log.txt.gz 2026-03-26 22:52:52,269 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 22:52:52,429 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 22:52:52,431 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c5b871abf66a8f4d9bcc6c/lava-logs.yaml 2026-03-26 22:52:52,437 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 22:52:52,571 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 22:52:52,573 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c5b871abf66a8f4d9bcc6c/lava_callback.json.gz 2026-03-26 22:52:52,595 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:52:52,609 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b871abf66a8f4d9bcc6c HTTP/1.1" 200 None 2026-03-26 22:52:52,610 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:52:52,625 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-03-26 22:52:52,626 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:52:52,652 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5b871abf66a8f4d9bcc6c HTTP/1.1" 200 None 2026-03-26 22:52:52,653 - lava_callback - INFO - Completed processing callback for node 69c5b871abf66a8f4d9bcc6c 2026-03-26 22:52:53,806 - lava_callback - INFO - Received callback for node 69c5b868abf66a8f4d9bcc52 INFO: 172.21.0.1:38512 - "POST /node/69c5b868abf66a8f4d9bcc52 HTTP/1.0" 202 Accepted 2026-03-26 22:52:53,813 - lava_callback - INFO - Starting async job submit for node 69c5b868abf66a8f4d9bcc52 2026-03-26 22:52:53,842 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:52:53,856 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b868abf66a8f4d9bcc52 HTTP/1.1" 200 None 2026-03-26 22:52:53,857 - lava_callback - DEBUG - Processing node 69c5b868abf66a8f4d9bcc52: baseline-arm64 2026-03-26 22:52:54,007 - lava_callback - WARNING - Failed to upload log 2026-03-26 22:52:54,078 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 22:52:54,242 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 22:52:54,244 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c5b868abf66a8f4d9bcc52/lava-logs.yaml 2026-03-26 22:52:54,247 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 22:52:54,365 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 22:52:54,367 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c5b868abf66a8f4d9bcc52/lava_callback.json.gz 2026-03-26 22:52:54,396 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:52:54,435 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b868abf66a8f4d9bcc52 HTTP/1.1" 200 None 2026-03-26 22:52:54,436 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:52:54,450 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-03-26 22:52:54,451 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:52:54,476 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5b868abf66a8f4d9bcc52 HTTP/1.1" 200 None 2026-03-26 22:52:54,537 - lava_callback - INFO - Completed processing callback for node 69c5b868abf66a8f4d9bcc52 INFO: 172.21.0.1:38516 - "GET / HTTP/1.0" 200 OK 2026-03-26 22:52:56,803 - lava_callback - INFO - Received callback for node 69c5b875abf66a8f4d9bcc76 INFO: 172.21.0.1:38518 - "POST /node/69c5b875abf66a8f4d9bcc76 HTTP/1.0" 202 Accepted 2026-03-26 22:52:56,810 - lava_callback - INFO - Starting async job submit for node 69c5b875abf66a8f4d9bcc76 2026-03-26 22:52:56,831 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:52:56,850 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b875abf66a8f4d9bcc76 HTTP/1.1" 200 None 2026-03-26 22:52:56,850 - lava_callback - DEBUG - Processing node 69c5b875abf66a8f4d9bcc76: baseline-arm64 2026-03-26 22:52:57,113 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 22:52:57,222 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 22:52:57,224 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c5b875abf66a8f4d9bcc76/log.txt.gz 2026-03-26 22:52:57,380 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 22:52:57,465 - lava_callback - INFO - Received callback for node 69c5b86babf66a8f4d9bcc5b INFO: 172.21.0.1:38530 - "POST /node/69c5b86babf66a8f4d9bcc5b HTTP/1.0" 202 Accepted 2026-03-26 22:52:57,471 - lava_callback - INFO - Starting async job submit for node 69c5b86babf66a8f4d9bcc5b 2026-03-26 22:52:57,494 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:52:57,510 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b86babf66a8f4d9bcc5b HTTP/1.1" 200 None 2026-03-26 22:52:57,511 - lava_callback - DEBUG - Processing node 69c5b86babf66a8f4d9bcc5b: baseline-arm64 2026-03-26 22:52:57,549 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 22:52:57,596 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c5b875abf66a8f4d9bcc76/lava-logs.yaml 2026-03-26 22:52:57,690 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 22:52:57,788 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 22:52:57,879 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 22:52:57,881 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c5b875abf66a8f4d9bcc76/lava_callback.json.gz 2026-03-26 22:52:57,904 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:52:57,918 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b875abf66a8f4d9bcc76 HTTP/1.1" 200 None 2026-03-26 22:52:57,920 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:52:57,921 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 22:52:57,922 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c5b86babf66a8f4d9bcc5b/log.txt.gz 2026-03-26 22:52:57,963 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-03-26 22:52:57,981 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:52:58,068 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5b875abf66a8f4d9bcc76 HTTP/1.1" 200 None 2026-03-26 22:52:58,072 - lava_callback - INFO - Completed processing callback for node 69c5b875abf66a8f4d9bcc76 2026-03-26 22:52:58,144 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 22:52:58,404 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 22:52:58,406 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c5b86babf66a8f4d9bcc5b/lava-logs.yaml 2026-03-26 22:52:58,413 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 22:52:58,557 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 22:52:58,558 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c5b86babf66a8f4d9bcc5b/lava_callback.json.gz 2026-03-26 22:52:58,586 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:52:58,601 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b86babf66a8f4d9bcc5b HTTP/1.1" 200 None 2026-03-26 22:52:58,603 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:52:58,616 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-03-26 22:52:58,618 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:52:58,641 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5b86babf66a8f4d9bcc5b HTTP/1.1" 200 None 2026-03-26 22:52:58,646 - lava_callback - INFO - Completed processing callback for node 69c5b86babf66a8f4d9bcc5b 2026-03-26 22:52:59,661 - lava_callback - INFO - Received callback for node 69c5b885abf66a8f4d9bccc0 INFO: 172.21.0.1:38536 - "POST /node/69c5b885abf66a8f4d9bccc0 HTTP/1.0" 202 Accepted 2026-03-26 22:52:59,668 - lava_callback - INFO - Starting async job submit for node 69c5b885abf66a8f4d9bccc0 2026-03-26 22:52:59,690 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:52:59,704 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b885abf66a8f4d9bccc0 HTTP/1.1" 200 None 2026-03-26 22:52:59,705 - lava_callback - DEBUG - Processing node 69c5b885abf66a8f4d9bccc0: baseline-nfs-arm64 2026-03-26 22:53:00,117 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 22:53:01,229 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 22:53:01,231 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c5b885abf66a8f4d9bccc0/log.txt.gz 2026-03-26 22:53:01,441 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 22:53:01,506 - lava_callback - INFO - Received callback for node 69c315c86778148209fd6c60 INFO: 172.21.0.1:39900 - "POST /node/69c315c86778148209fd6c60 HTTP/1.0" 202 Accepted 2026-03-26 22:53:01,512 - lava_callback - INFO - Starting async job submit for node 69c315c86778148209fd6c60 2026-03-26 22:53:01,512 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:53:01,528 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c315c86778148209fd6c60 HTTP/1.1" 200 None 2026-03-26 22:53:01,529 - lava_callback - DEBUG - Processing node 69c315c86778148209fd6c60: baseline-arm-baylibre 2026-03-26 22:53:01,529 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c315c86778148209fd6c60 2026-03-26 22:53:01,530 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:53:01,547 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c315c86778148209fd6c60 HTTP/1.1" 200 None 2026-03-26 22:53:01,548 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:53:01,562 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01633c3d73288b0d052f0 HTTP/1.1" 200 None 2026-03-26 22:53:01,564 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:53:01,590 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c315c86778148209fd6c60 HTTP/1.1" 200 None 2026-03-26 22:53:01,595 - lava_callback - ERROR - Error processing callback for node 69c315c86778148209fd6c60: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-26 22:53:01,650 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 22:53:01,653 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c5b885abf66a8f4d9bccc0/lava-logs.yaml 2026-03-26 22:53:01,665 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 22:53:01,837 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 22:53:01,839 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c5b885abf66a8f4d9bccc0/lava_callback.json.gz 2026-03-26 22:53:01,862 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:53:01,884 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b885abf66a8f4d9bccc0 HTTP/1.1" 200 None 2026-03-26 22:53:01,885 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:53:01,899 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-03-26 22:53:01,900 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:53:01,924 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5b885abf66a8f4d9bccc0 HTTP/1.1" 200 None 2026-03-26 22:53:01,929 - lava_callback - INFO - Completed processing callback for node 69c5b885abf66a8f4d9bccc0 2026-03-26 22:53:03,013 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:53:03,029 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 2026-03-26 22:53:05,322 - lava_callback - INFO - Received callback for node 69c5b874abf66a8f4d9bcc73 INFO: 172.21.0.1:39910 - "POST /node/69c5b874abf66a8f4d9bcc73 HTTP/1.0" 202 Accepted 2026-03-26 22:53:05,329 - lava_callback - INFO - Starting async job submit for node 69c5b874abf66a8f4d9bcc73 2026-03-26 22:53:05,350 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:53:05,364 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b874abf66a8f4d9bcc73 HTTP/1.1" 200 None 2026-03-26 22:53:05,365 - lava_callback - DEBUG - Processing node 69c5b874abf66a8f4d9bcc73: baseline-arm64 2026-03-26 22:53:05,632 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 22:53:05,769 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 22:53:05,771 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c5b874abf66a8f4d9bcc73/log.txt.gz 2026-03-26 22:53:05,931 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 22:53:06,123 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 22:53:06,125 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c5b874abf66a8f4d9bcc73/lava-logs.yaml 2026-03-26 22:53:06,131 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 22:53:06,278 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 22:53:06,279 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c5b874abf66a8f4d9bcc73/lava_callback.json.gz 2026-03-26 22:53:06,301 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:53:06,316 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b874abf66a8f4d9bcc73 HTTP/1.1" 200 None 2026-03-26 22:53:06,317 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:53:06,337 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-03-26 22:53:06,339 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:53:06,367 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5b874abf66a8f4d9bcc73 HTTP/1.1" 200 None 2026-03-26 22:53:06,373 - lava_callback - INFO - Completed processing callback for node 69c5b874abf66a8f4d9bcc73 INFO: 172.21.0.1:39918 - "GET / HTTP/1.0" 200 OK 2026-03-26 22:53:11,504 - lava_callback - INFO - Received callback for node 69c5b86dabf66a8f4d9bcc5e INFO: 172.21.0.1:40886 - "POST /node/69c5b86dabf66a8f4d9bcc5e HTTP/1.0" 202 Accepted 2026-03-26 22:53:11,511 - lava_callback - INFO - Starting async job submit for node 69c5b86dabf66a8f4d9bcc5e 2026-03-26 22:53:11,533 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:53:11,548 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b86dabf66a8f4d9bcc5e HTTP/1.1" 200 None 2026-03-26 22:53:11,548 - lava_callback - DEBUG - Processing node 69c5b86dabf66a8f4d9bcc5e: baseline-arm64 INFO: 172.21.0.1:40896 - "GET / HTTP/1.0" 200 OK 2026-03-26 22:53:11,925 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 22:53:12,035 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 22:53:12,037 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c5b86dabf66a8f4d9bcc5e/log.txt.gz 2026-03-26 22:53:12,245 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 22:53:12,497 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 22:53:12,499 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c5b86dabf66a8f4d9bcc5e/lava-logs.yaml 2026-03-26 22:53:12,507 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 22:53:12,657 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 22:53:12,658 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c5b86dabf66a8f4d9bcc5e/lava_callback.json.gz 2026-03-26 22:53:12,681 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:53:12,696 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b86dabf66a8f4d9bcc5e HTTP/1.1" 200 None 2026-03-26 22:53:12,697 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:53:12,711 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-03-26 22:53:12,712 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:53:12,747 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5b86dabf66a8f4d9bcc5e HTTP/1.1" 200 None 2026-03-26 22:53:12,748 - lava_callback - INFO - Completed processing callback for node 69c5b86dabf66a8f4d9bcc5e 2026-03-26 22:53:14,131 - lava_callback - INFO - Received callback for node 69c5b867abf66a8f4d9bcc4f INFO: 172.21.0.1:40902 - "POST /node/69c5b867abf66a8f4d9bcc4f HTTP/1.0" 202 Accepted 2026-03-26 22:53:14,137 - lava_callback - INFO - Starting async job submit for node 69c5b867abf66a8f4d9bcc4f 2026-03-26 22:53:14,159 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:53:14,174 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b867abf66a8f4d9bcc4f HTTP/1.1" 200 None 2026-03-26 22:53:14,175 - lava_callback - DEBUG - Processing node 69c5b867abf66a8f4d9bcc4f: baseline-arm64 2026-03-26 22:53:14,417 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 22:53:14,536 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 22:53:14,537 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c5b867abf66a8f4d9bcc4f/log.txt.gz 2026-03-26 22:53:14,681 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 22:53:14,849 - lava_callback - INFO - Received callback for node 69c5b86eabf66a8f4d9bcc61 INFO: 172.21.0.1:40904 - "POST /node/69c5b86eabf66a8f4d9bcc61 HTTP/1.0" 202 Accepted 2026-03-26 22:53:14,857 - lava_callback - INFO - Starting async job submit for node 69c5b86eabf66a8f4d9bcc61 2026-03-26 22:53:14,888 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 22:53:14,889 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:53:14,890 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c5b867abf66a8f4d9bcc4f/lava-logs.yaml 2026-03-26 22:53:14,896 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 22:53:14,904 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b86eabf66a8f4d9bcc61 HTTP/1.1" 200 None 2026-03-26 22:53:14,905 - lava_callback - DEBUG - Processing node 69c5b86eabf66a8f4d9bcc61: baseline-arm64 2026-03-26 22:53:15,086 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 22:53:15,139 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c5b867abf66a8f4d9bcc4f/lava_callback.json.gz INFO: 172.21.0.1:40914 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-26 22:53:15,255 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:53:15,342 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b867abf66a8f4d9bcc4f HTTP/1.1" 200 None 2026-03-26 22:53:15,375 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:53:15,409 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 22:53:15,420 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-03-26 22:53:15,421 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:53:15,452 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5b867abf66a8f4d9bcc4f HTTP/1.1" 200 None 2026-03-26 22:53:15,507 - lava_callback - INFO - Completed processing callback for node 69c5b867abf66a8f4d9bcc4f 2026-03-26 22:53:15,631 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 22:53:15,634 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c5b86eabf66a8f4d9bcc61/log.txt.gz 2026-03-26 22:53:15,887 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 22:53:15,985 - lava_callback - INFO - Received callback for node 69c5b8aaabf66a8f4d9bcd3b INFO: 172.21.0.1:40930 - "POST /node/69c5b8aaabf66a8f4d9bcd3b HTTP/1.0" 202 Accepted 2026-03-26 22:53:15,991 - lava_callback - INFO - Starting async job submit for node 69c5b8aaabf66a8f4d9bcd3b 2026-03-26 22:53:16,014 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:53:16,029 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b8aaabf66a8f4d9bcd3b HTTP/1.1" 200 None 2026-03-26 22:53:16,031 - lava_callback - DEBUG - Processing node 69c5b8aaabf66a8f4d9bcd3b: baseline-arm64-pengutronix 2026-03-26 22:53:16,086 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 22:53:16,134 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c5b86eabf66a8f4d9bcc61/lava-logs.yaml 2026-03-26 22:53:16,229 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 22:53:16,234 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 22:53:16,383 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 22:53:16,385 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c5b86eabf66a8f4d9bcc61/lava_callback.json.gz 2026-03-26 22:53:16,416 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 22:53:16,418 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:53:16,419 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69c5b8aaabf66a8f4d9bcd3b/log.txt.gz 2026-03-26 22:53:16,499 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b86eabf66a8f4d9bcc61 HTTP/1.1" 200 None 2026-03-26 22:53:16,531 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.21.0.1:40942 - "GET / HTTP/1.0" 200 OK 2026-03-26 22:53:16,534 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 22:53:16,549 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-03-26 22:53:16,551 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:53:16,593 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5b86eabf66a8f4d9bcc61 HTTP/1.1" 200 None 2026-03-26 22:53:16,613 - lava_callback - INFO - Completed processing callback for node 69c5b86eabf66a8f4d9bcc61 2026-03-26 22:53:16,781 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 22:53:16,783 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69c5b8aaabf66a8f4d9bcd3b/lava-logs.yaml 2026-03-26 22:53:16,789 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 22:53:16,937 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 22:53:16,939 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69c5b8aaabf66a8f4d9bcd3b/lava_callback.json.gz 2026-03-26 22:53:16,970 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:53:16,991 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b8aaabf66a8f4d9bcd3b HTTP/1.1" 200 None 2026-03-26 22:53:16,993 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:53:17,008 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-03-26 22:53:17,009 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:53:17,035 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5b8aaabf66a8f4d9bcd3b HTTP/1.1" 200 None 2026-03-26 22:53:17,036 - lava_callback - INFO - Completed processing callback for node 69c5b8aaabf66a8f4d9bcd3b 2026-03-26 22:53:23,280 - lava_callback - INFO - Received callback for node 69c5b880abf66a8f4d9bccb0 INFO: 172.21.0.1:51900 - "POST /node/69c5b880abf66a8f4d9bccb0 HTTP/1.0" 202 Accepted 2026-03-26 22:53:23,287 - lava_callback - INFO - Starting async job submit for node 69c5b880abf66a8f4d9bccb0 2026-03-26 22:53:23,309 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:53:23,324 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b880abf66a8f4d9bccb0 HTTP/1.1" 200 None 2026-03-26 22:53:23,325 - lava_callback - DEBUG - Processing node 69c5b880abf66a8f4d9bccb0: baseline-nfs-arm64 2026-03-26 22:53:23,908 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 22:53:24,070 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 22:53:24,072 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c5b880abf66a8f4d9bccb0/log.txt.gz 2026-03-26 22:53:24,407 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 22:53:24,624 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 22:53:24,626 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c5b880abf66a8f4d9bccb0/lava-logs.yaml 2026-03-26 22:53:24,638 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 22:53:24,799 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 22:53:24,801 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c5b880abf66a8f4d9bccb0/lava_callback.json.gz 2026-03-26 22:53:24,825 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:53:24,841 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b880abf66a8f4d9bccb0 HTTP/1.1" 200 None 2026-03-26 22:53:24,843 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:53:24,861 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-03-26 22:53:24,863 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:53:24,891 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5b880abf66a8f4d9bccb0 HTTP/1.1" 200 None 2026-03-26 22:53:24,896 - lava_callback - INFO - Completed processing callback for node 69c5b880abf66a8f4d9bccb0 INFO: 172.21.0.1:51912 - "GET / HTTP/1.0" 200 OK 2026-03-26 22:53:27,942 - lava_callback - INFO - Received callback for node 69c5b87fabf66a8f4d9bccad INFO: 172.21.0.1:51918 - "POST /node/69c5b87fabf66a8f4d9bccad HTTP/1.0" 202 Accepted 2026-03-26 22:53:27,951 - lava_callback - INFO - Starting async job submit for node 69c5b87fabf66a8f4d9bccad 2026-03-26 22:53:27,982 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:53:27,996 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b87fabf66a8f4d9bccad HTTP/1.1" 200 None 2026-03-26 22:53:27,997 - lava_callback - DEBUG - Processing node 69c5b87fabf66a8f4d9bccad: baseline-nfs-arm64 2026-03-26 22:53:28,274 - lava_callback - INFO - Received callback for node 69c5b87cabf66a8f4d9bcca7 INFO: 172.21.0.1:51920 - "POST /node/69c5b87cabf66a8f4d9bcca7 HTTP/1.0" 202 Accepted 2026-03-26 22:53:28,306 - lava_callback - INFO - Starting async job submit for node 69c5b87cabf66a8f4d9bcca7 2026-03-26 22:53:28,370 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:53:28,442 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b87cabf66a8f4d9bcca7 HTTP/1.1" 200 None 2026-03-26 22:53:28,453 - lava_callback - DEBUG - Processing node 69c5b87cabf66a8f4d9bcca7: baseline-nfs-arm64 2026-03-26 22:53:28,826 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 22:53:28,954 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 22:53:29,083 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 22:53:29,085 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c5b87fabf66a8f4d9bccad/log.txt.gz 2026-03-26 22:53:29,097 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 22:53:29,161 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c5b87cabf66a8f4d9bcca7/log.txt.gz 2026-03-26 22:53:29,586 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 22:53:29,629 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 22:53:29,966 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 22:53:29,969 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c5b87cabf66a8f4d9bcca7/lava-logs.yaml 2026-03-26 22:53:29,977 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 22:53:30,046 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 22:53:30,048 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c5b87fabf66a8f4d9bccad/lava-logs.yaml 2026-03-26 22:53:30,063 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 22:53:30,171 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 22:53:30,172 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c5b87cabf66a8f4d9bcca7/lava_callback.json.gz 2026-03-26 22:53:30,196 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:53:30,211 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b87cabf66a8f4d9bcca7 HTTP/1.1" 200 None 2026-03-26 22:53:30,213 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:53:30,227 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 22:53:30,229 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-03-26 22:53:30,231 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:53:30,231 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c5b87fabf66a8f4d9bccad/lava_callback.json.gz 2026-03-26 22:53:30,263 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:53:30,278 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b87fabf66a8f4d9bccad HTTP/1.1" 200 None 2026-03-26 22:53:30,280 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:53:30,297 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5b87cabf66a8f4d9bcca7 HTTP/1.1" 200 None 2026-03-26 22:53:30,300 - lava_callback - INFO - Completed processing callback for node 69c5b87cabf66a8f4d9bcca7 2026-03-26 22:53:30,337 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-03-26 22:53:30,340 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:53:30,379 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5b87fabf66a8f4d9bccad HTTP/1.1" 200 None 2026-03-26 22:53:30,381 - lava_callback - INFO - Completed processing callback for node 69c5b87fabf66a8f4d9bccad 2026-03-26 22:53:33,032 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:53:33,050 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 2026-03-26 22:53:34,293 - lava_callback - INFO - Received callback for node 69c5b882abf66a8f4d9bccb6 INFO: 172.21.0.1:52456 - "POST /node/69c5b882abf66a8f4d9bccb6 HTTP/1.0" 202 Accepted 2026-03-26 22:53:34,304 - lava_callback - INFO - Starting async job submit for node 69c5b882abf66a8f4d9bccb6 2026-03-26 22:53:34,338 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:53:34,354 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b882abf66a8f4d9bccb6 HTTP/1.1" 200 None 2026-03-26 22:53:34,355 - lava_callback - DEBUG - Processing node 69c5b882abf66a8f4d9bccb6: baseline-nfs-arm64 2026-03-26 22:53:34,769 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 22:53:34,887 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 22:53:34,889 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c5b882abf66a8f4d9bccb6/log.txt.gz 2026-03-26 22:53:35,115 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 22:53:35,327 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 22:53:35,329 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c5b882abf66a8f4d9bccb6/lava-logs.yaml 2026-03-26 22:53:35,339 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 22:53:35,485 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 22:53:35,487 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c5b882abf66a8f4d9bccb6/lava_callback.json.gz 2026-03-26 22:53:35,512 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:53:35,535 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b882abf66a8f4d9bccb6 HTTP/1.1" 200 None 2026-03-26 22:53:35,537 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:53:35,551 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-03-26 22:53:35,553 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:53:35,592 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5b882abf66a8f4d9bccb6 HTTP/1.1" 200 None 2026-03-26 22:53:35,600 - lava_callback - INFO - Completed processing callback for node 69c5b882abf66a8f4d9bccb6 INFO: 172.21.0.1:52466 - "GET / HTTP/1.0" 200 OK 2026-03-26 22:53:37,735 - lava_callback - INFO - Received callback for node 69c5b8aeabf66a8f4d9bcd44 INFO: 172.21.0.1:52472 - "POST /node/69c5b8aeabf66a8f4d9bcd44 HTTP/1.0" 202 Accepted 2026-03-26 22:53:37,742 - lava_callback - INFO - Starting async job submit for node 69c5b8aeabf66a8f4d9bcd44 2026-03-26 22:53:37,763 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:53:37,777 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b8aeabf66a8f4d9bcd44 HTTP/1.1" 200 None 2026-03-26 22:53:37,778 - lava_callback - DEBUG - Processing node 69c5b8aeabf66a8f4d9bcd44: baseline-arm64-foundriesio 2026-03-26 22:53:37,998 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 22:53:38,128 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 22:53:38,129 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69c5b8aeabf66a8f4d9bcd44/log.txt.gz 2026-03-26 22:53:38,216 - lava_callback - INFO - Received callback for node 69c5b886abf66a8f4d9bccc3 INFO: 172.21.0.1:52486 - "POST /node/69c5b886abf66a8f4d9bccc3 HTTP/1.0" 202 Accepted 2026-03-26 22:53:38,238 - lava_callback - INFO - Starting async job submit for node 69c5b886abf66a8f4d9bccc3 2026-03-26 22:53:38,289 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:53:38,289 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 22:53:38,313 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b886abf66a8f4d9bccc3 HTTP/1.1" 200 None 2026-03-26 22:53:38,314 - lava_callback - DEBUG - Processing node 69c5b886abf66a8f4d9bccc3: baseline-nfs-arm64 2026-03-26 22:53:38,467 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 22:53:38,515 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69c5b8aeabf66a8f4d9bcd44/lava-logs.yaml 2026-03-26 22:53:38,592 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 22:53:38,663 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 22:53:38,783 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 22:53:38,786 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69c5b8aeabf66a8f4d9bcd44/lava_callback.json.gz 2026-03-26 22:53:38,814 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 22:53:38,834 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:53:38,834 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c5b886abf66a8f4d9bccc3/log.txt.gz 2026-03-26 22:53:38,898 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b8aeabf66a8f4d9bcd44 HTTP/1.1" 200 None 2026-03-26 22:53:38,927 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:53:38,983 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-03-26 22:53:39,017 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:53:39,047 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 22:53:39,083 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5b8aeabf66a8f4d9bcd44 HTTP/1.1" 200 None 2026-03-26 22:53:39,140 - lava_callback - INFO - Completed processing callback for node 69c5b8aeabf66a8f4d9bcd44 2026-03-26 22:53:39,319 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 22:53:39,322 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c5b886abf66a8f4d9bccc3/lava-logs.yaml 2026-03-26 22:53:39,332 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 22:53:39,401 - lava_callback - INFO - Received callback for node 69c5b8acabf66a8f4d9bcd3e INFO: 172.21.0.1:52500 - "POST /node/69c5b8acabf66a8f4d9bcd3e HTTP/1.0" 202 Accepted 2026-03-26 22:53:39,411 - lava_callback - INFO - Starting async job submit for node 69c5b8acabf66a8f4d9bcd3e 2026-03-26 22:53:39,434 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:53:39,450 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b8acabf66a8f4d9bcd3e HTTP/1.1" 200 None 2026-03-26 22:53:39,451 - lava_callback - DEBUG - Processing node 69c5b8acabf66a8f4d9bcd3e: baseline-arm64-pengutronix 2026-03-26 22:53:39,471 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 22:53:39,513 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c5b886abf66a8f4d9bccc3/lava_callback.json.gz 2026-03-26 22:53:39,564 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:53:39,656 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b886abf66a8f4d9bccc3 HTTP/1.1" 200 None 2026-03-26 22:53:39,688 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:53:39,704 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 22:53:39,715 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-03-26 22:53:39,717 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:53:39,759 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5b886abf66a8f4d9bccc3 HTTP/1.1" 200 None 2026-03-26 22:53:39,784 - lava_callback - INFO - Completed processing callback for node 69c5b886abf66a8f4d9bccc3 2026-03-26 22:53:39,897 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 22:53:39,899 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69c5b8acabf66a8f4d9bcd3e/log.txt.gz 2026-03-26 22:53:40,077 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 22:53:40,266 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 22:53:40,268 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69c5b8acabf66a8f4d9bcd3e/lava-logs.yaml 2026-03-26 22:53:40,275 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 22:53:40,421 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 22:53:40,422 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69c5b8acabf66a8f4d9bcd3e/lava_callback.json.gz 2026-03-26 22:53:40,443 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:53:40,456 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b8acabf66a8f4d9bcd3e HTTP/1.1" 200 None 2026-03-26 22:53:40,458 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:53:40,471 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-03-26 22:53:40,472 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:53:40,496 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5b8acabf66a8f4d9bcd3e HTTP/1.1" 200 None 2026-03-26 22:53:40,501 - lava_callback - INFO - Completed processing callback for node 69c5b8acabf66a8f4d9bcd3e 2026-03-26 22:53:41,582 - lava_callback - INFO - Received callback for node 69c315ef6778148209fd6c69 INFO: 172.21.0.1:60778 - "POST /node/69c315ef6778148209fd6c69 HTTP/1.0" 202 Accepted 2026-03-26 22:53:41,591 - lava_callback - INFO - Starting async job submit for node 69c315ef6778148209fd6c69 2026-03-26 22:53:41,592 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:53:41,609 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c315ef6778148209fd6c69 HTTP/1.1" 200 None 2026-03-26 22:53:41,610 - lava_callback - DEBUG - Processing node 69c315ef6778148209fd6c69: baseline-arm-baylibre 2026-03-26 22:53:41,610 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c315ef6778148209fd6c69 2026-03-26 22:53:41,611 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:53:41,625 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c315ef6778148209fd6c69 HTTP/1.1" 200 None 2026-03-26 22:53:41,626 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:53:41,640 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01633c3d73288b0d052f0 HTTP/1.1" 200 None 2026-03-26 22:53:41,642 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:53:41,668 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c315ef6778148209fd6c69 HTTP/1.1" 200 None 2026-03-26 22:53:41,669 - lava_callback - ERROR - Error processing callback for node 69c315ef6778148209fd6c69: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:60794 - "GET / HTTP/1.0" 200 OK 2026-03-26 22:53:48,697 - lava_callback - INFO - Received callback for node 69c5b8adabf66a8f4d9bcd41 INFO: 172.21.0.1:60808 - "POST /node/69c5b8adabf66a8f4d9bcd41 HTTP/1.0" 202 Accepted 2026-03-26 22:53:48,703 - lava_callback - INFO - Starting async job submit for node 69c5b8adabf66a8f4d9bcd41 2026-03-26 22:53:48,723 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:53:48,747 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b8adabf66a8f4d9bcd41 HTTP/1.1" 200 None 2026-03-26 22:53:48,748 - lava_callback - DEBUG - Processing node 69c5b8adabf66a8f4d9bcd41: baseline-arm64-pengutronix 2026-03-26 22:53:48,948 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 22:53:49,067 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 22:53:49,069 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69c5b8adabf66a8f4d9bcd41/log.txt.gz 2026-03-26 22:53:49,189 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 22:53:49,440 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 22:53:49,442 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69c5b8adabf66a8f4d9bcd41/lava-logs.yaml 2026-03-26 22:53:49,447 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 22:53:49,583 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 22:53:49,585 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69c5b8adabf66a8f4d9bcd41/lava_callback.json.gz 2026-03-26 22:53:49,605 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:53:49,623 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b8adabf66a8f4d9bcd41 HTTP/1.1" 200 None 2026-03-26 22:53:49,625 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:53:49,638 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-03-26 22:53:49,640 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:53:49,664 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5b8adabf66a8f4d9bcd41 HTTP/1.1" 200 None 2026-03-26 22:53:49,669 - lava_callback - INFO - Completed processing callback for node 69c5b8adabf66a8f4d9bcd41 2026-03-26 22:53:50,870 - lava_callback - INFO - Received callback for node 69c5b879abf66a8f4d9bcc7f INFO: 172.21.0.1:37064 - "POST /node/69c5b879abf66a8f4d9bcc7f HTTP/1.0" 202 Accepted 2026-03-26 22:53:50,877 - lava_callback - INFO - Starting async job submit for node 69c5b879abf66a8f4d9bcc7f 2026-03-26 22:53:50,899 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:53:50,913 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b879abf66a8f4d9bcc7f HTTP/1.1" 200 None 2026-03-26 22:53:50,914 - lava_callback - DEBUG - Processing node 69c5b879abf66a8f4d9bcc7f: baseline-nfs-arm64 2026-03-26 22:53:51,225 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 22:53:51,354 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 22:53:51,356 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c5b879abf66a8f4d9bcc7f/log.txt.gz 2026-03-26 22:53:51,589 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 22:53:51,764 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 22:53:51,766 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c5b879abf66a8f4d9bcc7f/lava-logs.yaml 2026-03-26 22:53:51,773 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 22:53:51,894 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 22:53:51,895 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c5b879abf66a8f4d9bcc7f/lava_callback.json.gz 2026-03-26 22:53:51,918 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:53:51,933 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b879abf66a8f4d9bcc7f HTTP/1.1" 200 None 2026-03-26 22:53:51,935 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:53:51,948 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-03-26 22:53:51,950 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:53:51,974 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5b879abf66a8f4d9bcc7f HTTP/1.1" 200 None 2026-03-26 22:53:51,979 - lava_callback - INFO - Completed processing callback for node 69c5b879abf66a8f4d9bcc7f INFO: 172.21.0.1:37068 - "GET / HTTP/1.0" 200 OK 2026-03-26 22:54:02,952 - lava_callback - INFO - Received callback for node 69c5b86fabf66a8f4d9bcc64 INFO: 172.21.0.1:40372 - "POST /node/69c5b86fabf66a8f4d9bcc64 HTTP/1.0" 202 Accepted 2026-03-26 22:54:02,959 - lava_callback - INFO - Starting async job submit for node 69c5b86fabf66a8f4d9bcc64 2026-03-26 22:54:02,981 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:54:02,995 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b86fabf66a8f4d9bcc64 HTTP/1.1" 200 None 2026-03-26 22:54:02,995 - lava_callback - DEBUG - Processing node 69c5b86fabf66a8f4d9bcc64: baseline-arm64 2026-03-26 22:54:03,073 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:54:03,181 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 2026-03-26 22:54:03,494 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 22:54:03,793 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 22:54:03,795 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c5b86fabf66a8f4d9bcc64/log.txt.gz 2026-03-26 22:54:04,102 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 22:54:04,282 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 22:54:04,284 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c5b86fabf66a8f4d9bcc64/lava-logs.yaml 2026-03-26 22:54:04,295 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 22:54:04,437 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 22:54:04,439 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c5b86fabf66a8f4d9bcc64/lava_callback.json.gz 2026-03-26 22:54:04,482 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:54:04,497 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b86fabf66a8f4d9bcc64 HTTP/1.1" 200 None 2026-03-26 22:54:04,499 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:54:04,523 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-03-26 22:54:04,525 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:54:04,550 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5b86fabf66a8f4d9bcc64 HTTP/1.1" 200 None 2026-03-26 22:54:04,602 - lava_callback - INFO - Completed processing callback for node 69c5b86fabf66a8f4d9bcc64 INFO: 172.21.0.1:40388 - "GET / HTTP/1.0" 200 OK 2026-03-26 22:54:09,630 - lava_callback - INFO - Received callback for node 69c5b8a1abf66a8f4d9bcd21 INFO: 172.21.0.1:40390 - "POST /node/69c5b8a1abf66a8f4d9bcd21 HTTP/1.0" 202 Accepted 2026-03-26 22:54:09,636 - lava_callback - INFO - Starting async job submit for node 69c5b8a1abf66a8f4d9bcd21 2026-03-26 22:54:09,658 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:54:09,672 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b8a1abf66a8f4d9bcd21 HTTP/1.1" 200 None 2026-03-26 22:54:09,673 - lava_callback - DEBUG - Processing node 69c5b8a1abf66a8f4d9bcd21: baseline-arm64-broonie 2026-03-26 22:54:09,951 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 22:54:10,069 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 22:54:10,070 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c5b8a1abf66a8f4d9bcd21/log.txt.gz 2026-03-26 22:54:10,218 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 22:54:10,406 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 22:54:10,408 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c5b8a1abf66a8f4d9bcd21/lava-logs.yaml 2026-03-26 22:54:10,415 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 22:54:10,534 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 22:54:10,536 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c5b8a1abf66a8f4d9bcd21/lava_callback.json.gz 2026-03-26 22:54:10,559 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:54:10,588 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b8a1abf66a8f4d9bcd21 HTTP/1.1" 200 None 2026-03-26 22:54:10,589 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:54:10,603 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-03-26 22:54:10,605 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:54:10,630 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5b8a1abf66a8f4d9bcd21 HTTP/1.1" 200 None 2026-03-26 22:54:10,635 - lava_callback - INFO - Completed processing callback for node 69c5b8a1abf66a8f4d9bcd21 2026-03-26 22:54:10,755 - lava_callback - INFO - Received callback for node 69c5b8a6abf66a8f4d9bcd32 INFO: 172.21.0.1:47184 - "POST /node/69c5b8a6abf66a8f4d9bcd32 HTTP/1.0" 202 Accepted 2026-03-26 22:54:10,765 - lava_callback - INFO - Starting async job submit for node 69c5b8a6abf66a8f4d9bcd32 2026-03-26 22:54:10,801 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:54:10,830 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b8a6abf66a8f4d9bcd32 HTTP/1.1" 200 None 2026-03-26 22:54:10,832 - lava_callback - DEBUG - Processing node 69c5b8a6abf66a8f4d9bcd32: baseline-arm64-clabbe 2026-03-26 22:54:11,093 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 22:54:11,204 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 22:54:11,207 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-clabbe-69c5b8a6abf66a8f4d9bcd32/log.txt.gz 2026-03-26 22:54:11,348 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 22:54:11,523 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 22:54:11,525 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-clabbe-69c5b8a6abf66a8f4d9bcd32/lava-logs.yaml 2026-03-26 22:54:11,531 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 22:54:11,681 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 22:54:11,683 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-clabbe-69c5b8a6abf66a8f4d9bcd32/lava_callback.json.gz 2026-03-26 22:54:11,705 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:54:11,719 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b8a6abf66a8f4d9bcd32 HTTP/1.1" 200 None 2026-03-26 22:54:11,720 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:54:11,733 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-03-26 22:54:11,735 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.21.0.1:47188 - "GET / HTTP/1.0" 200 OK 2026-03-26 22:54:11,760 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5b8a6abf66a8f4d9bcd32 HTTP/1.1" 200 None 2026-03-26 22:54:11,767 - lava_callback - INFO - Completed processing callback for node 69c5b8a6abf66a8f4d9bcd32 INFO: 172.21.0.1:47204 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47218 - "GET / HTTP/1.0" 200 OK 2026-03-26 22:54:19,504 - lava_callback - INFO - Received callback for node 69c5b898abf66a8f4d9bccf2 INFO: 172.21.0.1:47232 - "POST /node/69c5b898abf66a8f4d9bccf2 HTTP/1.0" 202 Accepted 2026-03-26 22:54:19,510 - lava_callback - INFO - Starting async job submit for node 69c5b898abf66a8f4d9bccf2 2026-03-26 22:54:19,532 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:54:19,555 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b898abf66a8f4d9bccf2 HTTP/1.1" 200 None 2026-03-26 22:54:19,556 - lava_callback - DEBUG - Processing node 69c5b898abf66a8f4d9bccf2: baseline-arm64-broonie 2026-03-26 22:54:19,835 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 22:54:19,969 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 22:54:19,971 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c5b898abf66a8f4d9bccf2/log.txt.gz 2026-03-26 22:54:20,135 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 22:54:20,333 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 22:54:20,335 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c5b898abf66a8f4d9bccf2/lava-logs.yaml 2026-03-26 22:54:20,342 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 22:54:20,465 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 22:54:20,467 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c5b898abf66a8f4d9bccf2/lava_callback.json.gz 2026-03-26 22:54:20,491 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:54:20,514 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b898abf66a8f4d9bccf2 HTTP/1.1" 200 None 2026-03-26 22:54:20,515 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:54:20,529 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-03-26 22:54:20,531 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:54:20,555 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5b898abf66a8f4d9bccf2 HTTP/1.1" 200 None 2026-03-26 22:54:20,556 - lava_callback - INFO - Completed processing callback for node 69c5b898abf66a8f4d9bccf2 2026-03-26 22:54:21,586 - lava_callback - INFO - Received callback for node 69c316156778148209fd6c75 INFO: 172.21.0.1:36516 - "POST /node/69c316156778148209fd6c75 HTTP/1.0" 202 Accepted 2026-03-26 22:54:21,592 - lava_callback - INFO - Starting async job submit for node 69c316156778148209fd6c75 2026-03-26 22:54:21,593 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:54:21,608 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c316156778148209fd6c75 HTTP/1.1" 200 None 2026-03-26 22:54:21,609 - lava_callback - DEBUG - Processing node 69c316156778148209fd6c75: baseline-arm-baylibre 2026-03-26 22:54:21,609 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c316156778148209fd6c75 2026-03-26 22:54:21,610 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:54:21,626 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c316156778148209fd6c75 HTTP/1.1" 200 None 2026-03-26 22:54:21,628 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:54:21,651 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01633c3d73288b0d052f0 HTTP/1.1" 200 None 2026-03-26 22:54:21,652 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:54:21,672 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c316156778148209fd6c75 HTTP/1.1" 200 None 2026-03-26 22:54:21,673 - lava_callback - ERROR - Error processing callback for node 69c316156778148209fd6c75: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:36526 - "GET / HTTP/1.0" 200 OK 2026-03-26 22:54:33,188 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:54:33,206 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 INFO: 172.21.0.1:48646 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40514 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36562 - "GET / HTTP/1.0" 200 OK 2026-03-26 22:55:01,777 - lava_callback - INFO - Received callback for node 69c3163d6778148209fd6c86 INFO: 172.21.0.1:38918 - "POST /node/69c3163d6778148209fd6c86 HTTP/1.0" 202 Accepted 2026-03-26 22:55:01,786 - lava_callback - INFO - Starting async job submit for node 69c3163d6778148209fd6c86 2026-03-26 22:55:01,788 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:55:01,807 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3163d6778148209fd6c86 HTTP/1.1" 200 None 2026-03-26 22:55:01,808 - lava_callback - DEBUG - Processing node 69c3163d6778148209fd6c86: baseline-arm-baylibre 2026-03-26 22:55:01,808 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3163d6778148209fd6c86 2026-03-26 22:55:01,809 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:55:01,823 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3163d6778148209fd6c86 HTTP/1.1" 200 None 2026-03-26 22:55:01,824 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:55:01,837 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01633c3d73288b0d052f0 HTTP/1.1" 200 None 2026-03-26 22:55:01,839 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:55:01,862 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3163d6778148209fd6c86 HTTP/1.1" 200 None 2026-03-26 22:55:01,868 - lava_callback - ERROR - Error processing callback for node 69c3163d6778148209fd6c86: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:38928 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55212 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55226 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55236 - "GET / HTTP/1.0" 200 OK 2026-03-26 22:55:19,951 - lava_callback - INFO - Received callback for node 69c5b86aabf66a8f4d9bcc58 INFO: 172.21.0.1:36294 - "POST /node/69c5b86aabf66a8f4d9bcc58 HTTP/1.0" 202 Accepted 2026-03-26 22:55:19,957 - lava_callback - INFO - Starting async job submit for node 69c5b86aabf66a8f4d9bcc58 2026-03-26 22:55:19,974 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:55:19,989 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b86aabf66a8f4d9bcc58 HTTP/1.1" 200 None 2026-03-26 22:55:19,989 - lava_callback - DEBUG - Processing node 69c5b86aabf66a8f4d9bcc58: baseline-arm64 2026-03-26 22:55:20,316 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 22:55:20,616 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 22:55:20,618 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c5b86aabf66a8f4d9bcc58/log.txt.gz 2026-03-26 22:55:20,820 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 22:55:21,021 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 22:55:21,024 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c5b86aabf66a8f4d9bcc58/lava-logs.yaml 2026-03-26 22:55:21,032 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 22:55:21,195 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 22:55:21,196 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c5b86aabf66a8f4d9bcc58/lava_callback.json.gz 2026-03-26 22:55:21,215 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:55:21,230 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b86aabf66a8f4d9bcc58 HTTP/1.1" 200 None 2026-03-26 22:55:21,232 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:55:21,245 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-03-26 22:55:21,247 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:55:21,276 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5b86aabf66a8f4d9bcc58 HTTP/1.1" 200 None 2026-03-26 22:55:21,283 - lava_callback - INFO - Completed processing callback for node 69c5b86aabf66a8f4d9bcc58 INFO: 172.21.0.1:36310 - "GET / HTTP/1.0" 200 OK 2026-03-26 22:55:33,208 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:55:33,224 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:52230 - "GET / HTTP/1.0" 200 OK 2026-03-26 22:55:41,623 - lava_callback - INFO - Received callback for node 69c316636778148209fd6c94 INFO: 172.21.0.1:56292 - "POST /node/69c316636778148209fd6c94 HTTP/1.0" 202 Accepted 2026-03-26 22:55:41,629 - lava_callback - INFO - Starting async job submit for node 69c316636778148209fd6c94 2026-03-26 22:55:41,629 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:55:41,643 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c316636778148209fd6c94 HTTP/1.1" 200 None 2026-03-26 22:55:41,644 - lava_callback - DEBUG - Processing node 69c316636778148209fd6c94: baseline-arm-baylibre 2026-03-26 22:55:41,644 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c316636778148209fd6c94 2026-03-26 22:55:41,645 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:55:41,659 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c316636778148209fd6c94 HTTP/1.1" 200 None 2026-03-26 22:55:41,661 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:55:41,681 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01633c3d73288b0d052f0 HTTP/1.1" 200 None 2026-03-26 22:55:41,683 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:55:41,703 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c316636778148209fd6c94 HTTP/1.1" 200 None 2026-03-26 22:55:41,713 - lava_callback - ERROR - Error processing callback for node 69c316636778148209fd6c94: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:56296 - "GET / HTTP/1.0" 200 OK 2026-03-26 22:55:49,528 - lava_callback - INFO - Received callback for node 69c5b89aabf66a8f4d9bcd15 INFO: 172.21.0.1:56306 - "POST /node/69c5b89aabf66a8f4d9bcd15 HTTP/1.0" 202 Accepted 2026-03-26 22:55:49,534 - lava_callback - INFO - Starting async job submit for node 69c5b89aabf66a8f4d9bcd15 2026-03-26 22:55:49,557 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:55:49,573 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b89aabf66a8f4d9bcd15 HTTP/1.1" 200 None 2026-03-26 22:55:49,574 - lava_callback - DEBUG - Processing node 69c5b89aabf66a8f4d9bcd15: baseline-arm64-broonie 2026-03-26 22:55:49,856 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 22:55:49,992 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 22:55:49,993 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c5b89aabf66a8f4d9bcd15/log.txt.gz 2026-03-26 22:55:50,154 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 22:55:50,356 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 22:55:50,359 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c5b89aabf66a8f4d9bcd15/lava-logs.yaml 2026-03-26 22:55:50,365 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 22:55:50,496 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 22:55:50,498 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c5b89aabf66a8f4d9bcd15/lava_callback.json.gz 2026-03-26 22:55:50,521 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:55:50,534 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b89aabf66a8f4d9bcd15 HTTP/1.1" 200 None 2026-03-26 22:55:50,536 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:55:50,549 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-03-26 22:55:50,550 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:55:50,583 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5b89aabf66a8f4d9bcd15 HTTP/1.1" 200 None 2026-03-26 22:55:50,584 - lava_callback - INFO - Completed processing callback for node 69c5b89aabf66a8f4d9bcd15 INFO: 172.21.0.1:47124 - "GET / HTTP/1.0" 200 OK 2026-03-26 22:56:03,226 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:56:03,255 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:59256 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47332 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47346 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47358 - "GET / HTTP/1.0" 200 OK 2026-03-26 22:56:21,735 - lava_callback - INFO - Received callback for node 69c316866778148209fd6ca2 INFO: 172.21.0.1:46060 - "POST /node/69c316866778148209fd6ca2 HTTP/1.0" 202 Accepted 2026-03-26 22:56:21,740 - lava_callback - INFO - Starting async job submit for node 69c316866778148209fd6ca2 2026-03-26 22:56:21,741 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:56:21,760 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c316866778148209fd6ca2 HTTP/1.1" 200 None 2026-03-26 22:56:21,761 - lava_callback - DEBUG - Processing node 69c316866778148209fd6ca2: baseline-arm-baylibre 2026-03-26 22:56:21,761 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c316866778148209fd6ca2 2026-03-26 22:56:21,762 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:56:21,780 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c316866778148209fd6ca2 HTTP/1.1" 200 None 2026-03-26 22:56:21,781 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:56:21,797 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01633c3d73288b0d052f0 HTTP/1.1" 200 None 2026-03-26 22:56:21,798 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:56:21,828 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c316866778148209fd6ca2 HTTP/1.1" 200 None 2026-03-26 22:56:21,830 - lava_callback - ERROR - Error processing callback for node 69c316866778148209fd6ca2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:46062 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45354 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57920 - "GET / HTTP/1.0" 200 OK 2026-03-26 22:56:55,842 - lava_callback - INFO - Received callback for node 69c316b26778148209fd6cb2 INFO: 172.21.0.1:53058 - "POST /node/69c316b26778148209fd6cb2 HTTP/1.0" 202 Accepted 2026-03-26 22:56:55,848 - lava_callback - INFO - Starting async job submit for node 69c316b26778148209fd6cb2 2026-03-26 22:56:55,848 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:56:55,862 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c316b26778148209fd6cb2 HTTP/1.1" 200 None 2026-03-26 22:56:55,863 - lava_callback - DEBUG - Processing node 69c316b26778148209fd6cb2: baseline-arm-clabbe 2026-03-26 22:56:55,864 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c316b26778148209fd6cb2 2026-03-26 22:56:55,864 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:56:55,878 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c316b26778148209fd6cb2 HTTP/1.1" 200 None 2026-03-26 22:56:55,879 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:56:55,893 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01633c3d73288b0d052f0 HTTP/1.1" 200 None 2026-03-26 22:56:55,894 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:56:55,914 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c316b26778148209fd6cb2 HTTP/1.1" 200 None 2026-03-26 22:56:55,923 - lava_callback - ERROR - Error processing callback for node 69c316b26778148209fd6cb2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:53066 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54540 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48630 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48632 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48648 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39342 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51446 - "GET / HTTP/1.0" 200 OK 2026-03-26 22:57:41,878 - lava_callback - INFO - Received callback for node 69c316d46778148209fd6cbd INFO: 172.21.0.1:51670 - "POST /node/69c316d46778148209fd6cbd HTTP/1.0" 202 Accepted 2026-03-26 22:57:41,883 - lava_callback - INFO - Starting async job submit for node 69c316d46778148209fd6cbd 2026-03-26 22:57:41,884 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:57:41,909 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c316d46778148209fd6cbd HTTP/1.1" 200 None 2026-03-26 22:57:41,910 - lava_callback - DEBUG - Processing node 69c316d46778148209fd6cbd: baseline-arm-baylibre 2026-03-26 22:57:41,910 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c316d46778148209fd6cbd 2026-03-26 22:57:41,911 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:57:41,924 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c316d46778148209fd6cbd HTTP/1.1" 200 None 2026-03-26 22:57:41,925 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:57:41,939 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01633c3d73288b0d052f0 HTTP/1.1" 200 None 2026-03-26 22:57:41,941 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:57:41,968 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c316d46778148209fd6cbd HTTP/1.1" 200 None 2026-03-26 22:57:41,972 - lava_callback - ERROR - Error processing callback for node 69c316d46778148209fd6cbd: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:51678 - "GET / HTTP/1.0" 200 OK 2026-03-26 22:57:50,471 - lava_callback - INFO - Received callback for node 69c50ef9f63ce05331e2b4c6 INFO: 172.21.0.1:59614 - "POST /node/69c50ef9f63ce05331e2b4c6 HTTP/1.0" 202 Accepted 2026-03-26 22:57:50,477 - lava_callback - INFO - Starting async job submit for node 69c50ef9f63ce05331e2b4c6 2026-03-26 22:57:50,501 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:57:50,515 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c50ef9f63ce05331e2b4c6 HTTP/1.1" 200 None 2026-03-26 22:57:50,516 - lava_callback - DEBUG - Processing node 69c50ef9f63ce05331e2b4c6: baseline-arm64 2026-03-26 22:57:51,015 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 22:57:51,138 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 22:57:51,140 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c50ef9f63ce05331e2b4c6/log.txt.gz 2026-03-26 22:57:51,416 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 22:57:51,643 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 22:57:51,645 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c50ef9f63ce05331e2b4c6/lava-logs.yaml 2026-03-26 22:57:51,656 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 22:57:51,836 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 22:57:51,837 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c50ef9f63ce05331e2b4c6/lava_callback.json.gz 2026-03-26 22:57:51,862 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:57:51,877 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c50ef9f63ce05331e2b4c6 HTTP/1.1" 200 None 2026-03-26 22:57:51,878 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:57:51,901 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399cdbbc022e792643ea0 HTTP/1.1" 200 None 2026-03-26 22:57:51,903 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:57:51,926 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c50ef9f63ce05331e2b4c6 HTTP/1.1" 200 None 2026-03-26 22:57:52,010 - lava_callback - INFO - Completed processing callback for node 69c50ef9f63ce05331e2b4c6 INFO: 172.21.0.1:59622 - "GET / HTTP/1.0" 200 OK 2026-03-26 22:58:03,257 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:58:03,273 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:35498 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53304 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53306 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53320 - "GET / HTTP/1.0" 200 OK 2026-03-26 22:58:21,789 - lava_callback - INFO - Received callback for node 69c316fa6778148209fd6ccf INFO: 172.21.0.1:55798 - "POST /node/69c316fa6778148209fd6ccf HTTP/1.0" 202 Accepted 2026-03-26 22:58:21,794 - lava_callback - INFO - Starting async job submit for node 69c316fa6778148209fd6ccf 2026-03-26 22:58:21,795 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:58:21,830 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c316fa6778148209fd6ccf HTTP/1.1" 200 None 2026-03-26 22:58:21,831 - lava_callback - DEBUG - Processing node 69c316fa6778148209fd6ccf: baseline-arm-baylibre 2026-03-26 22:58:21,831 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c316fa6778148209fd6ccf 2026-03-26 22:58:21,832 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:58:21,845 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c316fa6778148209fd6ccf HTTP/1.1" 200 None 2026-03-26 22:58:21,846 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:58:21,859 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01633c3d73288b0d052f0 HTTP/1.1" 200 None 2026-03-26 22:58:21,860 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:58:21,890 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c316fa6778148209fd6ccf HTTP/1.1" 200 None 2026-03-26 22:58:21,891 - lava_callback - ERROR - Error processing callback for node 69c316fa6778148209fd6ccf: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:55802 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43220 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40356 - "GET / HTTP/1.0" 200 OK 2026-03-26 22:58:46,546 - lava_callback - INFO - Received callback for node 69c5b8afabf66a8f4d9bcd47 INFO: 172.21.0.1:40360 - "POST /node/69c5b8afabf66a8f4d9bcd47 HTTP/1.0" 202 Accepted 2026-03-26 22:58:46,555 - lava_callback - INFO - Starting async job submit for node 69c5b8afabf66a8f4d9bcd47 2026-03-26 22:58:46,613 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:58:46,634 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b8afabf66a8f4d9bcd47 HTTP/1.1" 200 None 2026-03-26 22:58:46,635 - lava_callback - DEBUG - Processing node 69c5b8afabf66a8f4d9bcd47: baseline-arm64-foundriesio 2026-03-26 22:58:47,825 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 22:58:48,147 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 22:58:48,149 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69c5b8afabf66a8f4d9bcd47/log.txt.gz 2026-03-26 22:58:48,870 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 22:58:49,128 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 22:58:49,130 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69c5b8afabf66a8f4d9bcd47/lava-logs.yaml 2026-03-26 22:58:49,158 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 22:58:49,343 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 22:58:49,345 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69c5b8afabf66a8f4d9bcd47/lava_callback.json.gz DEBUG: Setup failed for 69c5b8afabf66a8f4d9bcd47. Transitting job node result: incomplete -> fail 2026-03-26 22:58:49,451 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:58:49,466 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b8afabf66a8f4d9bcd47 HTTP/1.1" 200 None 2026-03-26 22:58:49,467 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:58:49,481 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-03-26 22:58:49,482 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:58:49,507 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5b8afabf66a8f4d9bcd47 HTTP/1.1" 200 None 2026-03-26 22:58:49,627 - lava_callback - INFO - Completed processing callback for node 69c5b8afabf66a8f4d9bcd47 2026-03-26 22:58:55,945 - lava_callback - INFO - Received callback for node 69c317246778148209fd6ce7 INFO: 172.21.0.1:38534 - "POST /node/69c317246778148209fd6ce7 HTTP/1.0" 202 Accepted 2026-03-26 22:58:55,950 - lava_callback - INFO - Starting async job submit for node 69c317246778148209fd6ce7 2026-03-26 22:58:55,951 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:58:55,985 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c317246778148209fd6ce7 HTTP/1.1" 200 None 2026-03-26 22:58:55,986 - lava_callback - DEBUG - Processing node 69c317246778148209fd6ce7: baseline-arm-clabbe 2026-03-26 22:58:55,986 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c317246778148209fd6ce7 2026-03-26 22:58:55,987 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:58:56,001 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c317246778148209fd6ce7 HTTP/1.1" 200 None 2026-03-26 22:58:56,002 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:58:56,016 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01633c3d73288b0d052f0 HTTP/1.1" 200 None 2026-03-26 22:58:56,018 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:58:56,038 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c317246778148209fd6ce7 HTTP/1.1" 200 None 2026-03-26 22:58:56,039 - lava_callback - ERROR - Error processing callback for node 69c317246778148209fd6ce7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:38550 - "GET / HTTP/1.0" 200 OK 2026-03-26 22:59:03,275 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:59:03,291 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:37834 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36804 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36806 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36816 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46126 - "GET / HTTP/1.0" 200 OK 2026-03-26 22:59:33,969 - lava_callback - INFO - Received callback for node 69c5b878abf66a8f4d9bcc7c INFO: 172.21.0.1:50830 - "POST /node/69c5b878abf66a8f4d9bcc7c HTTP/1.0" 202 Accepted 2026-03-26 22:59:33,976 - lava_callback - INFO - Starting async job submit for node 69c5b878abf66a8f4d9bcc7c 2026-03-26 22:59:34,014 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:59:34,039 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b878abf66a8f4d9bcc7c HTTP/1.1" 200 None 2026-03-26 22:59:34,040 - lava_callback - DEBUG - Processing node 69c5b878abf66a8f4d9bcc7c: baseline-arm64 2026-03-26 22:59:34,401 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 22:59:34,563 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 22:59:34,564 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c5b878abf66a8f4d9bcc7c/log.txt.gz 2026-03-26 22:59:34,783 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 22:59:35,034 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 22:59:35,036 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c5b878abf66a8f4d9bcc7c/lava-logs.yaml 2026-03-26 22:59:35,045 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 22:59:35,198 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 22:59:35,199 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c5b878abf66a8f4d9bcc7c/lava_callback.json.gz DEBUG: Setup failed for 69c5b878abf66a8f4d9bcc7c. Transitting job node result: incomplete -> fail 2026-03-26 22:59:35,290 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:59:35,310 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b878abf66a8f4d9bcc7c HTTP/1.1" 200 None 2026-03-26 22:59:35,312 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:59:35,324 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-03-26 22:59:35,326 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 22:59:35,349 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5b878abf66a8f4d9bcc7c HTTP/1.1" 200 None 2026-03-26 22:59:35,438 - lava_callback - INFO - Completed processing callback for node 69c5b878abf66a8f4d9bcc7c INFO: 172.21.0.1:50844 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36546 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39098 - "GET / HTTP/1.0" 200 OK 2026-03-26 23:00:03,294 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:00:03,310 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:49340 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57272 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57284 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-26 23:00:16,166 - lava_callback - INFO - Received callback for node 69c50f98f63ce05331e2b4ef INFO: 172.21.0.1:57290 - "POST /node/69c50f98f63ce05331e2b4ef HTTP/1.0" 202 Accepted 2026-03-26 23:00:16,173 - lava_callback - INFO - Starting async job submit for node 69c50f98f63ce05331e2b4ef 2026-03-26 23:00:16,198 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:00:16,222 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c50f98f63ce05331e2b4ef HTTP/1.1" 200 None 2026-03-26 23:00:16,223 - lava_callback - DEBUG - Processing node 69c50f98f63ce05331e2b4ef: baseline-nfs-arm64 2026-03-26 23:00:16,240 - lava_callback - INFO - Received callback for node 69c3176f6778148209fd6d2c INFO: 172.21.0.1:57300 - "POST /node/69c3176f6778148209fd6d2c HTTP/1.0" 202 Accepted 2026-03-26 23:00:16,256 - lava_callback - INFO - Starting async job submit for node 69c3176f6778148209fd6d2c 2026-03-26 23:00:16,277 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:00:16,403 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3176f6778148209fd6d2c HTTP/1.1" 200 None 2026-03-26 23:00:16,405 - lava_callback - DEBUG - Processing node 69c3176f6778148209fd6d2c: baseline-arm-clabbe 2026-03-26 23:00:16,406 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3176f6778148209fd6d2c 2026-03-26 23:00:16,407 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:00:16,483 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3176f6778148209fd6d2c HTTP/1.1" 200 None 2026-03-26 23:00:16,521 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.21.0.1:57310 - "GET / HTTP/1.0" 200 OK 2026-03-26 23:00:16,594 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01633c3d73288b0d052f0 HTTP/1.1" 200 None 2026-03-26 23:00:16,632 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:00:16,726 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3176f6778148209fd6d2c HTTP/1.1" 200 None 2026-03-26 23:00:16,768 - lava_callback - ERROR - Error processing callback for node 69c3176f6778148209fd6d2c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-26 23:00:16,893 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 23:00:17,178 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 23:00:17,180 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c50f98f63ce05331e2b4ef/log.txt.gz 2026-03-26 23:00:17,451 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 23:00:17,642 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 23:00:17,644 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c50f98f63ce05331e2b4ef/lava-logs.yaml 2026-03-26 23:00:17,656 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 23:00:17,849 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 23:00:17,850 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c50f98f63ce05331e2b4ef/lava_callback.json.gz 2026-03-26 23:00:17,894 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:00:17,908 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c50f98f63ce05331e2b4ef HTTP/1.1" 200 None 2026-03-26 23:00:17,910 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:00:17,924 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399cdbbc022e792643ea0 HTTP/1.1" 200 None 2026-03-26 23:00:17,926 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:00:17,946 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c50f98f63ce05331e2b4ef HTTP/1.1" 200 None 2026-03-26 23:00:18,021 - lava_callback - INFO - Completed processing callback for node 69c50f98f63ce05331e2b4ef INFO: 172.21.0.1:48936 - "GET / HTTP/1.0" 200 OK 2026-03-26 23:00:33,313 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:00:33,329 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:41532 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46518 - "GET / HTTP/1.0" 200 OK 2026-03-26 23:00:56,025 - lava_callback - INFO - Received callback for node 69c317976778148209fd6d3b INFO: 172.21.0.1:60634 - "POST /node/69c317976778148209fd6d3b HTTP/1.0" 202 Accepted 2026-03-26 23:00:56,031 - lava_callback - INFO - Starting async job submit for node 69c317976778148209fd6d3b 2026-03-26 23:00:56,031 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:00:56,046 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c317976778148209fd6d3b HTTP/1.1" 200 None 2026-03-26 23:00:56,047 - lava_callback - DEBUG - Processing node 69c317976778148209fd6d3b: baseline-arm-clabbe 2026-03-26 23:00:56,047 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c317976778148209fd6d3b 2026-03-26 23:00:56,048 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:00:56,061 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c317976778148209fd6d3b HTTP/1.1" 200 None 2026-03-26 23:00:56,062 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:00:56,076 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01633c3d73288b0d052f0 HTTP/1.1" 200 None 2026-03-26 23:00:56,077 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:00:56,113 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c317976778148209fd6d3b HTTP/1.1" 200 None 2026-03-26 23:00:56,115 - lava_callback - ERROR - Error processing callback for node 69c317976778148209fd6d3b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:60638 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43566 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47924 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47936 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47942 - "GET / HTTP/1.0" 200 OK 2026-03-26 23:01:22,010 - lava_callback - INFO - Received callback for node 69c317ba6778148209fd6d4a INFO: 172.21.0.1:41374 - "POST /node/69c317ba6778148209fd6d4a HTTP/1.0" 202 Accepted 2026-03-26 23:01:22,016 - lava_callback - INFO - Starting async job submit for node 69c317ba6778148209fd6d4a 2026-03-26 23:01:22,016 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:01:22,035 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c317ba6778148209fd6d4a HTTP/1.1" 200 None 2026-03-26 23:01:22,036 - lava_callback - DEBUG - Processing node 69c317ba6778148209fd6d4a: baseline-arm-baylibre 2026-03-26 23:01:22,037 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c317ba6778148209fd6d4a 2026-03-26 23:01:22,037 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:01:22,050 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c317ba6778148209fd6d4a HTTP/1.1" 200 None 2026-03-26 23:01:22,051 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:01:22,064 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01633c3d73288b0d052f0 HTTP/1.1" 200 None 2026-03-26 23:01:22,066 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:01:22,092 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c317ba6778148209fd6d4a HTTP/1.1" 200 None 2026-03-26 23:01:22,093 - lava_callback - ERROR - Error processing callback for node 69c317ba6778148209fd6d4a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:41386 - "GET / HTTP/1.0" 200 OK 2026-03-26 23:01:32,461 - lava_callback - INFO - Received callback for node 69c5154bf63ce05331e2b62b INFO: 172.21.0.1:60932 - "POST /node/69c5154bf63ce05331e2b62b HTTP/1.0" 202 Accepted 2026-03-26 23:01:32,468 - lava_callback - INFO - Starting async job submit for node 69c5154bf63ce05331e2b62b 2026-03-26 23:01:32,493 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:01:32,508 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5154bf63ce05331e2b62b HTTP/1.1" 200 None 2026-03-26 23:01:32,508 - lava_callback - DEBUG - Processing node 69c5154bf63ce05331e2b62b: baseline-nfs-arm64 2026-03-26 23:01:33,025 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 23:01:33,341 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 23:01:33,343 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c5154bf63ce05331e2b62b/log.txt.gz 2026-03-26 23:01:33,610 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 23:01:34,007 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 23:01:34,010 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c5154bf63ce05331e2b62b/lava-logs.yaml 2026-03-26 23:01:34,021 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 23:01:34,191 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 23:01:34,193 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c5154bf63ce05331e2b62b/lava_callback.json.gz 2026-03-26 23:01:34,248 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:01:34,263 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5154bf63ce05331e2b62b HTTP/1.1" 200 None 2026-03-26 23:01:34,264 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:01:34,284 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ad9944faaf0e7f5e41d HTTP/1.1" 200 None 2026-03-26 23:01:34,285 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:01:34,312 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5154bf63ce05331e2b62b HTTP/1.1" 200 None 2026-03-26 23:01:34,384 - lava_callback - INFO - Completed processing callback for node 69c5154bf63ce05331e2b62b INFO: 172.21.0.1:60948 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49760 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42844 - "GET / HTTP/1.0" 200 OK 2026-03-26 23:02:03,331 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:02:03,348 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:47130 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41834 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41844 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-26 23:02:16,124 - lava_callback - INFO - Received callback for node 69c317e36778148209fd6d57 INFO: 172.21.0.1:41846 - "POST /node/69c317e36778148209fd6d57 HTTP/1.0" 202 Accepted 2026-03-26 23:02:16,130 - lava_callback - INFO - Starting async job submit for node 69c317e36778148209fd6d57 2026-03-26 23:02:16,131 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:02:16,148 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c317e36778148209fd6d57 HTTP/1.1" 200 None 2026-03-26 23:02:16,149 - lava_callback - DEBUG - Processing node 69c317e36778148209fd6d57: baseline-arm-clabbe 2026-03-26 23:02:16,150 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c317e36778148209fd6d57 2026-03-26 23:02:16,150 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:02:16,163 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c317e36778148209fd6d57 HTTP/1.1" 200 None 2026-03-26 23:02:16,165 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:02:16,178 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01633c3d73288b0d052f0 HTTP/1.1" 200 None 2026-03-26 23:02:16,180 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:02:16,201 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c317e36778148209fd6d57 HTTP/1.1" 200 None 2026-03-26 23:02:16,208 - lava_callback - ERROR - Error processing callback for node 69c317e36778148209fd6d57: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:41850 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47178 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52446 - "GET / HTTP/1.0" 200 OK 2026-03-26 23:02:42,068 - lava_callback - INFO - Received callback for node 69c318076778148209fd6d62 INFO: 172.21.0.1:50854 - "POST /node/69c318076778148209fd6d62 HTTP/1.0" 202 Accepted 2026-03-26 23:02:42,074 - lava_callback - INFO - Starting async job submit for node 69c318076778148209fd6d62 2026-03-26 23:02:42,075 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:02:42,090 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c318076778148209fd6d62 HTTP/1.1" 200 None 2026-03-26 23:02:42,091 - lava_callback - DEBUG - Processing node 69c318076778148209fd6d62: baseline-arm-baylibre 2026-03-26 23:02:42,091 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c318076778148209fd6d62 2026-03-26 23:02:42,092 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:02:42,106 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c318076778148209fd6d62 HTTP/1.1" 200 None 2026-03-26 23:02:42,108 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:02:42,128 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01633c3d73288b0d052f0 HTTP/1.1" 200 None 2026-03-26 23:02:42,130 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:02:42,166 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c318076778148209fd6d62 HTTP/1.1" 200 None 2026-03-26 23:02:42,168 - lava_callback - ERROR - Error processing callback for node 69c318076778148209fd6d62: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:50856 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59834 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60868 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49732 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49734 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-26 23:03:16,214 - lava_callback - INFO - Received callback for node 69c3182f6778148209fd6d6d INFO: 172.21.0.1:49740 - "POST /node/69c3182f6778148209fd6d6d HTTP/1.0" 202 Accepted 2026-03-26 23:03:16,220 - lava_callback - INFO - Starting async job submit for node 69c3182f6778148209fd6d6d 2026-03-26 23:03:16,221 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:03:16,246 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3182f6778148209fd6d6d HTTP/1.1" 200 None 2026-03-26 23:03:16,247 - lava_callback - DEBUG - Processing node 69c3182f6778148209fd6d6d: baseline-arm-clabbe 2026-03-26 23:03:16,247 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3182f6778148209fd6d6d 2026-03-26 23:03:16,248 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:03:16,261 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3182f6778148209fd6d6d HTTP/1.1" 200 None 2026-03-26 23:03:16,263 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:03:16,276 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01633c3d73288b0d052f0 HTTP/1.1" 200 None 2026-03-26 23:03:16,277 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:03:16,298 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3182f6778148209fd6d6d HTTP/1.1" 200 None 2026-03-26 23:03:16,299 - lava_callback - ERROR - Error processing callback for node 69c3182f6778148209fd6d6d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:49744 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59318 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58794 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52308 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38100 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50116 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56118 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56122 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56134 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51270 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42800 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52064 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58244 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57392 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58370 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58380 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58388 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46338 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47004 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34348 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38302 - "GET / HTTP/1.0" 200 OK 2026-03-26 23:05:56,889 - lava_callback - INFO - Received callback for node 69c5bbceabf66a8f4d9bcf3b INFO: 172.21.0.1:38318 - "POST /node/69c5bbceabf66a8f4d9bcf3b HTTP/1.0" 202 Accepted 2026-03-26 23:05:56,897 - lava_callback - INFO - Starting async job submit for node 69c5bbceabf66a8f4d9bcf3b 2026-03-26 23:05:56,899 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:05:56,918 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5bbceabf66a8f4d9bcf3b HTTP/1.1" 200 None 2026-03-26 23:05:56,919 - lava_callback - DEBUG - Processing node 69c5bbceabf66a8f4d9bcf3b: baseline-nfs-arm64 2026-03-26 23:05:56,921 - lava_callback - WARNING - Failed to upload log 2026-03-26 23:05:56,922 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 23:05:57,071 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 23:05:57,072 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c5bbceabf66a8f4d9bcf3b/lava-logs.yaml 2026-03-26 23:05:57,074 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 23:05:57,196 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 23:05:57,198 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c5bbceabf66a8f4d9bcf3b/lava_callback.json.gz 2026-03-26 23:05:57,200 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:05:57,214 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5bbceabf66a8f4d9bcf3b HTTP/1.1" 200 None 2026-03-26 23:05:57,215 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:05:57,231 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-03-26 23:05:57,233 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:05:57,253 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5bbceabf66a8f4d9bcf3b HTTP/1.1" 200 None 2026-03-26 23:05:57,257 - lava_callback - INFO - Completed processing callback for node 69c5bbceabf66a8f4d9bcf3b 2026-03-26 23:06:03,351 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:06:03,367 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:48194 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48000 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48002 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48018 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51388 - "GET / HTTP/1.0" 200 OK 2026-03-26 23:06:32,055 - lava_callback - INFO - Received callback for node 69c5b877abf66a8f4d9bcc79 INFO: 172.21.0.1:55586 - "POST /node/69c5b877abf66a8f4d9bcc79 HTTP/1.0" 202 Accepted 2026-03-26 23:06:32,063 - lava_callback - INFO - Starting async job submit for node 69c5b877abf66a8f4d9bcc79 2026-03-26 23:06:32,111 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:06:32,126 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b877abf66a8f4d9bcc79 HTTP/1.1" 200 None 2026-03-26 23:06:32,126 - lava_callback - DEBUG - Processing node 69c5b877abf66a8f4d9bcc79: baseline-arm64 2026-03-26 23:06:32,844 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 23:06:32,991 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 23:06:32,993 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c5b877abf66a8f4d9bcc79/log.txt.gz 2026-03-26 23:06:33,291 - lava_callback - INFO - Received callback for node 69c5b888abf66a8f4d9bccc6 INFO: 172.21.0.1:55590 - "POST /node/69c5b888abf66a8f4d9bccc6 HTTP/1.0" 202 Accepted 2026-03-26 23:06:33,323 - lava_callback - INFO - Starting async job submit for node 69c5b888abf66a8f4d9bccc6 2026-03-26 23:06:33,440 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:06:33,502 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b888abf66a8f4d9bccc6 HTTP/1.1" 200 None 2026-03-26 23:06:33,503 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 23:06:33,504 - lava_callback - DEBUG - Processing node 69c5b888abf66a8f4d9bccc6: baseline-nfs-arm64 2026-03-26 23:06:33,983 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 23:06:34,021 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c5b877abf66a8f4d9bcc79/lava-logs.yaml 2026-03-26 23:06:34,119 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 23:06:34,325 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 23:06:34,429 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 23:06:34,430 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c5b877abf66a8f4d9bcc79/lava_callback.json.gz 2026-03-26 23:06:34,491 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 DEBUG: Setup failed for 69c5b877abf66a8f4d9bcc79. Transitting job node result: incomplete -> fail 2026-03-26 23:06:34,533 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:06:34,534 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c5b888abf66a8f4d9bccc6/log.txt.gz 2026-03-26 23:06:34,597 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b877abf66a8f4d9bcc79 HTTP/1.1" 200 None 2026-03-26 23:06:34,616 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:06:34,681 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-03-26 23:06:34,701 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:06:34,785 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5b877abf66a8f4d9bcc79 HTTP/1.1" 200 None 2026-03-26 23:06:34,960 - lava_callback - INFO - Completed processing callback for node 69c5b877abf66a8f4d9bcc79 2026-03-26 23:06:35,145 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 23:06:35,401 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 23:06:35,404 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c5b888abf66a8f4d9bccc6/lava-logs.yaml 2026-03-26 23:06:35,420 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 23:06:35,587 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 23:06:35,589 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c5b888abf66a8f4d9bccc6/lava_callback.json.gz DEBUG: Setup failed for 69c5b888abf66a8f4d9bccc6. Transitting job node result: incomplete -> fail 2026-03-26 23:06:35,688 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:06:35,703 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b888abf66a8f4d9bccc6 HTTP/1.1" 200 None 2026-03-26 23:06:35,704 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:06:35,717 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-03-26 23:06:35,719 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:06:35,749 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5b888abf66a8f4d9bccc6 HTTP/1.1" 200 None 2026-03-26 23:06:35,855 - lava_callback - INFO - Completed processing callback for node 69c5b888abf66a8f4d9bccc6 INFO: 172.21.0.1:55594 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46690 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42738 - "GET / HTTP/1.0" 200 OK 2026-03-26 23:07:03,374 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:07:03,407 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:38602 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55446 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55462 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55468 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55802 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55668 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46716 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44868 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55706 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55870 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55882 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55886 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47534 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55080 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58326 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45324 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41682 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59322 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59338 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59354 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58832 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57884 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39190 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60568 - "GET / HTTP/1.0" 200 OK 2026-03-26 23:10:06,099 - lava_callback - INFO - Received callback for node 69c5bc98abf66a8f4d9bcf93 INFO: 172.21.0.1:32896 - "POST /node/69c5bc98abf66a8f4d9bcf93 HTTP/1.0" 202 Accepted 2026-03-26 23:10:06,105 - lava_callback - INFO - Starting async job submit for node 69c5bc98abf66a8f4d9bcf93 2026-03-26 23:10:06,134 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:10:06,149 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5bc98abf66a8f4d9bcf93 HTTP/1.1" 200 None 2026-03-26 23:10:06,149 - lava_callback - DEBUG - Processing node 69c5bc98abf66a8f4d9bcf93: baseline-arm64 2026-03-26 23:10:06,309 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 23:10:06,441 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 23:10:06,443 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c5bc98abf66a8f4d9bcf93/log.txt.gz 2026-03-26 23:10:06,516 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.21.0.1:32904 - "GET / HTTP/1.0" 200 OK 2026-03-26 23:10:06,696 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 23:10:06,698 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c5bc98abf66a8f4d9bcf93/lava-logs.yaml 2026-03-26 23:10:06,701 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 23:10:06,851 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 23:10:06,853 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c5bc98abf66a8f4d9bcf93/lava_callback.json.gz 2026-03-26 23:10:06,883 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:10:06,899 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5bc98abf66a8f4d9bcf93 HTTP/1.1" 200 None 2026-03-26 23:10:06,901 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:10:06,916 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-03-26 23:10:06,918 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:10:06,938 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5bc98abf66a8f4d9bcf93 HTTP/1.1" 200 None 2026-03-26 23:10:07,019 - lava_callback - INFO - Completed processing callback for node 69c5bc98abf66a8f4d9bcf93 INFO: 172.21.0.1:41116 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41132 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41142 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38730 - "GET / HTTP/1.0" 200 OK 2026-03-26 23:10:33,410 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:10:33,425 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:52148 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41046 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37374 - "GET / HTTP/1.0" 200 OK 2026-03-26 23:10:56,548 - lava_callback - INFO - Received callback for node 69c319ed6778148209fd6db6 INFO: 172.21.0.1:37376 - "POST /node/69c319ed6778148209fd6db6 HTTP/1.0" 202 Accepted 2026-03-26 23:10:56,553 - lava_callback - INFO - Starting async job submit for node 69c319ed6778148209fd6db6 2026-03-26 23:10:56,554 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:10:56,589 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c319ed6778148209fd6db6 HTTP/1.1" 200 None 2026-03-26 23:10:56,590 - lava_callback - DEBUG - Processing node 69c319ed6778148209fd6db6: baseline-arm-clabbe 2026-03-26 23:10:56,590 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c319ed6778148209fd6db6 2026-03-26 23:10:56,590 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:10:56,630 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c319ed6778148209fd6db6 HTTP/1.1" 200 None 2026-03-26 23:10:56,632 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:10:56,646 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01633c3d73288b0d052f0 HTTP/1.1" 200 None 2026-03-26 23:10:56,648 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:10:56,668 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c319ed6778148209fd6db6 HTTP/1.1" 200 None 2026-03-26 23:10:56,670 - lava_callback - ERROR - Error processing callback for node 69c319ed6778148209fd6db6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:50414 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38100 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38106 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38114 - "GET / HTTP/1.0" 200 OK 2026-03-26 23:11:22,580 - lava_callback - INFO - Received callback for node 69c31a176778148209fd6dc0 INFO: 172.21.0.1:47370 - "POST /node/69c31a176778148209fd6dc0 HTTP/1.0" 202 Accepted 2026-03-26 23:11:22,586 - lava_callback - INFO - Starting async job submit for node 69c31a176778148209fd6dc0 2026-03-26 23:11:22,586 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:11:22,603 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c31a176778148209fd6dc0 HTTP/1.1" 200 None 2026-03-26 23:11:22,604 - lava_callback - DEBUG - Processing node 69c31a176778148209fd6dc0: baseline-arm64-baylibre 2026-03-26 23:11:22,604 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c31a176778148209fd6dc0 2026-03-26 23:11:22,605 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:11:22,618 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c31a176778148209fd6dc0 HTTP/1.1" 200 None 2026-03-26 23:11:22,620 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:11:22,637 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b9626778148209fd337b HTTP/1.1" 200 None 2026-03-26 23:11:22,638 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:11:22,659 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c31a176778148209fd6dc0 HTTP/1.1" 200 None 2026-03-26 23:11:22,660 - lava_callback - ERROR - Error processing callback for node 69c31a176778148209fd6dc0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:47378 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41602 - "GET / HTTP/1.0" 200 OK 2026-03-26 23:11:36,748 - lava_callback - INFO - Received callback for node 69c5bcedabf66a8f4d9bcfa4 INFO: 172.21.0.1:41604 - "POST /node/69c5bcedabf66a8f4d9bcfa4 HTTP/1.0" 202 Accepted 2026-03-26 23:11:36,754 - lava_callback - INFO - Starting async job submit for node 69c5bcedabf66a8f4d9bcfa4 2026-03-26 23:11:36,784 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:11:36,798 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5bcedabf66a8f4d9bcfa4 HTTP/1.1" 200 None 2026-03-26 23:11:36,799 - lava_callback - DEBUG - Processing node 69c5bcedabf66a8f4d9bcfa4: baseline-arm64 2026-03-26 23:11:36,938 - lava_callback - WARNING - Failed to upload log 2026-03-26 23:11:37,008 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 23:11:37,350 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 23:11:37,352 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c5bcedabf66a8f4d9bcfa4/lava-logs.yaml 2026-03-26 23:11:37,356 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 23:11:37,502 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 23:11:37,504 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c5bcedabf66a8f4d9bcfa4/lava_callback.json.gz 2026-03-26 23:11:37,535 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:11:37,561 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5bcedabf66a8f4d9bcfa4 HTTP/1.1" 200 None 2026-03-26 23:11:37,563 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:11:37,578 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-03-26 23:11:37,580 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:11:37,600 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5bcedabf66a8f4d9bcfa4 HTTP/1.1" 200 None 2026-03-26 23:11:37,671 - lava_callback - INFO - Completed processing callback for node 69c5bcedabf66a8f4d9bcfa4 INFO: 172.21.0.1:45692 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34050 - "GET / HTTP/1.0" 200 OK 2026-03-26 23:12:03,427 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:12:03,442 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:34126 - "GET / HTTP/1.0" 200 OK 2026-03-26 23:12:10,263 - lava_callback - INFO - Received callback for node 69c5b87eabf66a8f4d9bccaa INFO: 172.21.0.1:35206 - "POST /node/69c5b87eabf66a8f4d9bccaa HTTP/1.0" 202 Accepted 2026-03-26 23:12:10,272 - lava_callback - INFO - Starting async job submit for node 69c5b87eabf66a8f4d9bccaa 2026-03-26 23:12:10,315 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:12:10,329 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b87eabf66a8f4d9bccaa HTTP/1.1" 200 None 2026-03-26 23:12:10,329 - lava_callback - DEBUG - Processing node 69c5b87eabf66a8f4d9bccaa: baseline-nfs-arm64 2026-03-26 23:12:11,487 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 23:12:11,632 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 23:12:11,633 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c5b87eabf66a8f4d9bccaa/log.txt.gz INFO: 172.21.0.1:35214 - "GET / HTTP/1.0" 200 OK 2026-03-26 23:12:12,295 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 23:12:12,513 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 23:12:12,516 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c5b87eabf66a8f4d9bccaa/lava-logs.yaml 2026-03-26 23:12:12,537 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 23:12:12,718 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 23:12:12,720 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c5b87eabf66a8f4d9bccaa/lava_callback.json.gz DEBUG: Setup failed for 69c5b87eabf66a8f4d9bccaa. Transitting job node result: incomplete -> fail 2026-03-26 23:12:12,812 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:12:12,831 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b87eabf66a8f4d9bccaa HTTP/1.1" 200 None 2026-03-26 23:12:12,833 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:12:12,848 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-03-26 23:12:12,849 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:12:12,872 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5b87eabf66a8f4d9bccaa HTTP/1.1" 200 None 2026-03-26 23:12:12,986 - lava_callback - INFO - Completed processing callback for node 69c5b87eabf66a8f4d9bccaa INFO: 172.21.0.1:35228 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35232 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44378 - "GET / HTTP/1.0" 200 OK 2026-03-26 23:12:33,444 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:12:33,463 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:35724 - "GET / HTTP/1.0" 200 OK 2026-03-26 23:12:42,589 - lava_callback - INFO - Received callback for node 69c31a666778148209fd6dcc INFO: 172.21.0.1:58194 - "POST /node/69c31a666778148209fd6dcc HTTP/1.0" 202 Accepted 2026-03-26 23:12:42,594 - lava_callback - INFO - Starting async job submit for node 69c31a666778148209fd6dcc 2026-03-26 23:12:42,595 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:12:42,621 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c31a666778148209fd6dcc HTTP/1.1" 200 None 2026-03-26 23:12:42,622 - lava_callback - DEBUG - Processing node 69c31a666778148209fd6dcc: baseline-arm64-baylibre 2026-03-26 23:12:42,622 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c31a666778148209fd6dcc 2026-03-26 23:12:42,623 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:12:42,656 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c31a666778148209fd6dcc HTTP/1.1" 200 None 2026-03-26 23:12:42,658 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:12:42,674 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b9626778148209fd337b HTTP/1.1" 200 None 2026-03-26 23:12:42,676 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:12:42,696 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c31a666778148209fd6dcc HTTP/1.1" 200 None 2026-03-26 23:12:42,697 - lava_callback - ERROR - Error processing callback for node 69c31a666778148209fd6dcc: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:58196 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57926 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34402 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55834 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55836 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55838 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48926 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34248 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40612 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37566 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40148 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46446 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46458 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46462 - "GET / HTTP/1.0" 200 OK 2026-03-26 23:14:22,677 - lava_callback - INFO - Received callback for node 69c31ab96778148209fd6dd8 INFO: 172.21.0.1:56440 - "POST /node/69c31ab96778148209fd6dd8 HTTP/1.0" 202 Accepted 2026-03-26 23:14:22,683 - lava_callback - INFO - Starting async job submit for node 69c31ab96778148209fd6dd8 2026-03-26 23:14:22,683 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:14:22,698 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c31ab96778148209fd6dd8 HTTP/1.1" 200 None 2026-03-26 23:14:22,698 - lava_callback - DEBUG - Processing node 69c31ab96778148209fd6dd8: baseline-arm64-baylibre 2026-03-26 23:14:22,699 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c31ab96778148209fd6dd8 2026-03-26 23:14:22,699 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:14:22,712 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c31ab96778148209fd6dd8 HTTP/1.1" 200 None 2026-03-26 23:14:22,714 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:14:22,727 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b9626778148209fd337b HTTP/1.1" 200 None 2026-03-26 23:14:22,729 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:14:22,750 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c31ab96778148209fd6dd8 HTTP/1.1" 200 None 2026-03-26 23:14:22,759 - lava_callback - ERROR - Error processing callback for node 69c31ab96778148209fd6dd8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:56444 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40532 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46446 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44192 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49316 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48818 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48824 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-26 23:15:16,439 - lava_callback - INFO - Received callback for node 69c5bdd3abf66a8f4d9bcfe2 INFO: 172.21.0.1:48830 - "POST /node/69c5bdd3abf66a8f4d9bcfe2 HTTP/1.0" 202 Accepted 2026-03-26 23:15:16,455 - lava_callback - INFO - Starting async job submit for node 69c5bdd3abf66a8f4d9bcfe2 2026-03-26 23:15:16,492 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:15:16,506 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5bdd3abf66a8f4d9bcfe2 HTTP/1.1" 200 None 2026-03-26 23:15:16,507 - lava_callback - DEBUG - Processing node 69c5bdd3abf66a8f4d9bcfe2: baseline-arm64-foundriesio INFO: 172.21.0.1:48836 - "GET / HTTP/1.0" 200 OK 2026-03-26 23:15:16,721 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 23:15:16,844 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 23:15:16,846 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69c5bdd3abf66a8f4d9bcfe2/log.txt.gz 2026-03-26 23:15:16,969 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 23:15:17,200 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 23:15:17,202 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69c5bdd3abf66a8f4d9bcfe2/lava-logs.yaml 2026-03-26 23:15:17,209 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 23:15:17,350 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 23:15:17,351 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69c5bdd3abf66a8f4d9bcfe2/lava_callback.json.gz 2026-03-26 23:15:17,373 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:15:17,388 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5bdd3abf66a8f4d9bcfe2 HTTP/1.1" 200 None 2026-03-26 23:15:17,389 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:15:17,406 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-03-26 23:15:17,407 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:15:17,446 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5bdd3abf66a8f4d9bcfe2 HTTP/1.1" 200 None 2026-03-26 23:15:17,452 - lava_callback - INFO - Completed processing callback for node 69c5bdd3abf66a8f4d9bcfe2 INFO: 172.21.0.1:46882 - "GET / HTTP/1.0" 200 OK 2026-03-26 23:15:33,466 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:15:33,482 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:44428 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39012 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56162 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43038 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45518 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45522 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-26 23:16:15,455 - lava_callback - INFO - Received callback for node 69c5bd66abf66a8f4d9bcfc8 INFO: 172.21.0.1:45526 - "POST /node/69c5bd66abf66a8f4d9bcfc8 HTTP/1.0" 202 Accepted 2026-03-26 23:16:15,461 - lava_callback - INFO - Starting async job submit for node 69c5bd66abf66a8f4d9bcfc8 2026-03-26 23:16:15,483 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:16:15,497 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5bd66abf66a8f4d9bcfc8 HTTP/1.1" 200 None 2026-03-26 23:16:15,498 - lava_callback - DEBUG - Processing node 69c5bd66abf66a8f4d9bcfc8: baseline-arm64 2026-03-26 23:16:15,766 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 23:16:16,063 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 23:16:16,065 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c5bd66abf66a8f4d9bcfc8/log.txt.gz 2026-03-26 23:16:16,208 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 23:16:16,385 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 23:16:16,387 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c5bd66abf66a8f4d9bcfc8/lava-logs.yaml 2026-03-26 23:16:16,393 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.21.0.1:45536 - "GET / HTTP/1.0" 200 OK 2026-03-26 23:16:16,515 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 23:16:16,517 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c5bd66abf66a8f4d9bcfc8/lava_callback.json.gz 2026-03-26 23:16:16,539 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:16:16,558 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5bd66abf66a8f4d9bcfc8 HTTP/1.1" 200 None 2026-03-26 23:16:16,560 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:16:16,574 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-03-26 23:16:16,576 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:16:16,601 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5bd66abf66a8f4d9bcfc8 HTTP/1.1" 200 None 2026-03-26 23:16:16,604 - lava_callback - INFO - Completed processing callback for node 69c5bd66abf66a8f4d9bcfc8 INFO: 172.21.0.1:48524 - "GET / HTTP/1.0" 200 OK 2026-03-26 23:16:33,485 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:16:33,501 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:36610 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59602 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58354 - "GET / HTTP/1.0" 200 OK 2026-03-26 23:17:02,849 - lava_callback - INFO - Received callback for node 69c31b5a6778148209fd6de8 INFO: 172.21.0.1:53652 - "POST /node/69c31b5a6778148209fd6de8 HTTP/1.0" 202 Accepted 2026-03-26 23:17:02,855 - lava_callback - INFO - Starting async job submit for node 69c31b5a6778148209fd6de8 2026-03-26 23:17:02,856 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:17:02,870 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c31b5a6778148209fd6de8 HTTP/1.1" 200 None 2026-03-26 23:17:02,871 - lava_callback - DEBUG - Processing node 69c31b5a6778148209fd6de8: baseline-arm64-baylibre 2026-03-26 23:17:02,871 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c31b5a6778148209fd6de8 2026-03-26 23:17:02,872 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:17:02,885 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c31b5a6778148209fd6de8 HTTP/1.1" 200 None 2026-03-26 23:17:02,886 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:17:02,898 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b9626778148209fd337b HTTP/1.1" 200 None 2026-03-26 23:17:02,900 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:17:02,923 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c31b5a6778148209fd6de8 HTTP/1.1" 200 None 2026-03-26 23:17:02,924 - lava_callback - ERROR - Error processing callback for node 69c31b5a6778148209fd6de8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:53668 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37128 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37132 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37142 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49218 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56004 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47962 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56952 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47192 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43986 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44002 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44012 - "GET / HTTP/1.0" 200 OK 2026-03-26 23:18:22,885 - lava_callback - INFO - Received callback for node 69c31bae6778148209fd6df8 INFO: 172.21.0.1:52588 - "POST /node/69c31bae6778148209fd6df8 HTTP/1.0" 202 Accepted 2026-03-26 23:18:22,890 - lava_callback - INFO - Starting async job submit for node 69c31bae6778148209fd6df8 2026-03-26 23:18:22,891 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:18:22,909 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c31bae6778148209fd6df8 HTTP/1.1" 200 None 2026-03-26 23:18:22,910 - lava_callback - DEBUG - Processing node 69c31bae6778148209fd6df8: baseline-arm64-baylibre 2026-03-26 23:18:22,911 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c31bae6778148209fd6df8 2026-03-26 23:18:22,911 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:18:22,924 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c31bae6778148209fd6df8 HTTP/1.1" 200 None 2026-03-26 23:18:22,925 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:18:22,938 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b9626778148209fd337b HTTP/1.1" 200 None 2026-03-26 23:18:22,940 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:18:22,960 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c31bae6778148209fd6df8 HTTP/1.1" 200 None 2026-03-26 23:18:22,967 - lava_callback - ERROR - Error processing callback for node 69c31bae6778148209fd6df8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:52594 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43494 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60290 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40016 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49846 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44538 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44552 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44558 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58612 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53382 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57412 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34858 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60602 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34086 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34088 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34096 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42402 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50156 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42796 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46282 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41222 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43040 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43052 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43068 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59044 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49764 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59658 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41376 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36658 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46526 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46530 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46546 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40702 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32884 - "GET / HTTP/1.0" 200 OK 2026-03-26 23:22:37,064 - lava_callback - INFO - Received callback for node 69c31cac6778148209fd6e1e INFO: 172.21.0.1:32898 - "POST /node/69c31cac6778148209fd6e1e HTTP/1.0" 202 Accepted 2026-03-26 23:22:37,069 - lava_callback - INFO - Starting async job submit for node 69c31cac6778148209fd6e1e 2026-03-26 23:22:37,070 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:22:37,084 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c31cac6778148209fd6e1e HTTP/1.1" 200 None 2026-03-26 23:22:37,085 - lava_callback - DEBUG - Processing node 69c31cac6778148209fd6e1e: baseline-arm64-clabbe 2026-03-26 23:22:37,085 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c31cac6778148209fd6e1e 2026-03-26 23:22:37,086 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:22:37,099 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c31cac6778148209fd6e1e HTTP/1.1" 200 None 2026-03-26 23:22:37,101 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:22:37,114 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b9626778148209fd337b HTTP/1.1" 200 None 2026-03-26 23:22:37,115 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:22:37,147 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c31cac6778148209fd6e1e HTTP/1.1" 200 None 2026-03-26 23:22:37,156 - lava_callback - ERROR - Error processing callback for node 69c31cac6778148209fd6e1e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:45648 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50538 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52954 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53706 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53722 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53732 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57534 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58570 - "GET / HTTP/1.0" 200 OK 2026-03-26 23:23:37,097 - lava_callback - INFO - Received callback for node 69c31cee6778148209fd6e28 INFO: 172.21.0.1:58584 - "POST /node/69c31cee6778148209fd6e28 HTTP/1.0" 202 Accepted 2026-03-26 23:23:37,103 - lava_callback - INFO - Starting async job submit for node 69c31cee6778148209fd6e28 2026-03-26 23:23:37,104 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:23:37,118 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c31cee6778148209fd6e28 HTTP/1.1" 200 None 2026-03-26 23:23:37,119 - lava_callback - DEBUG - Processing node 69c31cee6778148209fd6e28: baseline-arm-clabbe 2026-03-26 23:23:37,119 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c31cee6778148209fd6e28 2026-03-26 23:23:37,120 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:23:37,133 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c31cee6778148209fd6e28 HTTP/1.1" 200 None 2026-03-26 23:23:37,134 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:23:37,158 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01633c3d73288b0d052f0 HTTP/1.1" 200 None 2026-03-26 23:23:37,160 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:23:37,188 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c31cee6778148209fd6e28 HTTP/1.1" 200 None 2026-03-26 23:23:37,189 - lava_callback - ERROR - Error processing callback for node 69c31cee6778148209fd6e28: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:44606 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36116 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51234 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60284 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60288 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60292 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50832 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60872 - "GET / HTTP/1.0" 200 OK 2026-03-26 23:24:37,109 - lava_callback - INFO - Received callback for node 69c31d216778148209fd6e38 INFO: 172.21.0.1:60878 - "POST /node/69c31d216778148209fd6e38 HTTP/1.0" 202 Accepted 2026-03-26 23:24:37,115 - lava_callback - INFO - Starting async job submit for node 69c31d216778148209fd6e38 2026-03-26 23:24:37,115 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:24:37,129 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c31d216778148209fd6e38 HTTP/1.1" 200 None 2026-03-26 23:24:37,130 - lava_callback - DEBUG - Processing node 69c31d216778148209fd6e38: baseline-arm64-clabbe 2026-03-26 23:24:37,131 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c31d216778148209fd6e38 2026-03-26 23:24:37,131 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:24:37,144 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c31d216778148209fd6e38 HTTP/1.1" 200 None 2026-03-26 23:24:37,145 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:24:37,159 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b9626778148209fd337b HTTP/1.1" 200 None 2026-03-26 23:24:37,160 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:24:37,194 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c31d216778148209fd6e38 HTTP/1.1" 200 None 2026-03-26 23:24:37,195 - lava_callback - ERROR - Error processing callback for node 69c31d216778148209fd6e38: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:35862 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52748 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34386 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44664 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44674 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44686 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33412 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59552 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49012 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49656 - "GET / HTTP/1.0" 200 OK 2026-03-26 23:25:57,170 - lava_callback - INFO - Received callback for node 69c31d736778148209fd6e45 INFO: 172.21.0.1:49664 - "POST /node/69c31d736778148209fd6e45 HTTP/1.0" 202 Accepted 2026-03-26 23:25:57,175 - lava_callback - INFO - Starting async job submit for node 69c31d736778148209fd6e45 2026-03-26 23:25:57,176 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:25:57,191 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c31d736778148209fd6e45 HTTP/1.1" 200 None 2026-03-26 23:25:57,192 - lava_callback - DEBUG - Processing node 69c31d736778148209fd6e45: baseline-arm64-clabbe 2026-03-26 23:25:57,192 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c31d736778148209fd6e45 2026-03-26 23:25:57,193 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:25:57,208 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c31d736778148209fd6e45 HTTP/1.1" 200 None 2026-03-26 23:25:57,210 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:25:57,229 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b9626778148209fd337b HTTP/1.1" 200 None 2026-03-26 23:25:57,231 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:25:57,253 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c31d736778148209fd6e45 HTTP/1.1" 200 None 2026-03-26 23:25:57,264 - lava_callback - ERROR - Error processing callback for node 69c31d736778148209fd6e45: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:59480 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41402 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41414 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41416 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50626 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53216 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48964 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39938 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35702 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37176 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37180 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37196 - "GET / HTTP/1.0" 200 OK 2026-03-26 23:27:17,259 - lava_callback - INFO - Received callback for node 69c31dc36778148209fd6e54 INFO: 172.21.0.1:37204 - "POST /node/69c31dc36778148209fd6e54 HTTP/1.0" 202 Accepted 2026-03-26 23:27:17,264 - lava_callback - INFO - Starting async job submit for node 69c31dc36778148209fd6e54 2026-03-26 23:27:17,265 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:27:17,279 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c31dc36778148209fd6e54 HTTP/1.1" 200 None 2026-03-26 23:27:17,280 - lava_callback - DEBUG - Processing node 69c31dc36778148209fd6e54: baseline-arm64-clabbe 2026-03-26 23:27:17,281 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c31dc36778148209fd6e54 2026-03-26 23:27:17,281 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:27:17,295 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c31dc36778148209fd6e54 HTTP/1.1" 200 None 2026-03-26 23:27:17,296 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:27:17,309 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b9626778148209fd337b HTTP/1.1" 200 None 2026-03-26 23:27:17,311 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:27:17,332 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c31dc36778148209fd6e54 HTTP/1.1" 200 None 2026-03-26 23:27:17,333 - lava_callback - ERROR - Error processing callback for node 69c31dc36778148209fd6e54: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:41106 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46270 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52200 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58060 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43990 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55218 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55234 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55248 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55256 - "GET / HTTP/1.0" 200 OK 2026-03-26 23:28:24,422 - lava_callback - INFO - Received callback for node 69c5bfa5abf66a8f4d9bd071 INFO: 172.21.0.1:52240 - "POST /node/69c5bfa5abf66a8f4d9bd071 HTTP/1.0" 202 Accepted 2026-03-26 23:28:24,430 - lava_callback - INFO - Starting async job submit for node 69c5bfa5abf66a8f4d9bd071 2026-03-26 23:28:24,483 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:28:24,498 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5bfa5abf66a8f4d9bd071 HTTP/1.1" 200 None 2026-03-26 23:28:24,498 - lava_callback - DEBUG - Processing node 69c5bfa5abf66a8f4d9bd071: baseline-arm64-foundriesio 2026-03-26 23:28:25,662 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 23:28:25,882 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 23:28:25,884 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69c5bfa5abf66a8f4d9bd071/log.txt.gz INFO: 172.21.0.1:52248 - "GET / HTTP/1.0" 200 OK 2026-03-26 23:28:26,632 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 23:28:26,927 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 23:28:26,930 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69c5bfa5abf66a8f4d9bd071/lava-logs.yaml 2026-03-26 23:28:26,957 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 23:28:27,168 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 23:28:27,170 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69c5bfa5abf66a8f4d9bd071/lava_callback.json.gz DEBUG: Setup failed for 69c5bfa5abf66a8f4d9bd071. Transitting job node result: incomplete -> fail 2026-03-26 23:28:27,276 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:28:27,290 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5bfa5abf66a8f4d9bd071 HTTP/1.1" 200 None 2026-03-26 23:28:27,292 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:28:27,309 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-03-26 23:28:27,311 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:28:27,336 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5bfa5abf66a8f4d9bd071 HTTP/1.1" 200 None 2026-03-26 23:28:27,456 - lava_callback - INFO - Completed processing callback for node 69c5bfa5abf66a8f4d9bd071 2026-03-26 23:28:33,504 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:28:33,520 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:44784 - "GET / HTTP/1.0" 200 OK 2026-03-26 23:28:37,311 - lava_callback - INFO - Received callback for node 69c31e136778148209fd6e5f INFO: 172.21.0.1:44786 - "POST /node/69c31e136778148209fd6e5f HTTP/1.0" 202 Accepted 2026-03-26 23:28:37,317 - lava_callback - INFO - Starting async job submit for node 69c31e136778148209fd6e5f 2026-03-26 23:28:37,317 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:28:37,332 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c31e136778148209fd6e5f HTTP/1.1" 200 None 2026-03-26 23:28:37,332 - lava_callback - DEBUG - Processing node 69c31e136778148209fd6e5f: baseline-arm64-clabbe 2026-03-26 23:28:37,333 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c31e136778148209fd6e5f 2026-03-26 23:28:37,333 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:28:37,357 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c31e136778148209fd6e5f HTTP/1.1" 200 None 2026-03-26 23:28:37,358 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:28:37,370 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b9626778148209fd337b HTTP/1.1" 200 None 2026-03-26 23:28:37,372 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:28:37,395 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c31e136778148209fd6e5f HTTP/1.1" 200 None 2026-03-26 23:28:37,400 - lava_callback - ERROR - Error processing callback for node 69c31e136778148209fd6e5f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:39348 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58504 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54740 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37878 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37890 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37904 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36034 - "GET / HTTP/1.0" 200 OK 2026-03-26 23:29:30,371 - lava_callback - INFO - Received callback for node 69c5c00babf66a8f4d9bd0a6 INFO: 172.21.0.1:44240 - "POST /node/69c5c00babf66a8f4d9bd0a6 HTTP/1.0" 202 Accepted 2026-03-26 23:29:30,378 - lava_callback - INFO - Starting async job submit for node 69c5c00babf66a8f4d9bd0a6 2026-03-26 23:29:30,412 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:29:30,428 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5c00babf66a8f4d9bd0a6 HTTP/1.1" 200 None 2026-03-26 23:29:30,428 - lava_callback - DEBUG - Processing node 69c5c00babf66a8f4d9bd0a6: baseline-arm64 2026-03-26 23:29:30,807 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 23:29:30,992 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 23:29:30,994 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c5c00babf66a8f4d9bd0a6/log.txt.gz 2026-03-26 23:29:31,225 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 23:29:31,451 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 23:29:31,453 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c5c00babf66a8f4d9bd0a6/lava-logs.yaml 2026-03-26 23:29:31,462 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 23:29:31,667 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 23:29:31,669 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c5c00babf66a8f4d9bd0a6/lava_callback.json.gz DEBUG: Setup failed for 69c5c00babf66a8f4d9bd0a6. Transitting job node result: incomplete -> fail 2026-03-26 23:29:31,744 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:29:31,758 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5c00babf66a8f4d9bd0a6 HTTP/1.1" 200 None 2026-03-26 23:29:31,760 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:29:31,773 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-03-26 23:29:31,774 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:29:31,797 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5c00babf66a8f4d9bd0a6 HTTP/1.1" 200 None 2026-03-26 23:29:31,881 - lava_callback - INFO - Completed processing callback for node 69c5c00babf66a8f4d9bd0a6 2026-03-26 23:29:33,523 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:29:33,550 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:44254 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46830 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60602 - "GET / HTTP/1.0" 200 OK 2026-03-26 23:29:57,448 - lava_callback - INFO - Received callback for node 69c31e646778148209fd6e6b INFO: 172.21.0.1:60614 - "POST /node/69c31e646778148209fd6e6b HTTP/1.0" 202 Accepted 2026-03-26 23:29:57,454 - lava_callback - INFO - Starting async job submit for node 69c31e646778148209fd6e6b 2026-03-26 23:29:57,455 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:29:57,469 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c31e646778148209fd6e6b HTTP/1.1" 200 None 2026-03-26 23:29:57,470 - lava_callback - DEBUG - Processing node 69c31e646778148209fd6e6b: baseline-arm64-clabbe 2026-03-26 23:29:57,470 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c31e646778148209fd6e6b 2026-03-26 23:29:57,471 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:29:57,484 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c31e646778148209fd6e6b HTTP/1.1" 200 None 2026-03-26 23:29:57,485 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:29:57,498 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b9626778148209fd337b HTTP/1.1" 200 None 2026-03-26 23:29:57,500 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:29:57,519 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c31e646778148209fd6e6b HTTP/1.1" 200 None 2026-03-26 23:29:57,528 - lava_callback - ERROR - Error processing callback for node 69c31e646778148209fd6e6b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:54954 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46446 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46458 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46468 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40910 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39384 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46926 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35042 - "GET / HTTP/1.0" 200 OK 2026-03-26 23:30:57,404 - lava_callback - INFO - Received callback for node 69c31ea76778148209fd6e74 INFO: 172.21.0.1:35052 - "POST /node/69c31ea76778148209fd6e74 HTTP/1.0" 202 Accepted 2026-03-26 23:30:57,409 - lava_callback - INFO - Starting async job submit for node 69c31ea76778148209fd6e74 2026-03-26 23:30:57,410 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:30:57,459 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c31ea76778148209fd6e74 HTTP/1.1" 200 None 2026-03-26 23:30:57,460 - lava_callback - DEBUG - Processing node 69c31ea76778148209fd6e74: baseline-arm-clabbe 2026-03-26 23:30:57,460 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c31ea76778148209fd6e74 2026-03-26 23:30:57,461 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:30:57,474 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c31ea76778148209fd6e74 HTTP/1.1" 200 None 2026-03-26 23:30:57,475 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:30:57,488 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01633c3d73288b0d052f0 HTTP/1.1" 200 None 2026-03-26 23:30:57,489 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:30:57,509 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c31ea76778148209fd6e74 HTTP/1.1" 200 None 2026-03-26 23:30:57,510 - lava_callback - ERROR - Error processing callback for node 69c31ea76778148209fd6e74: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:38876 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54098 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54110 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54114 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50462 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43880 - "GET / HTTP/1.0" 200 OK 2026-03-26 23:31:37,405 - lava_callback - INFO - Received callback for node 69c31ece6778148209fd6e80 INFO: 172.21.0.1:43896 - "POST /node/69c31ece6778148209fd6e80 HTTP/1.0" 202 Accepted 2026-03-26 23:31:37,410 - lava_callback - INFO - Starting async job submit for node 69c31ece6778148209fd6e80 2026-03-26 23:31:37,411 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:31:37,426 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c31ece6778148209fd6e80 HTTP/1.1" 200 None 2026-03-26 23:31:37,426 - lava_callback - DEBUG - Processing node 69c31ece6778148209fd6e80: baseline-arm-clabbe 2026-03-26 23:31:37,427 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c31ece6778148209fd6e80 2026-03-26 23:31:37,427 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:31:37,440 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c31ece6778148209fd6e80 HTTP/1.1" 200 None 2026-03-26 23:31:37,441 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:31:37,454 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01633c3d73288b0d052f0 HTTP/1.1" 200 None 2026-03-26 23:31:37,456 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:31:37,475 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c31ece6778148209fd6e80 HTTP/1.1" 200 None 2026-03-26 23:31:37,477 - lava_callback - ERROR - Error processing callback for node 69c31ece6778148209fd6e80: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:35432 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33300 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53454 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47928 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47932 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47948 - "GET / HTTP/1.0" 200 OK 2026-03-26 23:32:23,607 - lava_callback - INFO - Received callback for node 69c31efd6778148209fd6e95 INFO: 172.21.0.1:56054 - "POST /node/69c31efd6778148209fd6e95 HTTP/1.0" 202 Accepted 2026-03-26 23:32:23,613 - lava_callback - INFO - Starting async job submit for node 69c31efd6778148209fd6e95 2026-03-26 23:32:23,613 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:32:23,628 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c31efd6778148209fd6e95 HTTP/1.1" 200 None 2026-03-26 23:32:23,629 - lava_callback - DEBUG - Processing node 69c31efd6778148209fd6e95: baseline-arm64-baylibre 2026-03-26 23:32:23,629 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c31efd6778148209fd6e95 2026-03-26 23:32:23,629 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:32:23,646 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c31efd6778148209fd6e95 HTTP/1.1" 200 None 2026-03-26 23:32:23,647 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:32:23,662 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd73420e93e2314723e2c4 HTTP/1.1" 200 None 2026-03-26 23:32:23,663 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:32:23,684 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c31efd6778148209fd6e95 HTTP/1.1" 200 None 2026-03-26 23:32:23,694 - lava_callback - ERROR - Error processing callback for node 69c31efd6778148209fd6e95: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:56060 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35810 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47972 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55214 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36112 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51908 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51922 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51926 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57834 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58484 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58162 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50726 - "GET / HTTP/1.0" 200 OK 2026-03-26 23:33:57,522 - lava_callback - INFO - Received callback for node 69c31f526778148209fd6ea3 INFO: 172.21.0.1:50730 - "POST /node/69c31f526778148209fd6ea3 HTTP/1.0" 202 Accepted 2026-03-26 23:33:57,528 - lava_callback - INFO - Starting async job submit for node 69c31f526778148209fd6ea3 2026-03-26 23:33:57,529 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:33:57,543 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c31f526778148209fd6ea3 HTTP/1.1" 200 None 2026-03-26 23:33:57,544 - lava_callback - DEBUG - Processing node 69c31f526778148209fd6ea3: baseline-arm64-clabbe 2026-03-26 23:33:57,544 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c31f526778148209fd6ea3 2026-03-26 23:33:57,545 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:33:57,558 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c31f526778148209fd6ea3 HTTP/1.1" 200 None 2026-03-26 23:33:57,559 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:33:57,581 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd73420e93e2314723e2c4 HTTP/1.1" 200 None 2026-03-26 23:33:57,582 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:33:57,603 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c31f526778148209fd6ea3 HTTP/1.1" 200 None 2026-03-26 23:33:57,609 - lava_callback - ERROR - Error processing callback for node 69c31f526778148209fd6ea3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-26 23:34:05,730 - lava_callback - INFO - Received callback for node 69c5c25cabf66a8f4d9bd18a INFO: 172.21.0.1:59300 - "POST /node/69c5c25cabf66a8f4d9bd18a HTTP/1.0" 202 Accepted 2026-03-26 23:34:05,736 - lava_callback - INFO - Starting async job submit for node 69c5c25cabf66a8f4d9bd18a 2026-03-26 23:34:05,738 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:34:05,756 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5c25cabf66a8f4d9bd18a HTTP/1.1" 200 None 2026-03-26 23:34:05,757 - lava_callback - DEBUG - Processing node 69c5c25cabf66a8f4d9bd18a: baseline-nfs-arm64 2026-03-26 23:34:05,758 - lava_callback - WARNING - Failed to upload log 2026-03-26 23:34:05,760 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 23:34:05,925 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 23:34:05,927 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c5c25cabf66a8f4d9bd18a/lava-logs.yaml 2026-03-26 23:34:05,928 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 23:34:06,068 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 23:34:06,069 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c5c25cabf66a8f4d9bd18a/lava_callback.json.gz 2026-03-26 23:34:06,071 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:34:06,088 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5c25cabf66a8f4d9bd18a HTTP/1.1" 200 None 2026-03-26 23:34:06,090 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:34:06,116 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-03-26 23:34:06,118 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:34:06,139 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5c25cabf66a8f4d9bd18a HTTP/1.1" 200 None 2026-03-26 23:34:06,142 - lava_callback - INFO - Completed processing callback for node 69c5c25cabf66a8f4d9bd18a INFO: 172.21.0.1:59306 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59044 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59048 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59050 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55790 - "GET / HTTP/1.0" 200 OK 2026-03-26 23:34:33,553 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:34:33,590 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:36646 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54858 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49454 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33812 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57536 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57552 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57556 - "GET / HTTP/1.0" 200 OK 2026-03-26 23:35:17,565 - lava_callback - INFO - Received callback for node 69c31fa36778148209fd6eb1 INFO: 172.21.0.1:57564 - "POST /node/69c31fa36778148209fd6eb1 HTTP/1.0" 202 Accepted 2026-03-26 23:35:17,570 - lava_callback - INFO - Starting async job submit for node 69c31fa36778148209fd6eb1 2026-03-26 23:35:17,571 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:35:17,585 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c31fa36778148209fd6eb1 HTTP/1.1" 200 None 2026-03-26 23:35:17,586 - lava_callback - DEBUG - Processing node 69c31fa36778148209fd6eb1: baseline-arm64-clabbe 2026-03-26 23:35:17,586 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c31fa36778148209fd6eb1 2026-03-26 23:35:17,587 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:35:17,611 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c31fa36778148209fd6eb1 HTTP/1.1" 200 None 2026-03-26 23:35:17,613 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:35:17,626 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd73420e93e2314723e2c4 HTTP/1.1" 200 None 2026-03-26 23:35:17,628 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:35:17,647 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c31fa36778148209fd6eb1 HTTP/1.1" 200 None 2026-03-26 23:35:17,649 - lava_callback - ERROR - Error processing callback for node 69c31fa36778148209fd6eb1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:54486 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55568 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39760 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36744 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39784 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48698 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48710 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48722 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47646 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41612 - "GET / HTTP/1.0" 200 OK 2026-03-26 23:36:37,621 - lava_callback - INFO - Received callback for node 69c31ff36778148209fd6ebb INFO: 172.21.0.1:41620 - "POST /node/69c31ff36778148209fd6ebb HTTP/1.0" 202 Accepted 2026-03-26 23:36:37,626 - lava_callback - INFO - Starting async job submit for node 69c31ff36778148209fd6ebb 2026-03-26 23:36:37,628 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:36:37,643 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c31ff36778148209fd6ebb HTTP/1.1" 200 None 2026-03-26 23:36:37,644 - lava_callback - DEBUG - Processing node 69c31ff36778148209fd6ebb: baseline-arm64-clabbe 2026-03-26 23:36:37,644 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c31ff36778148209fd6ebb 2026-03-26 23:36:37,645 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:36:37,658 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c31ff36778148209fd6ebb HTTP/1.1" 200 None 2026-03-26 23:36:37,660 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:36:37,672 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd73420e93e2314723e2c4 HTTP/1.1" 200 None 2026-03-26 23:36:37,674 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:36:37,694 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c31ff36778148209fd6ebb HTTP/1.1" 200 None 2026-03-26 23:36:37,700 - lava_callback - ERROR - Error processing callback for node 69c31ff36778148209fd6ebb: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:37566 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41054 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50744 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53386 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53398 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53406 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49628 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59494 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49416 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48454 - "GET / HTTP/1.0" 200 OK 2026-03-26 23:37:57,691 - lava_callback - INFO - Received callback for node 69c320426778148209fd6ec6 INFO: 172.21.0.1:48470 - "POST /node/69c320426778148209fd6ec6 HTTP/1.0" 202 Accepted 2026-03-26 23:37:57,696 - lava_callback - INFO - Starting async job submit for node 69c320426778148209fd6ec6 2026-03-26 23:37:57,697 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:37:57,716 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c320426778148209fd6ec6 HTTP/1.1" 200 None 2026-03-26 23:37:57,717 - lava_callback - DEBUG - Processing node 69c320426778148209fd6ec6: baseline-arm64-clabbe 2026-03-26 23:37:57,718 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c320426778148209fd6ec6 2026-03-26 23:37:57,718 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:37:57,739 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c320426778148209fd6ec6 HTTP/1.1" 200 None 2026-03-26 23:37:57,741 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:37:57,754 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd73420e93e2314723e2c4 HTTP/1.1" 200 None 2026-03-26 23:37:57,756 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:37:57,780 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c320426778148209fd6ec6 HTTP/1.1" 200 None 2026-03-26 23:37:57,781 - lava_callback - ERROR - Error processing callback for node 69c320426778148209fd6ec6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:42036 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42912 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42918 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42922 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36732 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50304 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37792 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37784 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42896 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53004 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53018 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53022 - "GET / HTTP/1.0" 200 OK 2026-03-26 23:39:17,771 - lava_callback - INFO - Received callback for node 69c320906778148209fd6ecc INFO: 172.21.0.1:53028 - "POST /node/69c320906778148209fd6ecc HTTP/1.0" 202 Accepted 2026-03-26 23:39:17,777 - lava_callback - INFO - Starting async job submit for node 69c320906778148209fd6ecc 2026-03-26 23:39:17,778 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:39:17,792 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c320906778148209fd6ecc HTTP/1.1" 200 None 2026-03-26 23:39:17,793 - lava_callback - DEBUG - Processing node 69c320906778148209fd6ecc: baseline-arm64-clabbe 2026-03-26 23:39:17,793 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c320906778148209fd6ecc 2026-03-26 23:39:17,794 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:39:17,815 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c320906778148209fd6ecc HTTP/1.1" 200 None 2026-03-26 23:39:17,817 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:39:17,831 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd73420e93e2314723e2c4 HTTP/1.1" 200 None 2026-03-26 23:39:17,833 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:39:17,855 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c320906778148209fd6ecc HTTP/1.1" 200 None 2026-03-26 23:39:17,861 - lava_callback - ERROR - Error processing callback for node 69c320906778148209fd6ecc: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:48058 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60424 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44662 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37808 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34444 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46388 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46402 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46408 - "GET / HTTP/1.0" 200 OK 2026-03-26 23:40:24,050 - lava_callback - INFO - Received callback for node 69c320d96778148209fd6ed7 INFO: 172.21.0.1:36826 - "POST /node/69c320d96778148209fd6ed7 HTTP/1.0" 202 Accepted 2026-03-26 23:40:24,056 - lava_callback - INFO - Starting async job submit for node 69c320d96778148209fd6ed7 2026-03-26 23:40:24,056 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:40:24,071 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c320d96778148209fd6ed7 HTTP/1.1" 200 None 2026-03-26 23:40:24,072 - lava_callback - DEBUG - Processing node 69c320d96778148209fd6ed7: baseline-arm64-baylibre 2026-03-26 23:40:24,072 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c320d96778148209fd6ed7 2026-03-26 23:40:24,073 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:40:24,086 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c320d96778148209fd6ed7 HTTP/1.1" 200 None 2026-03-26 23:40:24,087 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:40:24,101 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd73420e93e2314723e2c4 HTTP/1.1" 200 None 2026-03-26 23:40:24,102 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:40:24,124 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c320d96778148209fd6ed7 HTTP/1.1" 200 None 2026-03-26 23:40:24,126 - lava_callback - ERROR - Error processing callback for node 69c320d96778148209fd6ed7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:36832 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41188 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39000 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33078 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51392 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39222 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39228 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39236 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52466 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54446 - "GET / HTTP/1.0" 200 OK 2026-03-26 23:41:37,822 - lava_callback - INFO - Received callback for node 69c3212d6778148209fd6ee2 INFO: 172.21.0.1:54458 - "POST /node/69c3212d6778148209fd6ee2 HTTP/1.0" 202 Accepted 2026-03-26 23:41:37,828 - lava_callback - INFO - Starting async job submit for node 69c3212d6778148209fd6ee2 2026-03-26 23:41:37,829 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:41:37,851 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3212d6778148209fd6ee2 HTTP/1.1" 200 None 2026-03-26 23:41:37,852 - lava_callback - DEBUG - Processing node 69c3212d6778148209fd6ee2: baseline-arm64-clabbe 2026-03-26 23:41:37,852 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3212d6778148209fd6ee2 2026-03-26 23:41:37,852 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:41:37,866 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3212d6778148209fd6ee2 HTTP/1.1" 200 None 2026-03-26 23:41:37,867 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:41:37,881 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd73420e93e2314723e2c4 HTTP/1.1" 200 None 2026-03-26 23:41:37,882 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:41:37,902 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3212d6778148209fd6ee2 HTTP/1.1" 200 None 2026-03-26 23:41:37,904 - lava_callback - ERROR - Error processing callback for node 69c3212d6778148209fd6ee2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-26 23:41:45,670 - lava_callback - INFO - Received callback for node 69c5c3f6abf66a8f4d9bd1f4 INFO: 172.21.0.1:53380 - "POST /node/69c5c3f6abf66a8f4d9bd1f4 HTTP/1.0" 202 Accepted 2026-03-26 23:41:45,677 - lava_callback - INFO - Starting async job submit for node 69c5c3f6abf66a8f4d9bd1f4 2026-03-26 23:41:45,706 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:41:45,723 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5c3f6abf66a8f4d9bd1f4 HTTP/1.1" 200 None 2026-03-26 23:41:45,724 - lava_callback - DEBUG - Processing node 69c5c3f6abf66a8f4d9bd1f4: baseline-arm64 2026-03-26 23:41:45,862 - lava_callback - WARNING - Failed to upload log 2026-03-26 23:41:45,934 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 23:41:46,122 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 23:41:46,124 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c5c3f6abf66a8f4d9bd1f4/lava-logs.yaml 2026-03-26 23:41:46,128 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 23:41:46,275 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 23:41:46,277 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c5c3f6abf66a8f4d9bd1f4/lava_callback.json.gz 2026-03-26 23:41:46,308 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:41:46,322 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5c3f6abf66a8f4d9bd1f4 HTTP/1.1" 200 None 2026-03-26 23:41:46,324 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:41:46,338 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-03-26 23:41:46,339 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:41:46,360 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5c3f6abf66a8f4d9bd1f4 HTTP/1.1" 200 None 2026-03-26 23:41:46,429 - lava_callback - INFO - Completed processing callback for node 69c5c3f6abf66a8f4d9bd1f4 INFO: 172.21.0.1:53386 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43634 - "GET / HTTP/1.0" 200 OK 2026-03-26 23:42:03,593 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:42:03,611 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:45780 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43560 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43576 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43588 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33672 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54598 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44798 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53066 - "GET / HTTP/1.0" 200 OK 2026-03-26 23:42:57,901 - lava_callback - INFO - Received callback for node 69c3217d6778148209fd6ee7 INFO: 172.21.0.1:53074 - "POST /node/69c3217d6778148209fd6ee7 HTTP/1.0" 202 Accepted 2026-03-26 23:42:57,910 - lava_callback - INFO - Starting async job submit for node 69c3217d6778148209fd6ee7 2026-03-26 23:42:57,911 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:42:57,926 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3217d6778148209fd6ee7 HTTP/1.1" 200 None 2026-03-26 23:42:57,927 - lava_callback - DEBUG - Processing node 69c3217d6778148209fd6ee7: baseline-arm64-clabbe 2026-03-26 23:42:57,927 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3217d6778148209fd6ee7 2026-03-26 23:42:57,928 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:42:57,941 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3217d6778148209fd6ee7 HTTP/1.1" 200 None 2026-03-26 23:42:57,942 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:42:57,959 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd73420e93e2314723e2c4 HTTP/1.1" 200 None 2026-03-26 23:42:57,960 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:42:57,982 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3217d6778148209fd6ee7 HTTP/1.1" 200 None 2026-03-26 23:42:57,983 - lava_callback - ERROR - Error processing callback for node 69c3217d6778148209fd6ee7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:60912 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47800 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47802 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47814 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52022 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50352 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53814 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59534 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49630 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52698 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52708 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52712 - "GET / HTTP/1.0" 200 OK 2026-03-26 23:44:24,307 - lava_callback - INFO - Received callback for node 69c321c56778148209fd6ef6 INFO: 172.21.0.1:53876 - "POST /node/69c321c56778148209fd6ef6 HTTP/1.0" 202 Accepted 2026-03-26 23:44:24,313 - lava_callback - INFO - Starting async job submit for node 69c321c56778148209fd6ef6 2026-03-26 23:44:24,313 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:44:24,329 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c321c56778148209fd6ef6 HTTP/1.1" 200 None 2026-03-26 23:44:24,330 - lava_callback - DEBUG - Processing node 69c321c56778148209fd6ef6: baseline-arm64-baylibre 2026-03-26 23:44:24,330 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c321c56778148209fd6ef6 2026-03-26 23:44:24,331 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:44:24,344 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c321c56778148209fd6ef6 HTTP/1.1" 200 None 2026-03-26 23:44:24,345 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:44:24,366 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd73420e93e2314723e2c4 HTTP/1.1" 200 None 2026-03-26 23:44:24,368 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:44:24,412 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c321c56778148209fd6ef6 HTTP/1.1" 200 None 2026-03-26 23:44:24,416 - lava_callback - ERROR - Error processing callback for node 69c321c56778148209fd6ef6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:53886 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45150 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41080 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49914 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45574 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41560 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41574 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41582 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41158 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47992 - "GET / HTTP/1.0" 200 OK 2026-03-26 23:45:37,992 - lava_callback - INFO - Received callback for node 69c3220c6778148209fd6f05 INFO: 172.21.0.1:48006 - "POST /node/69c3220c6778148209fd6f05 HTTP/1.0" 202 Accepted 2026-03-26 23:45:37,998 - lava_callback - INFO - Starting async job submit for node 69c3220c6778148209fd6f05 2026-03-26 23:45:37,999 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:45:38,014 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3220c6778148209fd6f05 HTTP/1.1" 200 None 2026-03-26 23:45:38,015 - lava_callback - DEBUG - Processing node 69c3220c6778148209fd6f05: baseline-arm-clabbe 2026-03-26 23:45:38,015 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3220c6778148209fd6f05 2026-03-26 23:45:38,016 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:45:38,029 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3220c6778148209fd6f05 HTTP/1.1" 200 None 2026-03-26 23:45:38,030 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:45:38,043 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd732d0e93e2314723e2b5 HTTP/1.1" 200 None 2026-03-26 23:45:38,044 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:45:38,067 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3220c6778148209fd6f05 HTTP/1.1" 200 None 2026-03-26 23:45:38,070 - lava_callback - ERROR - Error processing callback for node 69c3220c6778148209fd6f05: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:45046 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41348 - "GET / HTTP/1.0" 200 OK 2026-03-26 23:45:58,010 - lava_callback - INFO - Received callback for node 69c322306778148209fd6f15 INFO: 172.21.0.1:41352 - "POST /node/69c322306778148209fd6f15 HTTP/1.0" 202 Accepted 2026-03-26 23:45:58,015 - lava_callback - INFO - Starting async job submit for node 69c322306778148209fd6f15 2026-03-26 23:45:58,016 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:45:58,030 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c322306778148209fd6f15 HTTP/1.1" 200 None 2026-03-26 23:45:58,031 - lava_callback - DEBUG - Processing node 69c322306778148209fd6f15: baseline-arm-clabbe 2026-03-26 23:45:58,031 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c322306778148209fd6f15 2026-03-26 23:45:58,032 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:45:58,045 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c322306778148209fd6f15 HTTP/1.1" 200 None 2026-03-26 23:45:58,046 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:45:58,059 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd732d0e93e2314723e2b5 HTTP/1.1" 200 None 2026-03-26 23:45:58,061 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:45:58,087 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c322306778148209fd6f15 HTTP/1.1" 200 None 2026-03-26 23:45:58,099 - lava_callback - ERROR - Error processing callback for node 69c322306778148209fd6f15: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:40932 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33584 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33598 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33600 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57072 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47834 - "GET / HTTP/1.0" 200 OK 2026-03-26 23:46:38,029 - lava_callback - INFO - Received callback for node 69c322546778148209fd6f29 INFO: 172.21.0.1:47844 - "POST /node/69c322546778148209fd6f29 HTTP/1.0" 202 Accepted 2026-03-26 23:46:38,035 - lava_callback - INFO - Starting async job submit for node 69c322546778148209fd6f29 2026-03-26 23:46:38,036 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:46:38,050 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c322546778148209fd6f29 HTTP/1.1" 200 None 2026-03-26 23:46:38,050 - lava_callback - DEBUG - Processing node 69c322546778148209fd6f29: baseline-arm-clabbe 2026-03-26 23:46:38,051 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c322546778148209fd6f29 2026-03-26 23:46:38,051 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:46:38,064 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c322546778148209fd6f29 HTTP/1.1" 200 None 2026-03-26 23:46:38,066 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:46:38,121 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd732d0e93e2314723e2b5 HTTP/1.1" 200 None 2026-03-26 23:46:38,123 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:46:38,145 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c322546778148209fd6f29 HTTP/1.1" 200 None 2026-03-26 23:46:38,146 - lava_callback - ERROR - Error processing callback for node 69c322546778148209fd6f29: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:36686 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57780 - "GET / HTTP/1.0" 200 OK 2026-03-26 23:46:58,017 - lava_callback - INFO - Received callback for node 69c5c53cabf66a8f4d9bd246 INFO: 172.21.0.1:57790 - "POST /node/69c5c53cabf66a8f4d9bd246 HTTP/1.0" 202 Accepted 2026-03-26 23:46:58,024 - lava_callback - INFO - Starting async job submit for node 69c5c53cabf66a8f4d9bd246 2026-03-26 23:46:58,053 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:46:58,069 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5c53cabf66a8f4d9bd246 HTTP/1.1" 200 None 2026-03-26 23:46:58,070 - lava_callback - DEBUG - Processing node 69c5c53cabf66a8f4d9bd246: baseline-arm64 2026-03-26 23:46:58,202 - lava_callback - WARNING - Failed to upload log 2026-03-26 23:46:58,271 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 23:46:58,533 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 23:46:58,535 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c5c53cabf66a8f4d9bd246/lava-logs.yaml 2026-03-26 23:46:58,538 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 23:46:58,710 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 23:46:58,712 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c5c53cabf66a8f4d9bd246/lava_callback.json.gz 2026-03-26 23:46:58,742 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:46:58,762 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5c53cabf66a8f4d9bd246 HTTP/1.1" 200 None 2026-03-26 23:46:58,763 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:46:58,776 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-03-26 23:46:58,777 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:46:58,798 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5c53cabf66a8f4d9bd246 HTTP/1.1" 200 None 2026-03-26 23:46:58,864 - lava_callback - INFO - Completed processing callback for node 69c5c53cabf66a8f4d9bd246 2026-03-26 23:47:03,614 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:47:03,629 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:42844 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55498 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55506 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55522 - "GET / HTTP/1.0" 200 OK 2026-03-26 23:47:18,034 - lava_callback - INFO - Received callback for node 69c322796778148209fd6f34 INFO: 172.21.0.1:55528 - "POST /node/69c322796778148209fd6f34 HTTP/1.0" 202 Accepted 2026-03-26 23:47:18,041 - lava_callback - INFO - Starting async job submit for node 69c322796778148209fd6f34 2026-03-26 23:47:18,041 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:47:18,056 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c322796778148209fd6f34 HTTP/1.1" 200 None 2026-03-26 23:47:18,057 - lava_callback - DEBUG - Processing node 69c322796778148209fd6f34: baseline-arm-clabbe 2026-03-26 23:47:18,057 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c322796778148209fd6f34 2026-03-26 23:47:18,058 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:47:18,103 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c322796778148209fd6f34 HTTP/1.1" 200 None 2026-03-26 23:47:18,104 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:47:18,289 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd732d0e93e2314723e2b5 HTTP/1.1" 200 None 2026-03-26 23:47:18,290 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:47:18,329 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c322796778148209fd6f34 HTTP/1.1" 200 None 2026-03-26 23:47:18,331 - lava_callback - ERROR - Error processing callback for node 69c322796778148209fd6f34: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:51656 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54136 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39630 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38062 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41530 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41832 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41840 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41842 - "GET / HTTP/1.0" 200 OK 2026-03-26 23:48:18,138 - lava_callback - INFO - Received callback for node 69c322ac6778148209fd6f41 INFO: 172.21.0.1:41848 - "POST /node/69c322ac6778148209fd6f41 HTTP/1.0" 202 Accepted 2026-03-26 23:48:18,144 - lava_callback - INFO - Starting async job submit for node 69c322ac6778148209fd6f41 2026-03-26 23:48:18,144 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:48:18,159 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c322ac6778148209fd6f41 HTTP/1.1" 200 None 2026-03-26 23:48:18,160 - lava_callback - DEBUG - Processing node 69c322ac6778148209fd6f41: baseline-arm64-clabbe 2026-03-26 23:48:18,160 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c322ac6778148209fd6f41 2026-03-26 23:48:18,161 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:48:18,174 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c322ac6778148209fd6f41 HTTP/1.1" 200 None 2026-03-26 23:48:18,175 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:48:18,190 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd73420e93e2314723e2c4 HTTP/1.1" 200 None 2026-03-26 23:48:18,191 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:48:18,211 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c322ac6778148209fd6f41 HTTP/1.1" 200 None 2026-03-26 23:48:18,212 - lava_callback - ERROR - Error processing callback for node 69c322ac6778148209fd6f41: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:50526 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50064 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42482 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50888 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54714 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52668 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52682 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52690 - "GET / HTTP/1.0" 200 OK 2026-03-26 23:49:18,156 - lava_callback - INFO - Received callback for node 69c322ec6778148209fd6f4e INFO: 172.21.0.1:52702 - "POST /node/69c322ec6778148209fd6f4e HTTP/1.0" 202 Accepted 2026-03-26 23:49:18,161 - lava_callback - INFO - Starting async job submit for node 69c322ec6778148209fd6f4e 2026-03-26 23:49:18,162 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:49:18,178 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c322ec6778148209fd6f4e HTTP/1.1" 200 None 2026-03-26 23:49:18,179 - lava_callback - DEBUG - Processing node 69c322ec6778148209fd6f4e: baseline-arm-clabbe 2026-03-26 23:49:18,179 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c322ec6778148209fd6f4e 2026-03-26 23:49:18,180 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:49:18,193 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c322ec6778148209fd6f4e HTTP/1.1" 200 None 2026-03-26 23:49:18,194 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:49:18,207 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd732d0e93e2314723e2b5 HTTP/1.1" 200 None 2026-03-26 23:49:18,209 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:49:18,230 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c322ec6778148209fd6f4e HTTP/1.1" 200 None 2026-03-26 23:49:18,231 - lava_callback - ERROR - Error processing callback for node 69c322ec6778148209fd6f4e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:48596 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56948 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47346 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44036 - "GET / HTTP/1.0" 200 OK 2026-03-26 23:49:58,142 - lava_callback - INFO - Received callback for node 69c3231f6778148209fd6f57 INFO: 172.21.0.1:44038 - "POST /node/69c3231f6778148209fd6f57 HTTP/1.0" 202 Accepted 2026-03-26 23:49:58,148 - lava_callback - INFO - Starting async job submit for node 69c3231f6778148209fd6f57 2026-03-26 23:49:58,149 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:49:58,162 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3231f6778148209fd6f57 HTTP/1.1" 200 None 2026-03-26 23:49:58,163 - lava_callback - DEBUG - Processing node 69c3231f6778148209fd6f57: baseline-arm64-clabbe 2026-03-26 23:49:58,163 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3231f6778148209fd6f57 2026-03-26 23:49:58,164 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:49:58,177 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3231f6778148209fd6f57 HTTP/1.1" 200 None 2026-03-26 23:49:58,178 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:49:58,191 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd73420e93e2314723e2c4 HTTP/1.1" 200 None 2026-03-26 23:49:58,193 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:49:58,214 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3231f6778148209fd6f57 HTTP/1.1" 200 None 2026-03-26 23:49:58,215 - lava_callback - ERROR - Error processing callback for node 69c3231f6778148209fd6f57: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:35194 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48584 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48600 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-26 23:50:15,171 - lava_callback - INFO - Received callback for node 69c5c2d5abf66a8f4d9bd1bd INFO: 172.21.0.1:48606 - "POST /node/69c5c2d5abf66a8f4d9bd1bd HTTP/1.0" 202 Accepted 2026-03-26 23:50:15,179 - lava_callback - INFO - Starting async job submit for node 69c5c2d5abf66a8f4d9bd1bd 2026-03-26 23:50:15,241 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:50:15,256 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5c2d5abf66a8f4d9bd1bd HTTP/1.1" 200 None 2026-03-26 23:50:15,257 - lava_callback - DEBUG - Processing node 69c5c2d5abf66a8f4d9bd1bd: baseline-arm64 2026-03-26 23:50:15,975 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 23:50:16,309 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 23:50:16,310 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c5c2d5abf66a8f4d9bd1bd/log.txt.gz INFO: 172.21.0.1:48610 - "GET / HTTP/1.0" 200 OK 2026-03-26 23:50:16,745 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 23:50:17,006 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 23:50:17,008 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c5c2d5abf66a8f4d9bd1bd/lava-logs.yaml 2026-03-26 23:50:17,024 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 23:50:17,204 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 23:50:17,205 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c5c2d5abf66a8f4d9bd1bd/lava_callback.json.gz DEBUG: Setup failed for 69c5c2d5abf66a8f4d9bd1bd. Transitting job node result: incomplete -> fail 2026-03-26 23:50:17,308 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:50:17,332 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5c2d5abf66a8f4d9bd1bd HTTP/1.1" 200 None 2026-03-26 23:50:17,333 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:50:17,354 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-03-26 23:50:17,355 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:50:17,382 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5c2d5abf66a8f4d9bd1bd HTTP/1.1" 200 None 2026-03-26 23:50:17,476 - lava_callback - INFO - Completed processing callback for node 69c5c2d5abf66a8f4d9bd1bd INFO: 172.21.0.1:33692 - "GET / HTTP/1.0" 200 OK 2026-03-26 23:50:33,632 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:50:33,648 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:52272 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50904 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36792 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37432 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46106 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46108 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46110 - "GET / HTTP/1.0" 200 OK 2026-03-26 23:51:18,196 - lava_callback - INFO - Received callback for node 69c3236d6778148209fd6f66 INFO: 172.21.0.1:46116 - "POST /node/69c3236d6778148209fd6f66 HTTP/1.0" 202 Accepted 2026-03-26 23:51:18,202 - lava_callback - INFO - Starting async job submit for node 69c3236d6778148209fd6f66 2026-03-26 23:51:18,203 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:51:18,219 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3236d6778148209fd6f66 HTTP/1.1" 200 None 2026-03-26 23:51:18,220 - lava_callback - DEBUG - Processing node 69c3236d6778148209fd6f66: baseline-arm64-clabbe 2026-03-26 23:51:18,220 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3236d6778148209fd6f66 2026-03-26 23:51:18,221 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:51:18,234 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3236d6778148209fd6f66 HTTP/1.1" 200 None 2026-03-26 23:51:18,235 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:51:18,249 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd73420e93e2314723e2c4 HTTP/1.1" 200 None 2026-03-26 23:51:18,250 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:51:18,271 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3236d6778148209fd6f66 HTTP/1.1" 200 None 2026-03-26 23:51:18,272 - lava_callback - ERROR - Error processing callback for node 69c3236d6778148209fd6f66: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:36326 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43294 - "GET / HTTP/1.0" 200 OK 2026-03-26 23:51:42,270 - lava_callback - INFO - Received callback for node 69c5c331abf66a8f4d9bd1cd INFO: 172.21.0.1:54436 - "POST /node/69c5c331abf66a8f4d9bd1cd HTTP/1.0" 202 Accepted 2026-03-26 23:51:42,278 - lava_callback - INFO - Starting async job submit for node 69c5c331abf66a8f4d9bd1cd 2026-03-26 23:51:42,330 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:51:42,347 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5c331abf66a8f4d9bd1cd HTTP/1.1" 200 None 2026-03-26 23:51:42,348 - lava_callback - DEBUG - Processing node 69c5c331abf66a8f4d9bd1cd: baseline-nfs-arm64 2026-03-26 23:51:43,098 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 23:51:43,265 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 23:51:43,267 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c5c331abf66a8f4d9bd1cd/log.txt.gz 2026-03-26 23:51:43,701 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 23:51:43,941 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 23:51:43,943 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c5c331abf66a8f4d9bd1cd/lava-logs.yaml 2026-03-26 23:51:43,959 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 23:51:44,130 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 23:51:44,132 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c5c331abf66a8f4d9bd1cd/lava_callback.json.gz DEBUG: Setup failed for 69c5c331abf66a8f4d9bd1cd. Transitting job node result: incomplete -> fail 2026-03-26 23:51:44,232 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:51:44,254 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5c331abf66a8f4d9bd1cd HTTP/1.1" 200 None 2026-03-26 23:51:44,256 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:51:44,269 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-03-26 23:51:44,271 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:51:44,296 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5c331abf66a8f4d9bd1cd HTTP/1.1" 200 None 2026-03-26 23:51:44,407 - lava_callback - INFO - Completed processing callback for node 69c5c331abf66a8f4d9bd1cd INFO: 172.21.0.1:54452 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38248 - "GET / HTTP/1.0" 200 OK 2026-03-26 23:52:03,650 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:52:03,667 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:39108 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33906 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33920 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33932 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46570 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57526 - "GET / HTTP/1.0" 200 OK 2026-03-26 23:52:38,318 - lava_callback - INFO - Received callback for node 69c323bc6778148209fd6f72 INFO: 172.21.0.1:57534 - "POST /node/69c323bc6778148209fd6f72 HTTP/1.0" 202 Accepted 2026-03-26 23:52:38,324 - lava_callback - INFO - Starting async job submit for node 69c323bc6778148209fd6f72 2026-03-26 23:52:38,325 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:52:38,339 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c323bc6778148209fd6f72 HTTP/1.1" 200 None 2026-03-26 23:52:38,340 - lava_callback - DEBUG - Processing node 69c323bc6778148209fd6f72: baseline-arm64-clabbe 2026-03-26 23:52:38,340 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c323bc6778148209fd6f72 2026-03-26 23:52:38,341 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:52:38,353 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c323bc6778148209fd6f72 HTTP/1.1" 200 None 2026-03-26 23:52:38,355 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:52:38,374 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd73420e93e2314723e2c4 HTTP/1.1" 200 None 2026-03-26 23:52:38,375 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:52:38,395 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c323bc6778148209fd6f72 HTTP/1.1" 200 None 2026-03-26 23:52:38,401 - lava_callback - ERROR - Error processing callback for node 69c323bc6778148209fd6f72: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:42364 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40702 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41630 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34362 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34378 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34384 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33442 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45418 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45760 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47904 - "GET / HTTP/1.0" 200 OK 2026-03-26 23:53:58,372 - lava_callback - INFO - Received callback for node 69c3240b6778148209fd6f7a INFO: 172.21.0.1:47920 - "POST /node/69c3240b6778148209fd6f7a HTTP/1.0" 202 Accepted 2026-03-26 23:53:58,378 - lava_callback - INFO - Starting async job submit for node 69c3240b6778148209fd6f7a 2026-03-26 23:53:58,379 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:53:58,406 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3240b6778148209fd6f7a HTTP/1.1" 200 None 2026-03-26 23:53:58,407 - lava_callback - DEBUG - Processing node 69c3240b6778148209fd6f7a: baseline-arm64-clabbe 2026-03-26 23:53:58,407 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3240b6778148209fd6f7a 2026-03-26 23:53:58,407 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:53:58,421 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3240b6778148209fd6f7a HTTP/1.1" 200 None 2026-03-26 23:53:58,422 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:53:58,435 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd73420e93e2314723e2c4 HTTP/1.1" 200 None 2026-03-26 23:53:58,436 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:53:58,464 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3240b6778148209fd6f7a HTTP/1.1" 200 None 2026-03-26 23:53:58,466 - lava_callback - ERROR - Error processing callback for node 69c3240b6778148209fd6f7a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:55162 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35808 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35818 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35822 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45154 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46374 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42546 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44496 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59624 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41684 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41700 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41716 - "GET / HTTP/1.0" 200 OK 2026-03-26 23:55:18,377 - lava_callback - INFO - Received callback for node 69c3245b6778148209fd6f88 INFO: 172.21.0.1:41732 - "POST /node/69c3245b6778148209fd6f88 HTTP/1.0" 202 Accepted 2026-03-26 23:55:18,383 - lava_callback - INFO - Starting async job submit for node 69c3245b6778148209fd6f88 2026-03-26 23:55:18,383 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:55:18,400 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3245b6778148209fd6f88 HTTP/1.1" 200 None 2026-03-26 23:55:18,401 - lava_callback - DEBUG - Processing node 69c3245b6778148209fd6f88: baseline-arm64-clabbe 2026-03-26 23:55:18,401 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3245b6778148209fd6f88 2026-03-26 23:55:18,402 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:55:18,428 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3245b6778148209fd6f88 HTTP/1.1" 200 None 2026-03-26 23:55:18,430 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:55:18,448 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd73420e93e2314723e2c4 HTTP/1.1" 200 None 2026-03-26 23:55:18,450 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:55:18,476 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3245b6778148209fd6f88 HTTP/1.1" 200 None 2026-03-26 23:55:18,477 - lava_callback - ERROR - Error processing callback for node 69c3245b6778148209fd6f88: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:47294 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60580 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50694 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47664 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53894 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40544 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40552 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40566 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34090 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45642 - "GET / HTTP/1.0" 200 OK 2026-03-26 23:56:38,427 - lava_callback - INFO - Received callback for node 69c324ac6778148209fd6f94 INFO: 172.21.0.1:45656 - "POST /node/69c324ac6778148209fd6f94 HTTP/1.0" 202 Accepted 2026-03-26 23:56:38,433 - lava_callback - INFO - Starting async job submit for node 69c324ac6778148209fd6f94 2026-03-26 23:56:38,434 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:56:38,447 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c324ac6778148209fd6f94 HTTP/1.1" 200 None 2026-03-26 23:56:38,448 - lava_callback - DEBUG - Processing node 69c324ac6778148209fd6f94: baseline-arm64-clabbe 2026-03-26 23:56:38,448 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c324ac6778148209fd6f94 2026-03-26 23:56:38,449 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:56:38,462 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c324ac6778148209fd6f94 HTTP/1.1" 200 None 2026-03-26 23:56:38,464 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:56:38,476 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd73420e93e2314723e2c4 HTTP/1.1" 200 None 2026-03-26 23:56:38,478 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:56:38,499 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c324ac6778148209fd6f94 HTTP/1.1" 200 None 2026-03-26 23:56:38,500 - lava_callback - ERROR - Error processing callback for node 69c324ac6778148209fd6f94: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:38928 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48908 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52680 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54938 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54946 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54960 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40754 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58042 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53410 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55892 - "GET / HTTP/1.0" 200 OK 2026-03-26 23:57:58,468 - lava_callback - INFO - Received callback for node 69c324fb6778148209fd6f9f INFO: 172.21.0.1:55906 - "POST /node/69c324fb6778148209fd6f9f HTTP/1.0" 202 Accepted 2026-03-26 23:57:58,473 - lava_callback - INFO - Starting async job submit for node 69c324fb6778148209fd6f9f 2026-03-26 23:57:58,474 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:57:58,502 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c324fb6778148209fd6f9f HTTP/1.1" 200 None 2026-03-26 23:57:58,503 - lava_callback - DEBUG - Processing node 69c324fb6778148209fd6f9f: baseline-arm64-clabbe 2026-03-26 23:57:58,503 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c324fb6778148209fd6f9f 2026-03-26 23:57:58,504 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:57:58,518 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c324fb6778148209fd6f9f HTTP/1.1" 200 None 2026-03-26 23:57:58,520 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:57:58,559 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd73420e93e2314723e2c4 HTTP/1.1" 200 None 2026-03-26 23:57:58,560 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:57:58,588 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c324fb6778148209fd6f9f HTTP/1.1" 200 None 2026-03-26 23:57:58,593 - lava_callback - ERROR - Error processing callback for node 69c324fb6778148209fd6f9f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:59896 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41806 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41808 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41816 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56412 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52366 - "GET / HTTP/1.0" 200 OK 2026-03-26 23:58:44,331 - lava_callback - INFO - Received callback for node 69c5be35abf66a8f4d9bd000 INFO: 172.21.0.1:55234 - "POST /node/69c5be35abf66a8f4d9bd000 HTTP/1.0" 202 Accepted 2026-03-26 23:58:44,338 - lava_callback - INFO - Starting async job submit for node 69c5be35abf66a8f4d9bd000 2026-03-26 23:58:44,360 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:58:44,375 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5be35abf66a8f4d9bd000 HTTP/1.1" 200 None 2026-03-26 23:58:44,376 - lava_callback - DEBUG - Processing node 69c5be35abf66a8f4d9bd000: baseline-arm64 2026-03-26 23:58:44,890 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 23:58:45,073 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 23:58:45,075 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c5be35abf66a8f4d9bd000/log.txt.gz 2026-03-26 23:58:45,374 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 23:58:45,588 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 23:58:45,590 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c5be35abf66a8f4d9bd000/lava-logs.yaml 2026-03-26 23:58:45,600 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 23:58:45,770 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 23:58:45,772 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c5be35abf66a8f4d9bd000/lava_callback.json.gz 2026-03-26 23:58:45,815 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:58:45,830 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5be35abf66a8f4d9bd000 HTTP/1.1" 200 None 2026-03-26 23:58:45,832 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:58:45,845 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-03-26 23:58:45,846 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:58:45,873 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5be35abf66a8f4d9bd000 HTTP/1.1" 200 None 2026-03-26 23:58:45,915 - lava_callback - INFO - Completed processing callback for node 69c5be35abf66a8f4d9bd000 INFO: 172.21.0.1:55236 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55470 - "GET / HTTP/1.0" 200 OK 2026-03-26 23:59:03,670 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:59:03,687 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:33626 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55024 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55026 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55040 - "GET / HTTP/1.0" 200 OK 2026-03-26 23:59:19,266 - lava_callback - INFO - Received callback for node 69c3254a6778148209fd6fae INFO: 172.21.0.1:55046 - "POST /node/69c3254a6778148209fd6fae HTTP/1.0" 202 Accepted 2026-03-26 23:59:19,272 - lava_callback - INFO - Starting async job submit for node 69c3254a6778148209fd6fae 2026-03-26 23:59:19,273 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:59:19,288 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3254a6778148209fd6fae HTTP/1.1" 200 None 2026-03-26 23:59:19,289 - lava_callback - DEBUG - Processing node 69c3254a6778148209fd6fae: baseline-arm64-clabbe 2026-03-26 23:59:19,289 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3254a6778148209fd6fae 2026-03-26 23:59:19,290 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:59:19,303 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3254a6778148209fd6fae HTTP/1.1" 200 None 2026-03-26 23:59:19,304 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:59:19,318 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd73420e93e2314723e2c4 HTTP/1.1" 200 None 2026-03-26 23:59:19,320 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 23:59:19,348 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3254a6778148209fd6fae HTTP/1.1" 200 None 2026-03-26 23:59:19,352 - lava_callback - ERROR - Error processing callback for node 69c3254a6778148209fd6fae: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:48480 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39356 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42008 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44100 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59556 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52842 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52856 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52864 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39378 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59408 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56238 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57714 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58830 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35854 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35870 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35878 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54478 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35298 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57724 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47152 - "GET / HTTP/1.0" 200 OK 2026-03-27 00:01:58,642 - lava_callback - INFO - Received callback for node 69c325f16778148209fd6fc3 INFO: 172.21.0.1:47164 - "POST /node/69c325f16778148209fd6fc3 HTTP/1.0" 202 Accepted 2026-03-27 00:01:58,648 - lava_callback - INFO - Starting async job submit for node 69c325f16778148209fd6fc3 2026-03-27 00:01:58,649 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:01:58,670 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c325f16778148209fd6fc3 HTTP/1.1" 200 None 2026-03-27 00:01:58,671 - lava_callback - DEBUG - Processing node 69c325f16778148209fd6fc3: baseline-arm64-clabbe 2026-03-27 00:01:58,671 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c325f16778148209fd6fc3 2026-03-27 00:01:58,672 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:01:58,685 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c325f16778148209fd6fc3 HTTP/1.1" 200 None 2026-03-27 00:01:58,686 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:01:58,703 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd73420e93e2314723e2c4 HTTP/1.1" 200 None 2026-03-27 00:01:58,705 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:01:58,726 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c325f16778148209fd6fc3 HTTP/1.1" 200 None 2026-03-27 00:01:58,733 - lava_callback - ERROR - Error processing callback for node 69c325f16778148209fd6fc3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:54234 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44990 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45006 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45012 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42924 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51592 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38370 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42712 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44546 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41268 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41274 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41286 - "GET / HTTP/1.0" 200 OK 2026-03-27 00:03:18,757 - lava_callback - INFO - Received callback for node 69c326426778148209fd6fd1 INFO: 172.21.0.1:41294 - "POST /node/69c326426778148209fd6fd1 HTTP/1.0" 202 Accepted 2026-03-27 00:03:18,763 - lava_callback - INFO - Starting async job submit for node 69c326426778148209fd6fd1 2026-03-27 00:03:18,764 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:03:18,780 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c326426778148209fd6fd1 HTTP/1.1" 200 None 2026-03-27 00:03:18,781 - lava_callback - DEBUG - Processing node 69c326426778148209fd6fd1: baseline-arm64-clabbe 2026-03-27 00:03:18,781 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c326426778148209fd6fd1 2026-03-27 00:03:18,782 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:03:18,796 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c326426778148209fd6fd1 HTTP/1.1" 200 None 2026-03-27 00:03:18,797 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:03:18,810 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd73420e93e2314723e2c4 HTTP/1.1" 200 None 2026-03-27 00:03:18,811 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:03:18,832 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c326426778148209fd6fd1 HTTP/1.1" 200 None 2026-03-27 00:03:18,833 - lava_callback - ERROR - Error processing callback for node 69c326426778148209fd6fd1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:34658 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35966 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40712 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48632 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41856 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34934 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34950 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34956 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40804 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47476 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40954 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57884 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37610 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44478 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44488 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44492 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59672 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38368 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35952 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52142 - "GET / HTTP/1.0" 200 OK 2026-03-27 00:05:58,790 - lava_callback - INFO - Received callback for node 69c326d56778148209fd6fe9 INFO: 172.21.0.1:52152 - "POST /node/69c326d56778148209fd6fe9 HTTP/1.0" 202 Accepted 2026-03-27 00:05:58,796 - lava_callback - INFO - Starting async job submit for node 69c326d56778148209fd6fe9 2026-03-27 00:05:58,797 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:05:58,812 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c326d56778148209fd6fe9 HTTP/1.1" 200 None 2026-03-27 00:05:58,813 - lava_callback - DEBUG - Processing node 69c326d56778148209fd6fe9: baseline-arm-clabbe 2026-03-27 00:05:58,813 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c326d56778148209fd6fe9 2026-03-27 00:05:58,814 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:05:58,827 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c326d56778148209fd6fe9 HTTP/1.1" 200 None 2026-03-27 00:05:58,829 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:05:58,842 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd732d0e93e2314723e2b5 HTTP/1.1" 200 None 2026-03-27 00:05:58,843 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:05:58,864 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c326d56778148209fd6fe9 HTTP/1.1" 200 None 2026-03-27 00:05:58,869 - lava_callback - ERROR - Error processing callback for node 69c326d56778148209fd6fe9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:35524 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48614 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48626 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48636 - "GET / HTTP/1.0" 200 OK 2026-03-27 00:06:25,899 - lava_callback - INFO - Received callback for node 69c326fe6778148209fd6ff5 INFO: 172.21.0.1:59480 - "POST /node/69c326fe6778148209fd6ff5 HTTP/1.0" 202 Accepted 2026-03-27 00:06:25,904 - lava_callback - INFO - Starting async job submit for node 69c326fe6778148209fd6ff5 2026-03-27 00:06:25,905 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:06:25,924 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c326fe6778148209fd6ff5 HTTP/1.1" 200 None 2026-03-27 00:06:25,925 - lava_callback - DEBUG - Processing node 69c326fe6778148209fd6ff5: baseline-arm64-baylibre 2026-03-27 00:06:25,925 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c326fe6778148209fd6ff5 2026-03-27 00:06:25,926 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:06:25,940 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c326fe6778148209fd6ff5 HTTP/1.1" 200 None 2026-03-27 00:06:25,941 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:06:25,954 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0163fc3d73288b0d05306 HTTP/1.1" 200 None 2026-03-27 00:06:25,956 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:06:25,986 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c326fe6778148209fd6ff5 HTTP/1.1" 200 None 2026-03-27 00:06:25,987 - lava_callback - ERROR - Error processing callback for node 69c326fe6778148209fd6ff5: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:59488 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55368 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44236 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41932 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38528 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37466 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37478 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37484 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39138 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39374 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33410 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35756 - "GET / HTTP/1.0" 200 OK 2026-03-27 00:07:58,913 - lava_callback - INFO - Received callback for node 69c327496778148209fd7004 INFO: 172.21.0.1:35762 - "POST /node/69c327496778148209fd7004 HTTP/1.0" 202 Accepted 2026-03-27 00:07:58,919 - lava_callback - INFO - Starting async job submit for node 69c327496778148209fd7004 2026-03-27 00:07:58,920 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:07:58,941 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c327496778148209fd7004 HTTP/1.1" 200 None 2026-03-27 00:07:58,942 - lava_callback - DEBUG - Processing node 69c327496778148209fd7004: baseline-arm-clabbe 2026-03-27 00:07:58,942 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c327496778148209fd7004 2026-03-27 00:07:58,943 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:07:58,957 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c327496778148209fd7004 HTTP/1.1" 200 None 2026-03-27 00:07:58,958 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:07:58,985 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd732d0e93e2314723e2b5 HTTP/1.1" 200 None 2026-03-27 00:07:58,986 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:07:59,008 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c327496778148209fd7004 HTTP/1.1" 200 None 2026-03-27 00:07:59,009 - lava_callback - ERROR - Error processing callback for node 69c327496778148209fd7004: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:55254 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48666 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48680 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48682 - "GET / HTTP/1.0" 200 OK 2026-03-27 00:08:18,885 - lava_callback - INFO - Received callback for node 69c3276d6778148209fd700e INFO: 172.21.0.1:48688 - "POST /node/69c3276d6778148209fd700e HTTP/1.0" 202 Accepted 2026-03-27 00:08:18,891 - lava_callback - INFO - Starting async job submit for node 69c3276d6778148209fd700e 2026-03-27 00:08:18,892 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:08:18,926 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3276d6778148209fd700e HTTP/1.1" 200 None 2026-03-27 00:08:18,926 - lava_callback - DEBUG - Processing node 69c3276d6778148209fd700e: baseline-arm-clabbe 2026-03-27 00:08:18,927 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3276d6778148209fd700e 2026-03-27 00:08:18,927 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:08:18,941 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3276d6778148209fd700e HTTP/1.1" 200 None 2026-03-27 00:08:18,942 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:08:18,955 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd732d0e93e2314723e2b5 HTTP/1.1" 200 None 2026-03-27 00:08:18,957 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:08:18,977 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3276d6778148209fd700e HTTP/1.1" 200 None 2026-03-27 00:08:18,983 - lava_callback - ERROR - Error processing callback for node 69c3276d6778148209fd700e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:55168 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36134 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59598 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41598 - "GET / HTTP/1.0" 200 OK 2026-03-27 00:09:05,549 - lava_callback - INFO - Received callback for node 69c327976778148209fd701a INFO: 172.21.0.1:55434 - "POST /node/69c327976778148209fd701a HTTP/1.0" 202 Accepted 2026-03-27 00:09:05,556 - lava_callback - INFO - Starting async job submit for node 69c327976778148209fd701a 2026-03-27 00:09:05,557 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:09:05,573 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c327976778148209fd701a HTTP/1.1" 200 None 2026-03-27 00:09:05,574 - lava_callback - DEBUG - Processing node 69c327976778148209fd701a: baseline-arm64-baylibre 2026-03-27 00:09:05,574 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c327976778148209fd701a 2026-03-27 00:09:05,575 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:09:05,588 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c327976778148209fd701a HTTP/1.1" 200 None 2026-03-27 00:09:05,590 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:09:05,603 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0163fc3d73288b0d05306 HTTP/1.1" 200 None 2026-03-27 00:09:05,604 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:09:05,633 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c327976778148209fd701a HTTP/1.1" 200 None 2026-03-27 00:09:05,639 - lava_callback - ERROR - Error processing callback for node 69c327976778148209fd701a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:55450 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40742 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40746 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40754 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35140 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53756 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58116 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39190 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47958 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43008 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43024 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43026 - "GET / HTTP/1.0" 200 OK 2026-03-27 00:10:19,035 - lava_callback - INFO - Received callback for node 69c327e26778148209fd7026 INFO: 172.21.0.1:43038 - "POST /node/69c327e26778148209fd7026 HTTP/1.0" 202 Accepted 2026-03-27 00:10:19,040 - lava_callback - INFO - Starting async job submit for node 69c327e26778148209fd7026 2026-03-27 00:10:19,041 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:10:19,056 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c327e26778148209fd7026 HTTP/1.1" 200 None 2026-03-27 00:10:19,057 - lava_callback - DEBUG - Processing node 69c327e26778148209fd7026: baseline-arm-clabbe 2026-03-27 00:10:19,057 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c327e26778148209fd7026 2026-03-27 00:10:19,058 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:10:19,072 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c327e26778148209fd7026 HTTP/1.1" 200 None 2026-03-27 00:10:19,073 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:10:19,087 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd732d0e93e2314723e2b5 HTTP/1.1" 200 None 2026-03-27 00:10:19,089 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:10:19,114 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c327e26778148209fd7026 HTTP/1.1" 200 None 2026-03-27 00:10:19,115 - lava_callback - ERROR - Error processing callback for node 69c327e26778148209fd7026: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:42622 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57040 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55976 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53918 - "GET / HTTP/1.0" 200 OK 2026-03-27 00:11:05,615 - lava_callback - INFO - Received callback for node 69c3280d6778148209fd7030 INFO: 172.21.0.1:46678 - "POST /node/69c3280d6778148209fd7030 HTTP/1.0" 202 Accepted 2026-03-27 00:11:05,624 - lava_callback - INFO - Starting async job submit for node 69c3280d6778148209fd7030 2026-03-27 00:11:05,625 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:11:05,671 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3280d6778148209fd7030 HTTP/1.1" 200 None 2026-03-27 00:11:05,673 - lava_callback - DEBUG - Processing node 69c3280d6778148209fd7030: baseline-arm64-baylibre 2026-03-27 00:11:05,673 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3280d6778148209fd7030 2026-03-27 00:11:05,674 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:11:05,701 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3280d6778148209fd7030 HTTP/1.1" 200 None 2026-03-27 00:11:05,703 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:11:05,721 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0163fc3d73288b0d05306 HTTP/1.1" 200 None 2026-03-27 00:11:05,723 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:11:05,863 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3280d6778148209fd7030 HTTP/1.1" 200 None 2026-03-27 00:11:05,865 - lava_callback - ERROR - Error processing callback for node 69c3280d6778148209fd7030: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:46682 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49770 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49778 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49790 - "GET / HTTP/1.0" 200 OK INFO: Shutting down INFO: Waiting for application shutdown. INFO: Application shutdown complete. INFO: Finished server process [1]