Format string error in param 'http://storage.kernelci.org/images/rootfs/debian/bookworm/20240313.0/{debarch}': 'debarch' Format string error in param 'http://storage.kernelci.org/images/rootfs/debian/bookworm/20240313.0/{debarch}': 'debarch' Format string error in param 'http://storage.kernelci.org/images/rootfs/debian/bookworm/20240313.0/{debarch}': 'debarch' Format string error in param 'http://storage.kernelci.org/images/rootfs/debian/bookworm/20240313.0/{debarch}': 'debarch' Format string error in param 'http://storage.kernelci.org/images/rootfs/debian/bookworm/20240313.0/{debarch}': 'debarch' Format string error in param 'http://storage.kernelci.org/images/rootfs/debian/bookworm/20240313.0/{debarch}': 'debarch' Format string error in param 'http://storage.kernelci.org/images/rootfs/debian/bookworm/20240313.0/{debarch}': 'debarch' Format string error in param 'http://storage.kernelci.org/images/rootfs/debian/bookworm/20240313.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-kselftest/20250724.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-kselftest/20250724.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-kselftest/20250724.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-kselftest/20250724.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-kselftest/20250724.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-kselftest/20250724.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-kselftest/20250724.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-kselftest/20250724.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-kselftest/20250724.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-kselftest/20250724.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-kselftest/20250724.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-kselftest/20250724.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-kselftest/20250724.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-kselftest/20250724.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-kselftest/20250724.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-kselftest/20250724.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-kselftest/20250724.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-kselftest/20250724.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-kselftest/20250724.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-kselftest/20250724.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-kselftest/20250724.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-kselftest/20250724.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-kselftest/20250724.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-kselftest/20250724.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-kselftest/20250724.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-kselftest/20250724.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-kselftest/20250724.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-kselftest/20250724.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-kselftest/20250724.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-kselftest/20250724.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-kselftest/20250724.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-kselftest/20250724.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-kselftest/20250724.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-kselftest/20250724.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-kselftest/20250724.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-kselftest/20250724.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-kselftest/20250724.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-kselftest/20250724.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-kselftest/20250724.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-kselftest/20250724.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-kselftest/20250724.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-kselftest/20250724.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-kselftest/20250724.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-kselftest/20250724.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-kselftest/20250724.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-kselftest/20250724.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-kselftest/20250724.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-kselftest/20250724.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-kselftest/20250724.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-kselftest/20250724.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-kselftest/20250724.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-kselftest/20250724.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-kselftest/20250724.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-kvm-unit-tests/20250624.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-ltp/20250818.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-ltp/20250818.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-ltp/20250818.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-ltp/20250818.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-ltp/20250818.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-ltp/20250818.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-ltp/20250818.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-ltp/20250818.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-ltp/20250818.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-ltp/20250818.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-ltp/20250818.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-ltp/20250818.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-ltp/20250818.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-ltp/20250818.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-ltp/20250818.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-ltp/20250818.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-ltp/20250818.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-ltp/20250818.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-ltp/20250818.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-ltp/20250818.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-ltp/20250818.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-ltp/20250818.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-ltp/20250818.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-ltp/20250818.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-ltp/20250818.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-ltp/20250818.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-ltp/20250818.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-ltp/20250818.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-rt/20240806.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-rt/20240806.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-rt/20240806.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-rt/20240806.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-rt/20240806.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-rt/20240806.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-rt/20240806.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-rt/20240806.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-rt/20240806.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-rt/20240806.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-rt/20240806.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-rt/20240806.0/{debarch}': 'debarch' Format string error in param 'http://storage.kernelci.org/images/rootfs/debian/bullseye/20240129.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-gst-h26forge/20250117.0/{debarch}/': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-wifi/20250925.0/{debarch}/': 'debarch' Format string error in param 'http://storage.kernelci.org/images/rootfs/debian/bookworm/20240313.0/{debarch}': 'debarch' Format string error in param 'http://storage.kernelci.org/images/rootfs/debian/bookworm/20240313.0/{debarch}': 'debarch' Format string error in param 'http://storage.kernelci.org/images/rootfs/debian/bookworm/20240313.0/{debarch}': 'debarch' Format string error in param 'http://storage.kernelci.org/images/rootfs/debian/bookworm/20240313.0/{debarch}': 'debarch' Format string error in param 'http://storage.kernelci.org/images/rootfs/debian/bookworm/20240313.0/{debarch}': 'debarch' Format string error in param 'http://storage.kernelci.org/images/rootfs/debian/bookworm/20240313.0/{debarch}': 'debarch' Format string error in param 'http://storage.kernelci.org/images/rootfs/debian/bookworm/20240313.0/{debarch}': 'debarch' Format string error in param 'http://storage.kernelci.org/images/rootfs/debian/bookworm/20240313.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-fault-injection/20250520.0/{debarch}/': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-ltp/20250818.0/{debarch}': 'debarch' Format string error in param 'cros://chromeos-{krev}/{crosarch}/chromiumos-{flavour}.flavour.config': 'krev' Format string error in param 'cros://chromeos-{krev}/{crosarch}/chromiumos-{flavour}.flavour.config': 'krev' Format string error in param 'cros://chromeos-{krev}/{crosarch}/chromiumos-{flavour}.flavour.config': 'krev' Format string error in param 'cros://chromeos-{krev}/{crosarch}/chromiumos-{flavour}.flavour.config': 'krev' Format string error in param 'cros://chromeos-{krev}/{crosarch}/chromiumos-{flavour}.flavour.config': 'krev' Format string error in param 'cros://chromeos-{krev}/{crosarch}/chromiumos-{flavour}.flavour.config': 'krev' Format string error in param 'cros://chromeos-{krev}/{crosarch}/chromiumos-{flavour}.flavour.config': 'krev' Format string error in param 'cros://chromeos-{krev}/{crosarch}/chromiumos-{flavour}.flavour.config': 'krev' Format string error in param 'cros://chromeos-{krev}/{crosarch}/chromiumos-{flavour}.flavour.config': 'krev' Format string error in param 'cros://chromeos-{krev}/{crosarch}/chromiumos-{flavour}.flavour.config': 'krev' Format string error in param 'cros://chromeos-{krev}/{crosarch}/chromeos-{flavour}.flavour.config': 'krev' Format string error in param 'cros://chromeos-{krev}/{crosarch}/chromeos-{flavour}.flavour.config': 'krev' Format string error in param 'cros://chromeos-{krev}/{crosarch}/chromeos-{flavour}.flavour.config': 'krev' Format string error in param 'cros://chromeos-{krev}/{crosarch}/chromeos-{flavour}.flavour.config': 'krev' Format string error in param 'cros://chromeos-{krev}/{crosarch}/chromeos-{flavour}.flavour.config': 'krev' INFO: Started server process [1] INFO: Waiting for application startup. INFO: Application startup complete. INFO: Uvicorn running on http://0.0.0.0:8000 (Press CTRL+C to quit) Format string error in param 'cros://chromeos-{krev}/{crosarch}/chromeos-{flavour}.flavour.config': 'krev' Format string error in param 'cros://chromeos-{krev}/{crosarch}/chromeos-{flavour}.flavour.config': 'krev' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-kselftest/20250724.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-kselftest/20250724.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-ltp/20250818.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-ltp/20250818.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-ltp/20250818.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-ltp/20250818.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-ltp/20250818.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-ltp/20250818.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-ltp/20250818.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-ltp/20250818.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-ltp/20250818.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-ltp/20250818.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-gst-fluster/20240926.0/{debarch}/': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-gst-fluster/20240926.0/{debarch}/': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-gst-fluster/20240926.0/{debarch}/': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-gst-fluster/20240926.0/{debarch}/': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-gst-fluster/20240926.0/{debarch}/': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-gst-fluster/20240926.0/{debarch}/': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-gst-fluster/20240926.0/{debarch}/': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-wifi/20250925.0/{debarch}/': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-wifi/20250925.0/{debarch}/': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-wifi/20250925.0/{debarch}/': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-wifi/20250925.0/{debarch}/': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-wifi/20250925.0/{debarch}/': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-wifi/20250925.0/{debarch}/': 'debarch' INFO: 172.21.0.1:35340 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51868 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51872 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32802 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51838 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41212 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41214 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52616 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52628 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57396 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44342 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44346 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48944 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51046 - "GET / HTTP/1.0" 200 OK 2025-09-26 16:19:08,053 - lava_callback - INFO - Received callback for node 68d419f6a5c2c05277bc5aa9 2025-09-26 16:19:08,060 - lava_callback - INFO - Starting async job submit for node 68d419f6a5c2c05277bc5aa9 INFO: 172.21.0.1:51050 - "POST /node/68d419f6a5c2c05277bc5aa9 HTTP/1.0" 202 Accepted 2025-09-26 16:19:08,062 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:19:08,079 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d419f6a5c2c05277bc5aa9 HTTP/1.1" 200 None 2025-09-26 16:19:08,080 - lava_callback - DEBUG - Processing node 68d419f6a5c2c05277bc5aa9: baseline-arm64-clabbe 2025-09-26 16:19:08,080 - lava_callback - ERROR - Error processing callback for node 68d419f6a5c2c05277bc5aa9: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:55424 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55440 - "GET / HTTP/1.0" 200 OK 2025-09-26 16:19:20,558 - lava_callback - INFO - Received callback for node 68d41a90a5c2c05277bc5b96 INFO: 172.21.0.1:39530 - "POST /node/68d41a90a5c2c05277bc5b96 HTTP/1.0" 202 Accepted 2025-09-26 16:19:20,564 - lava_callback - INFO - Starting async job submit for node 68d41a90a5c2c05277bc5b96 2025-09-26 16:19:20,586 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:19:20,619 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d41a90a5c2c05277bc5b96 HTTP/1.1" 200 None 2025-09-26 16:19:20,619 - lava_callback - DEBUG - Processing node 68d41a90a5c2c05277bc5b96: baseline-arm-broonie 2025-09-26 16:19:20,812 - lava_callback - ERROR - Error processing callback for node 68d41a90a5c2c05277bc5b96: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:39536 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39548 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44018 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60038 - "GET / HTTP/1.0" 200 OK 2025-09-26 16:19:48,050 - lava_callback - INFO - Received callback for node 68d41a0da5c2c05277bc5ab7 INFO: 172.21.0.1:60054 - "POST /node/68d41a0da5c2c05277bc5ab7 HTTP/1.0" 202 Accepted 2025-09-26 16:19:48,055 - lava_callback - INFO - Starting async job submit for node 68d41a0da5c2c05277bc5ab7 2025-09-26 16:19:48,056 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:19:48,070 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d41a0da5c2c05277bc5ab7 HTTP/1.1" 200 None 2025-09-26 16:19:48,071 - lava_callback - DEBUG - Processing node 68d41a0da5c2c05277bc5ab7: baseline-arm64-clabbe 2025-09-26 16:19:48,071 - lava_callback - ERROR - Error processing callback for node 68d41a0da5c2c05277bc5ab7: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:60062 - "GET / HTTP/1.0" 200 OK 2025-09-26 16:19:48,705 - lava_callback - INFO - Received callback for node 68d41a13a5c2c05277bc5acb INFO: 172.21.0.1:60068 - "POST /node/68d41a13a5c2c05277bc5acb HTTP/1.0" 202 Accepted 2025-09-26 16:19:48,711 - lava_callback - INFO - Starting async job submit for node 68d41a13a5c2c05277bc5acb 2025-09-26 16:19:48,712 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:19:48,735 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d41a13a5c2c05277bc5acb HTTP/1.1" 200 None 2025-09-26 16:19:48,735 - lava_callback - DEBUG - Processing node 68d41a13a5c2c05277bc5acb: baseline-arm64-clabbe 2025-09-26 16:19:48,735 - lava_callback - ERROR - Error processing callback for node 68d41a13a5c2c05277bc5acb: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 16:19:49,352 - lava_callback - INFO - Received callback for node 68d41a19a5c2c05277bc5acf INFO: 172.21.0.1:45958 - "POST /node/68d41a19a5c2c05277bc5acf HTTP/1.0" 202 Accepted 2025-09-26 16:19:49,357 - lava_callback - INFO - Starting async job submit for node 68d41a19a5c2c05277bc5acf 2025-09-26 16:19:49,358 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:19:49,371 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d41a19a5c2c05277bc5acf HTTP/1.1" 200 None 2025-09-26 16:19:49,371 - lava_callback - DEBUG - Processing node 68d41a19a5c2c05277bc5acf: baseline-arm64-clabbe 2025-09-26 16:19:49,371 - lava_callback - ERROR - Error processing callback for node 68d41a19a5c2c05277bc5acf: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:45964 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44244 - "GET / HTTP/1.0" 200 OK 2025-09-26 16:20:08,186 - lava_callback - INFO - Received callback for node 68d41a24a5c2c05277bc5b09 INFO: 172.21.0.1:44246 - "POST /node/68d41a24a5c2c05277bc5b09 HTTP/1.0" 202 Accepted 2025-09-26 16:20:08,191 - lava_callback - INFO - Starting async job submit for node 68d41a24a5c2c05277bc5b09 2025-09-26 16:20:08,192 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:20:08,206 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d41a24a5c2c05277bc5b09 HTTP/1.1" 200 None 2025-09-26 16:20:08,206 - lava_callback - DEBUG - Processing node 68d41a24a5c2c05277bc5b09: baseline-arm64-clabbe 2025-09-26 16:20:08,206 - lava_callback - ERROR - Error processing callback for node 68d41a24a5c2c05277bc5b09: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:55634 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55638 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56346 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56356 - "GET / HTTP/1.0" 200 OK 2025-09-26 16:20:29,953 - lava_callback - INFO - Received callback for node 68d6b9e82f83206171a92b59 INFO: 172.21.0.1:40594 - "POST /node/68d6b9e82f83206171a92b59 HTTP/1.0" 202 Accepted 2025-09-26 16:20:29,962 - lava_callback - INFO - Starting async job submit for node 68d6b9e82f83206171a92b59 2025-09-26 16:20:30,002 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:20:30,019 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6b9e82f83206171a92b59 HTTP/1.1" 200 None 2025-09-26 16:20:30,019 - lava_callback - DEBUG - Processing node 68d6b9e82f83206171a92b59: baseline-arm64 2025-09-26 16:20:31,151 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 16:20:31,300 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 16:20:31,301 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6b9e82f83206171a92b59/log.txt.gz 2025-09-26 16:20:31,320 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 16:20:31,517 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 16:20:31,517 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6b9e82f83206171a92b59/lava_callback.json.gz DEBUG: Setup failed for 68d6b9e82f83206171a92b59. Transitting job node result: incomplete -> fail 2025-09-26 16:20:31,593 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:20:31,633 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6b9e82f83206171a92b59 HTTP/1.1" 200 None 2025-09-26 16:20:31,634 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:20:31,649 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d651ddcca86fb17472aa24 HTTP/1.1" 200 None 2025-09-26 16:20:31,650 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:20:31,684 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6b9e82f83206171a92b59 HTTP/1.1" 200 None 2025-09-26 16:20:31,685 - lava_callback - INFO - Completed processing callback for node 68d6b9e82f83206171a92b59 2025-09-26 16:20:36,094 - lava_callback - INFO - Received callback for node 68d6b9e72f83206171a92b56 INFO: 172.21.0.1:40600 - "POST /node/68d6b9e72f83206171a92b56 HTTP/1.0" 202 Accepted 2025-09-26 16:20:36,103 - lava_callback - INFO - Starting async job submit for node 68d6b9e72f83206171a92b56 2025-09-26 16:20:36,155 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:20:36,168 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6b9e72f83206171a92b56 HTTP/1.1" 200 None 2025-09-26 16:20:36,169 - lava_callback - DEBUG - Processing node 68d6b9e72f83206171a92b56: baseline-arm64 INFO: 172.21.0.1:40610 - "GET / HTTP/1.0" 200 OK 2025-09-26 16:20:37,516 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 16:20:37,637 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 16:20:37,638 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6b9e72f83206171a92b56/log.txt.gz 2025-09-26 16:20:37,656 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 16:20:37,823 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 16:20:37,824 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6b9e72f83206171a92b56/lava_callback.json.gz DEBUG: Setup failed for 68d6b9e72f83206171a92b56. Transitting job node result: incomplete -> fail 2025-09-26 16:20:37,899 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:20:37,911 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6b9e72f83206171a92b56 HTTP/1.1" 200 None 2025-09-26 16:20:37,912 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:20:37,923 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d651ddcca86fb17472aa24 HTTP/1.1" 200 None 2025-09-26 16:20:37,924 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:20:37,950 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6b9e72f83206171a92b56 HTTP/1.1" 200 None 2025-09-26 16:20:37,951 - lava_callback - INFO - Completed processing callback for node 68d6b9e72f83206171a92b56 INFO: 172.21.0.1:33804 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33812 - "GET / HTTP/1.0" 200 OK 2025-09-26 16:20:50,276 - lava_callback - INFO - Received callback for node 68d6b9f42f83206171a92b6b INFO: 172.21.0.1:48662 - "POST /node/68d6b9f42f83206171a92b6b HTTP/1.0" 202 Accepted 2025-09-26 16:20:50,284 - lava_callback - INFO - Starting async job submit for node 68d6b9f42f83206171a92b6b 2025-09-26 16:20:50,321 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:20:50,340 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6b9f42f83206171a92b6b HTTP/1.1" 200 None 2025-09-26 16:20:50,340 - lava_callback - DEBUG - Processing node 68d6b9f42f83206171a92b6b: baseline-arm64 2025-09-26 16:20:51,408 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 16:20:51,519 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 16:20:51,520 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6b9f42f83206171a92b6b/log.txt.gz 2025-09-26 16:20:51,540 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 16:20:51,747 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 16:20:51,747 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6b9f42f83206171a92b6b/lava_callback.json.gz DEBUG: Setup failed for 68d6b9f42f83206171a92b6b. Transitting job node result: incomplete -> fail 2025-09-26 16:20:51,830 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:20:51,845 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6b9f42f83206171a92b6b HTTP/1.1" 200 None 2025-09-26 16:20:51,846 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:20:51,859 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d651ddcca86fb17472aa24 HTTP/1.1" 200 None 2025-09-26 16:20:51,860 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:20:51,890 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6b9f42f83206171a92b6b HTTP/1.1" 200 None 2025-09-26 16:20:51,891 - lava_callback - INFO - Completed processing callback for node 68d6b9f42f83206171a92b6b 2025-09-26 16:20:56,093 - lava_callback - INFO - Received callback for node 68d6b9f52f83206171a92b6e INFO: 172.21.0.1:48672 - "POST /node/68d6b9f52f83206171a92b6e HTTP/1.0" 202 Accepted 2025-09-26 16:20:56,100 - lava_callback - INFO - Starting async job submit for node 68d6b9f52f83206171a92b6e 2025-09-26 16:20:56,138 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:20:56,151 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6b9f52f83206171a92b6e HTTP/1.1" 200 None 2025-09-26 16:20:56,151 - lava_callback - DEBUG - Processing node 68d6b9f52f83206171a92b6e: baseline-arm64 INFO: 172.21.0.1:48682 - "GET / HTTP/1.0" 200 OK 2025-09-26 16:20:57,315 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 16:20:57,443 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 16:20:57,443 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6b9f52f83206171a92b6e/log.txt.gz 2025-09-26 16:20:57,464 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 16:20:57,636 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 16:20:57,637 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6b9f52f83206171a92b6e/lava_callback.json.gz DEBUG: Setup failed for 68d6b9f52f83206171a92b6e. Transitting job node result: incomplete -> fail 2025-09-26 16:20:57,747 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:20:57,762 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6b9f52f83206171a92b6e HTTP/1.1" 200 None 2025-09-26 16:20:57,763 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:20:57,776 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d651ddcca86fb17472aa24 HTTP/1.1" 200 None 2025-09-26 16:20:57,777 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:20:57,804 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6b9f52f83206171a92b6e HTTP/1.1" 200 None 2025-09-26 16:20:57,805 - lava_callback - INFO - Completed processing callback for node 68d6b9f52f83206171a92b6e INFO: 172.21.0.1:52268 - "GET / HTTP/1.0" 200 OK 2025-09-26 16:21:09,209 - lava_callback - INFO - Received callback for node 68d41a60a5c2c05277bc5b5b INFO: 172.21.0.1:52278 - "POST /node/68d41a60a5c2c05277bc5b5b HTTP/1.0" 202 Accepted 2025-09-26 16:21:09,217 - lava_callback - INFO - Starting async job submit for node 68d41a60a5c2c05277bc5b5b 2025-09-26 16:21:09,218 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:21:09,230 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d41a60a5c2c05277bc5b5b HTTP/1.1" 200 None 2025-09-26 16:21:09,230 - lava_callback - DEBUG - Processing node 68d41a60a5c2c05277bc5b5b: baseline-arm 2025-09-26 16:21:09,230 - lava_callback - ERROR - Error processing callback for node 68d41a60a5c2c05277bc5b5b: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 16:21:09,927 - lava_callback - INFO - Received callback for node 68d41a61a5c2c05277bc5b5e INFO: 172.21.0.1:59082 - "POST /node/68d41a61a5c2c05277bc5b5e HTTP/1.0" 202 Accepted 2025-09-26 16:21:09,933 - lava_callback - INFO - Starting async job submit for node 68d41a61a5c2c05277bc5b5e 2025-09-26 16:21:09,934 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:21:09,946 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d41a61a5c2c05277bc5b5e HTTP/1.1" 200 None 2025-09-26 16:21:09,946 - lava_callback - DEBUG - Processing node 68d41a61a5c2c05277bc5b5e: baseline-arm 2025-09-26 16:21:09,946 - lava_callback - ERROR - Error processing callback for node 68d41a61a5c2c05277bc5b5e: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:59092 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59100 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39950 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39956 - "GET / HTTP/1.0" 200 OK 2025-09-26 16:21:28,415 - lava_callback - INFO - Received callback for node 68d41a73a5c2c05277bc5b78 INFO: 172.21.0.1:39968 - "POST /node/68d41a73a5c2c05277bc5b78 HTTP/1.0" 202 Accepted 2025-09-26 16:21:28,421 - lava_callback - INFO - Starting async job submit for node 68d41a73a5c2c05277bc5b78 2025-09-26 16:21:28,422 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:21:28,434 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d41a73a5c2c05277bc5b78 HTTP/1.1" 200 None 2025-09-26 16:21:28,434 - lava_callback - DEBUG - Processing node 68d41a73a5c2c05277bc5b78: baseline-arm 2025-09-26 16:21:28,434 - lava_callback - ERROR - Error processing callback for node 68d41a73a5c2c05277bc5b78: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 16:21:29,135 - lava_callback - INFO - Received callback for node 68d41a74a5c2c05277bc5b7b INFO: 172.21.0.1:39970 - "POST /node/68d41a74a5c2c05277bc5b7b HTTP/1.0" 202 Accepted 2025-09-26 16:21:29,143 - lava_callback - INFO - Starting async job submit for node 68d41a74a5c2c05277bc5b7b 2025-09-26 16:21:29,144 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:21:29,156 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d41a74a5c2c05277bc5b7b HTTP/1.1" 200 None 2025-09-26 16:21:29,157 - lava_callback - DEBUG - Processing node 68d41a74a5c2c05277bc5b7b: baseline-arm 2025-09-26 16:21:29,157 - lava_callback - ERROR - Error processing callback for node 68d41a74a5c2c05277bc5b7b: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:45554 - "GET / HTTP/1.0" 200 OK 2025-09-26 16:21:40,894 - lava_callback - INFO - Received callback for node 68d41a98a5c2c05277bc5b9f INFO: 172.21.0.1:37628 - "POST /node/68d41a98a5c2c05277bc5b9f HTTP/1.0" 202 Accepted 2025-09-26 16:21:40,900 - lava_callback - INFO - Starting async job submit for node 68d41a98a5c2c05277bc5b9f 2025-09-26 16:21:40,924 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:21:40,938 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d41a98a5c2c05277bc5b9f HTTP/1.1" 200 None 2025-09-26 16:21:40,938 - lava_callback - DEBUG - Processing node 68d41a98a5c2c05277bc5b9f: baseline-arm-broonie 2025-09-26 16:21:41,165 - lava_callback - ERROR - Error processing callback for node 68d41a98a5c2c05277bc5b9f: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:37642 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37644 - "GET / HTTP/1.0" 200 OK 2025-09-26 16:21:50,975 - lava_callback - INFO - Received callback for node 68d46e26a5c2c05277bcc416 INFO: 172.21.0.1:51840 - "POST /node/68d46e26a5c2c05277bcc416 HTTP/1.0" 202 Accepted 2025-09-26 16:21:50,981 - lava_callback - INFO - Starting async job submit for node 68d46e26a5c2c05277bcc416 2025-09-26 16:21:51,002 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:21:51,015 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d46e26a5c2c05277bcc416 HTTP/1.1" 200 None 2025-09-26 16:21:51,015 - lava_callback - DEBUG - Processing node 68d46e26a5c2c05277bcc416: baseline-arm 2025-09-26 16:21:51,326 - lava_callback - ERROR - Error processing callback for node 68d46e26a5c2c05277bcc416: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:51852 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55818 - "GET / HTTP/1.0" 200 OK 2025-09-26 16:22:08,366 - lava_callback - INFO - Received callback for node 68d41aa8a5c2c05277bc5bc0 INFO: 172.21.0.1:55824 - "POST /node/68d41aa8a5c2c05277bc5bc0 HTTP/1.0" 202 Accepted 2025-09-26 16:22:08,376 - lava_callback - INFO - Starting async job submit for node 68d41aa8a5c2c05277bc5bc0 2025-09-26 16:22:08,376 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:22:08,393 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d41aa8a5c2c05277bc5bc0 HTTP/1.1" 200 None 2025-09-26 16:22:08,393 - lava_callback - DEBUG - Processing node 68d41aa8a5c2c05277bc5bc0: baseline-arm-clabbe 2025-09-26 16:22:08,394 - lava_callback - ERROR - Error processing callback for node 68d41aa8a5c2c05277bc5bc0: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:44714 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44726 - "GET / HTTP/1.0" 200 OK 2025-09-26 16:22:20,943 - lava_callback - INFO - Received callback for node 68d41ac9a5c2c05277bc5bf0 INFO: 172.21.0.1:51800 - "POST /node/68d41ac9a5c2c05277bc5bf0 HTTP/1.0" 202 Accepted 2025-09-26 16:22:20,950 - lava_callback - INFO - Starting async job submit for node 68d41ac9a5c2c05277bc5bf0 2025-09-26 16:22:20,978 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:22:20,992 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d41ac9a5c2c05277bc5bf0 HTTP/1.1" 200 None 2025-09-26 16:22:20,992 - lava_callback - DEBUG - Processing node 68d41ac9a5c2c05277bc5bf0: ltp-cap-bounds 2025-09-26 16:22:21,364 - lava_callback - ERROR - Error processing callback for node 68d41ac9a5c2c05277bc5bf0: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:51802 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51818 - "GET / HTTP/1.0" 200 OK 2025-09-26 16:22:28,181 - lava_callback - INFO - Received callback for node 68d41aaea5c2c05277bc5bc6 INFO: 172.21.0.1:51820 - "POST /node/68d41aaea5c2c05277bc5bc6 HTTP/1.0" 202 Accepted 2025-09-26 16:22:28,189 - lava_callback - INFO - Starting async job submit for node 68d41aaea5c2c05277bc5bc6 2025-09-26 16:22:28,189 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:22:28,202 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d41aaea5c2c05277bc5bc6 HTTP/1.1" 200 None 2025-09-26 16:22:28,202 - lava_callback - DEBUG - Processing node 68d41aaea5c2c05277bc5bc6: baseline-arm-clabbe 2025-09-26 16:22:28,202 - lava_callback - ERROR - Error processing callback for node 68d41aaea5c2c05277bc5bc6: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 16:22:28,829 - lava_callback - INFO - Received callback for node 68d41abea5c2c05277bc5bde INFO: 172.21.0.1:51832 - "POST /node/68d41abea5c2c05277bc5bde HTTP/1.0" 202 Accepted 2025-09-26 16:22:28,834 - lava_callback - INFO - Starting async job submit for node 68d41abea5c2c05277bc5bde 2025-09-26 16:22:28,835 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:22:28,847 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d41abea5c2c05277bc5bde HTTP/1.1" 200 None 2025-09-26 16:22:28,847 - lava_callback - DEBUG - Processing node 68d41abea5c2c05277bc5bde: baseline-arm-clabbe 2025-09-26 16:22:28,848 - lava_callback - ERROR - Error processing callback for node 68d41abea5c2c05277bc5bde: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 16:22:29,468 - lava_callback - INFO - Received callback for node 68d41ab2a5c2c05277bc5bc9 INFO: 172.21.0.1:33186 - "POST /node/68d41ab2a5c2c05277bc5bc9 HTTP/1.0" 202 Accepted 2025-09-26 16:22:29,474 - lava_callback - INFO - Starting async job submit for node 68d41ab2a5c2c05277bc5bc9 2025-09-26 16:22:29,475 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:22:29,509 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d41ab2a5c2c05277bc5bc9 HTTP/1.1" 200 None 2025-09-26 16:22:29,510 - lava_callback - DEBUG - Processing node 68d41ab2a5c2c05277bc5bc9: baseline-arm-clabbe 2025-09-26 16:22:29,510 - lava_callback - ERROR - Error processing callback for node 68d41ab2a5c2c05277bc5bc9: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:33200 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43618 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43620 - "GET / HTTP/1.0" 200 OK 2025-09-26 16:22:52,241 - lava_callback - INFO - Received callback for node 68d6bdb23c94e462c6881517 INFO: 172.21.0.1:45158 - "POST /node/68d6bdb23c94e462c6881517 HTTP/1.0" 202 Accepted 2025-09-26 16:22:52,247 - lava_callback - INFO - Starting async job submit for node 68d6bdb23c94e462c6881517 2025-09-26 16:22:52,264 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:22:52,300 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6bdb23c94e462c6881517 HTTP/1.1" 200 None 2025-09-26 16:22:52,300 - lava_callback - DEBUG - Processing node 68d6bdb23c94e462c6881517: baseline-x86 2025-09-26 16:22:52,464 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 16:22:52,611 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 16:22:52,611 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-68d6bdb23c94e462c6881517/log.txt.gz 2025-09-26 16:22:52,617 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 16:22:52,770 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 16:22:52,771 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-68d6bdb23c94e462c6881517/lava_callback.json.gz 2025-09-26 16:22:52,789 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:22:52,802 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6bdb23c94e462c6881517 HTTP/1.1" 200 None 2025-09-26 16:22:52,803 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:22:52,815 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6b9f22f83206171a92b66 HTTP/1.1" 200 None 2025-09-26 16:22:52,816 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:22:52,846 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6bdb23c94e462c6881517 HTTP/1.1" 200 None 2025-09-26 16:22:52,847 - lava_callback - INFO - Completed processing callback for node 68d6bdb23c94e462c6881517 INFO: 172.21.0.1:45166 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53294 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44694 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44708 - "GET / HTTP/1.0" 200 OK 2025-09-26 16:23:25,743 - lava_callback - INFO - Received callback for node 68d41aeca5c2c05277bc5c3c INFO: 172.21.0.1:47260 - "POST /node/68d41aeca5c2c05277bc5c3c HTTP/1.0" 202 Accepted 2025-09-26 16:23:25,749 - lava_callback - INFO - Starting async job submit for node 68d41aeca5c2c05277bc5c3c 2025-09-26 16:23:25,750 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:23:25,770 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d41aeca5c2c05277bc5c3c HTTP/1.1" 200 None 2025-09-26 16:23:25,770 - lava_callback - DEBUG - Processing node 68d41aeca5c2c05277bc5c3c: ltp-cve 2025-09-26 16:23:25,771 - lava_callback - ERROR - Error processing callback for node 68d41aeca5c2c05277bc5c3c: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:47274 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47276 - "GET / HTTP/1.0" 200 OK 2025-09-26 16:23:29,515 - lava_callback - INFO - Received callback for node 68d41af4a5c2c05277bc5c45 INFO: 172.21.0.1:57398 - "POST /node/68d41af4a5c2c05277bc5c45 HTTP/1.0" 202 Accepted 2025-09-26 16:23:29,521 - lava_callback - INFO - Starting async job submit for node 68d41af4a5c2c05277bc5c45 2025-09-26 16:23:29,522 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:23:29,539 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d41af4a5c2c05277bc5c45 HTTP/1.1" 200 None 2025-09-26 16:23:29,539 - lava_callback - DEBUG - Processing node 68d41af4a5c2c05277bc5c45: ltp-fs-bind 2025-09-26 16:23:29,539 - lava_callback - ERROR - Error processing callback for node 68d41af4a5c2c05277bc5c45: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 16:23:33,293 - lava_callback - INFO - Received callback for node 68d41af7a5c2c05277bc5c4b INFO: 172.21.0.1:57404 - "POST /node/68d41af7a5c2c05277bc5c4b HTTP/1.0" 202 Accepted 2025-09-26 16:23:33,299 - lava_callback - INFO - Starting async job submit for node 68d41af7a5c2c05277bc5c4b 2025-09-26 16:23:33,300 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:23:33,312 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d41af7a5c2c05277bc5c4b HTTP/1.1" 200 None 2025-09-26 16:23:33,312 - lava_callback - DEBUG - Processing node 68d41af7a5c2c05277bc5c4b: ltp-fsx 2025-09-26 16:23:33,312 - lava_callback - ERROR - Error processing callback for node 68d41af7a5c2c05277bc5c4b: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 16:23:35,873 - lava_callback - INFO - Received callback for node 68d41afaa5c2c05277bc5c51 INFO: 172.21.0.1:57406 - "POST /node/68d41afaa5c2c05277bc5c51 HTTP/1.0" 202 Accepted 2025-09-26 16:23:35,879 - lava_callback - INFO - Starting async job submit for node 68d41afaa5c2c05277bc5c51 2025-09-26 16:23:35,880 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:23:35,892 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d41afaa5c2c05277bc5c51 HTTP/1.1" 200 None 2025-09-26 16:23:35,893 - lava_callback - DEBUG - Processing node 68d41afaa5c2c05277bc5c51: ltp-input 2025-09-26 16:23:35,893 - lava_callback - ERROR - Error processing callback for node 68d41afaa5c2c05277bc5c51: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:57408 - "GET / HTTP/1.0" 200 OK 2025-09-26 16:23:36,616 - lava_callback - INFO - Received callback for node 68d41afea5c2c05277bc5c56 INFO: 172.21.0.1:57418 - "POST /node/68d41afea5c2c05277bc5c56 HTTP/1.0" 202 Accepted 2025-09-26 16:23:36,622 - lava_callback - INFO - Starting async job submit for node 68d41afea5c2c05277bc5c56 2025-09-26 16:23:36,623 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:23:36,635 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d41afea5c2c05277bc5c56 HTTP/1.1" 200 None 2025-09-26 16:23:36,635 - lava_callback - DEBUG - Processing node 68d41afea5c2c05277bc5c56: ltp-ipc 2025-09-26 16:23:36,635 - lava_callback - ERROR - Error processing callback for node 68d41afea5c2c05277bc5c56: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 16:23:37,425 - lava_callback - INFO - Received callback for node 68d41b03a5c2c05277bc5c5c INFO: 172.21.0.1:57426 - "POST /node/68d41b03a5c2c05277bc5c5c HTTP/1.0" 202 Accepted 2025-09-26 16:23:37,430 - lava_callback - INFO - Starting async job submit for node 68d41b03a5c2c05277bc5c5c 2025-09-26 16:23:37,431 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:23:37,443 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d41b03a5c2c05277bc5c5c HTTP/1.1" 200 None 2025-09-26 16:23:37,444 - lava_callback - DEBUG - Processing node 68d41b03a5c2c05277bc5c5c: ltp-pty 2025-09-26 16:23:37,444 - lava_callback - ERROR - Error processing callback for node 68d41b03a5c2c05277bc5c5c: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 16:23:38,174 - lava_callback - INFO - Received callback for node 68d41b06a5c2c05277bc5c62 INFO: 172.21.0.1:57432 - "POST /node/68d41b06a5c2c05277bc5c62 HTTP/1.0" 202 Accepted 2025-09-26 16:23:38,179 - lava_callback - INFO - Starting async job submit for node 68d41b06a5c2c05277bc5c62 2025-09-26 16:23:38,180 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:23:38,192 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d41b06a5c2c05277bc5c62 HTTP/1.1" 200 None 2025-09-26 16:23:38,193 - lava_callback - DEBUG - Processing node 68d41b06a5c2c05277bc5c62: ltp-sched 2025-09-26 16:23:38,193 - lava_callback - ERROR - Error processing callback for node 68d41b06a5c2c05277bc5c62: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 16:23:38,972 - lava_callback - INFO - Received callback for node 68d41b09a5c2c05277bc5c66 INFO: 172.21.0.1:57442 - "POST /node/68d41b09a5c2c05277bc5c66 HTTP/1.0" 202 Accepted 2025-09-26 16:23:38,978 - lava_callback - INFO - Starting async job submit for node 68d41b09a5c2c05277bc5c66 2025-09-26 16:23:38,978 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:23:38,990 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d41b09a5c2c05277bc5c66 HTTP/1.1" 200 None 2025-09-26 16:23:38,991 - lava_callback - DEBUG - Processing node 68d41b09a5c2c05277bc5c66: ltp-smoketest 2025-09-26 16:23:38,991 - lava_callback - ERROR - Error processing callback for node 68d41b09a5c2c05277bc5c66: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 16:23:45,619 - lava_callback - INFO - Received callback for node 68d41b0ca5c2c05277bc5c6b INFO: 172.21.0.1:57380 - "POST /node/68d41b0ca5c2c05277bc5c6b HTTP/1.0" 202 Accepted 2025-09-26 16:23:45,625 - lava_callback - INFO - Starting async job submit for node 68d41b0ca5c2c05277bc5c6b 2025-09-26 16:23:45,626 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:23:45,638 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d41b0ca5c2c05277bc5c6b HTTP/1.1" 200 None 2025-09-26 16:23:45,638 - lava_callback - DEBUG - Processing node 68d41b0ca5c2c05277bc5c6b: ltp-syscalls 2025-09-26 16:23:45,638 - lava_callback - ERROR - Error processing callback for node 68d41b0ca5c2c05277bc5c6b: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:57390 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36834 - "GET / HTTP/1.0" 200 OK 2025-09-26 16:23:50,072 - lava_callback - INFO - Received callback for node 68d41687cd36afa834e18732 INFO: 172.21.0.1:36838 - "POST /node/68d41687cd36afa834e18732 HTTP/1.0" 202 Accepted 2025-09-26 16:23:50,078 - lava_callback - INFO - Starting async job submit for node 68d41687cd36afa834e18732 2025-09-26 16:23:50,108 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:23:50,142 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d41687cd36afa834e18732 HTTP/1.1" 200 None 2025-09-26 16:23:50,143 - lava_callback - DEBUG - Processing node 68d41687cd36afa834e18732: baseline-arm 2025-09-26 16:23:50,413 - lava_callback - ERROR - Error processing callback for node 68d41687cd36afa834e18732: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None 2025-09-26 16:23:54,940 - lava_callback - INFO - Received callback for node 68d6bdb23c94e462c688151a INFO: 172.21.0.1:36846 - "POST /node/68d6bdb23c94e462c688151a HTTP/1.0" 202 Accepted 2025-09-26 16:23:54,946 - lava_callback - INFO - Starting async job submit for node 68d6bdb23c94e462c688151a 2025-09-26 16:23:54,962 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:23:54,991 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6bdb23c94e462c688151a HTTP/1.1" 200 None 2025-09-26 16:23:54,991 - lava_callback - DEBUG - Processing node 68d6bdb23c94e462c688151a: baseline-x86 2025-09-26 16:23:55,277 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 16:23:55,420 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 16:23:55,421 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-68d6bdb23c94e462c688151a/log.txt.gz 2025-09-26 16:23:55,428 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 16:23:55,562 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 16:23:55,563 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-68d6bdb23c94e462c688151a/lava_callback.json.gz 2025-09-26 16:23:55,580 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:23:55,605 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6bdb23c94e462c688151a HTTP/1.1" 200 None 2025-09-26 16:23:55,606 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:23:55,618 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6b9f22f83206171a92b66 HTTP/1.1" 200 None 2025-09-26 16:23:55,619 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:23:55,645 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6bdb23c94e462c688151a HTTP/1.1" 200 None 2025-09-26 16:23:55,645 - lava_callback - INFO - Completed processing callback for node 68d6bdb23c94e462c688151a INFO: 172.21.0.1:36858 - "GET / HTTP/1.0" 200 OK 2025-09-26 16:23:57,139 - lava_callback - INFO - Received callback for node 68d46e27a5c2c05277bcc419 INFO: 172.21.0.1:36860 - "POST /node/68d46e27a5c2c05277bcc419 HTTP/1.0" 202 Accepted 2025-09-26 16:23:57,145 - lava_callback - INFO - Starting async job submit for node 68d46e27a5c2c05277bcc419 2025-09-26 16:23:57,167 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:23:57,178 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d46e27a5c2c05277bcc419 HTTP/1.1" 200 None 2025-09-26 16:23:57,179 - lava_callback - DEBUG - Processing node 68d46e27a5c2c05277bcc419: baseline-arm 2025-09-26 16:23:57,411 - lava_callback - ERROR - Error processing callback for node 68d46e27a5c2c05277bcc419: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None 2025-09-26 16:24:05,595 - lava_callback - INFO - Received callback for node 68d41b0fa5c2c05277bc5c71 INFO: 172.21.0.1:38038 - "POST /node/68d41b0fa5c2c05277bc5c71 HTTP/1.0" 202 Accepted 2025-09-26 16:24:05,601 - lava_callback - INFO - Starting async job submit for node 68d41b0fa5c2c05277bc5c71 2025-09-26 16:24:05,602 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:24:05,614 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d41b0fa5c2c05277bc5c71 HTTP/1.1" 200 None 2025-09-26 16:24:05,614 - lava_callback - DEBUG - Processing node 68d41b0fa5c2c05277bc5c71: ltp-syscalls-ipc 2025-09-26 16:24:05,614 - lava_callback - ERROR - Error processing callback for node 68d41b0fa5c2c05277bc5c71: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 16:24:06,383 - lava_callback - INFO - Received callback for node 68d41b15a5c2c05277bc5c77 INFO: 172.21.0.1:38040 - "POST /node/68d41b15a5c2c05277bc5c77 HTTP/1.0" 202 Accepted 2025-09-26 16:24:06,389 - lava_callback - INFO - Starting async job submit for node 68d41b15a5c2c05277bc5c77 2025-09-26 16:24:06,390 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:24:06,403 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d41b15a5c2c05277bc5c77 HTTP/1.1" 200 None 2025-09-26 16:24:06,403 - lava_callback - DEBUG - Processing node 68d41b15a5c2c05277bc5c77: ltp-timers 2025-09-26 16:24:06,403 - lava_callback - ERROR - Error processing callback for node 68d41b15a5c2c05277bc5c77: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:38052 - "GET / HTTP/1.0" 200 OK 2025-09-26 16:24:07,149 - lava_callback - INFO - Received callback for node 68d41b1aa5c2c05277bc5c9c INFO: 172.21.0.1:38054 - "POST /node/68d41b1aa5c2c05277bc5c9c HTTP/1.0" 202 Accepted 2025-09-26 16:24:07,154 - lava_callback - INFO - Starting async job submit for node 68d41b1aa5c2c05277bc5c9c 2025-09-26 16:24:07,155 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:24:07,167 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d41b1aa5c2c05277bc5c9c HTTP/1.1" 200 None 2025-09-26 16:24:07,168 - lava_callback - DEBUG - Processing node 68d41b1aa5c2c05277bc5c9c: ltp-watchqueue 2025-09-26 16:24:07,168 - lava_callback - ERROR - Error processing callback for node 68d41b1aa5c2c05277bc5c9c: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:42074 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42080 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58614 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58618 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60010 - "GET / HTTP/1.0" 200 OK 2025-09-26 16:24:45,675 - lava_callback - INFO - Received callback for node 68d41b40a5c2c05277bc5ccd INFO: 172.21.0.1:55688 - "POST /node/68d41b40a5c2c05277bc5ccd HTTP/1.0" 202 Accepted 2025-09-26 16:24:45,681 - lava_callback - INFO - Starting async job submit for node 68d41b40a5c2c05277bc5ccd 2025-09-26 16:24:45,682 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:24:45,694 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d41b40a5c2c05277bc5ccd HTTP/1.1" 200 None 2025-09-26 16:24:45,695 - lava_callback - DEBUG - Processing node 68d41b40a5c2c05277bc5ccd: baseline-arm-broonie 2025-09-26 16:24:45,695 - lava_callback - ERROR - Error processing callback for node 68d41b40a5c2c05277bc5ccd: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 16:24:46,487 - lava_callback - INFO - Received callback for node 68d41b41a5c2c05277bc5cd6 INFO: 172.21.0.1:55694 - "POST /node/68d41b41a5c2c05277bc5cd6 HTTP/1.0" 202 Accepted 2025-09-26 16:24:46,493 - lava_callback - INFO - Starting async job submit for node 68d41b41a5c2c05277bc5cd6 2025-09-26 16:24:46,494 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:24:46,520 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d41b41a5c2c05277bc5cd6 HTTP/1.1" 200 None 2025-09-26 16:24:46,520 - lava_callback - DEBUG - Processing node 68d41b41a5c2c05277bc5cd6: baseline-arm-broonie 2025-09-26 16:24:46,520 - lava_callback - ERROR - Error processing callback for node 68d41b41a5c2c05277bc5cd6: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:55704 - "GET / HTTP/1.0" 200 OK 2025-09-26 16:24:47,249 - lava_callback - INFO - Received callback for node 68d41b4da5c2c05277bc5cf4 INFO: 172.21.0.1:55716 - "POST /node/68d41b4da5c2c05277bc5cf4 HTTP/1.0" 202 Accepted 2025-09-26 16:24:47,255 - lava_callback - INFO - Starting async job submit for node 68d41b4da5c2c05277bc5cf4 2025-09-26 16:24:47,256 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:24:47,270 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d41b4da5c2c05277bc5cf4 HTTP/1.1" 200 None 2025-09-26 16:24:47,270 - lava_callback - DEBUG - Processing node 68d41b4da5c2c05277bc5cf4: ltp-cap-bounds 2025-09-26 16:24:47,270 - lava_callback - ERROR - Error processing callback for node 68d41b4da5c2c05277bc5cf4: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 16:24:48,269 - lava_callback - INFO - Received callback for node 68d41b4ea5c2c05277bc5cf7 INFO: 172.21.0.1:55718 - "POST /node/68d41b4ea5c2c05277bc5cf7 HTTP/1.0" 202 Accepted 2025-09-26 16:24:48,275 - lava_callback - INFO - Starting async job submit for node 68d41b4ea5c2c05277bc5cf7 2025-09-26 16:24:48,276 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:24:48,289 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d41b4ea5c2c05277bc5cf7 HTTP/1.1" 200 None 2025-09-26 16:24:48,289 - lava_callback - DEBUG - Processing node 68d41b4ea5c2c05277bc5cf7: ltp-containers 2025-09-26 16:24:48,289 - lava_callback - ERROR - Error processing callback for node 68d41b4ea5c2c05277bc5cf7: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 16:24:49,034 - lava_callback - INFO - Received callback for node 68d41b4fa5c2c05277bc5cfc INFO: 172.21.0.1:55722 - "POST /node/68d41b4fa5c2c05277bc5cfc HTTP/1.0" 202 Accepted 2025-09-26 16:24:49,040 - lava_callback - INFO - Starting async job submit for node 68d41b4fa5c2c05277bc5cfc 2025-09-26 16:24:49,041 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:24:49,053 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d41b4fa5c2c05277bc5cfc HTTP/1.1" 200 None 2025-09-26 16:24:49,054 - lava_callback - DEBUG - Processing node 68d41b4fa5c2c05277bc5cfc: ltp-controllers 2025-09-26 16:24:49,054 - lava_callback - ERROR - Error processing callback for node 68d41b4fa5c2c05277bc5cfc: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 16:24:49,565 - lava_callback - INFO - Received callback for node 68d41b3fa5c2c05277bc5cc7 INFO: 172.21.0.1:52474 - "POST /node/68d41b3fa5c2c05277bc5cc7 HTTP/1.0" 202 Accepted 2025-09-26 16:24:49,570 - lava_callback - INFO - Starting async job submit for node 68d41b3fa5c2c05277bc5cc7 2025-09-26 16:24:49,571 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:24:49,585 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d41b3fa5c2c05277bc5cc7 HTTP/1.1" 200 None 2025-09-26 16:24:49,585 - lava_callback - DEBUG - Processing node 68d41b3fa5c2c05277bc5cc7: baseline-arm 2025-09-26 16:24:49,585 - lava_callback - ERROR - Error processing callback for node 68d41b3fa5c2c05277bc5cc7: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:52490 - "GET / HTTP/1.0" 200 OK 2025-09-26 16:24:49,795 - lava_callback - INFO - Received callback for node 68d41b4fa5c2c05277bc5cff INFO: 172.21.0.1:52492 - "POST /node/68d41b4fa5c2c05277bc5cff HTTP/1.0" 202 Accepted 2025-09-26 16:24:49,801 - lava_callback - INFO - Starting async job submit for node 68d41b4fa5c2c05277bc5cff 2025-09-26 16:24:49,802 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:24:49,817 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d41b4fa5c2c05277bc5cff HTTP/1.1" 200 None 2025-09-26 16:24:49,817 - lava_callback - DEBUG - Processing node 68d41b4fa5c2c05277bc5cff: ltp-cve 2025-09-26 16:24:49,817 - lava_callback - ERROR - Error processing callback for node 68d41b4fa5c2c05277bc5cff: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 16:24:50,602 - lava_callback - INFO - Received callback for node 68d41b51a5c2c05277bc5d06 INFO: 172.21.0.1:52506 - "POST /node/68d41b51a5c2c05277bc5d06 HTTP/1.0" 202 Accepted 2025-09-26 16:24:50,608 - lava_callback - INFO - Starting async job submit for node 68d41b51a5c2c05277bc5d06 2025-09-26 16:24:50,609 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:24:50,625 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d41b51a5c2c05277bc5d06 HTTP/1.1" 200 None 2025-09-26 16:24:50,626 - lava_callback - DEBUG - Processing node 68d41b51a5c2c05277bc5d06: ltp-fs-bind 2025-09-26 16:24:50,626 - lava_callback - ERROR - Error processing callback for node 68d41b51a5c2c05277bc5d06: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 16:24:51,388 - lava_callback - INFO - Received callback for node 68d41b51a5c2c05277bc5d09 INFO: 172.21.0.1:52520 - "POST /node/68d41b51a5c2c05277bc5d09 HTTP/1.0" 202 Accepted 2025-09-26 16:24:51,394 - lava_callback - INFO - Starting async job submit for node 68d41b51a5c2c05277bc5d09 2025-09-26 16:24:51,395 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:24:51,429 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d41b51a5c2c05277bc5d09 HTTP/1.1" 200 None 2025-09-26 16:24:51,429 - lava_callback - DEBUG - Processing node 68d41b51a5c2c05277bc5d09: ltp-fsx 2025-09-26 16:24:51,429 - lava_callback - ERROR - Error processing callback for node 68d41b51a5c2c05277bc5d09: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 16:24:52,177 - lava_callback - INFO - Received callback for node 68d41b52a5c2c05277bc5d0e INFO: 172.21.0.1:52522 - "POST /node/68d41b52a5c2c05277bc5d0e HTTP/1.0" 202 Accepted 2025-09-26 16:24:52,183 - lava_callback - INFO - Starting async job submit for node 68d41b52a5c2c05277bc5d0e 2025-09-26 16:24:52,184 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:24:52,201 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d41b52a5c2c05277bc5d0e HTTP/1.1" 200 None 2025-09-26 16:24:52,201 - lava_callback - DEBUG - Processing node 68d41b52a5c2c05277bc5d0e: ltp-input 2025-09-26 16:24:52,201 - lava_callback - ERROR - Error processing callback for node 68d41b52a5c2c05277bc5d0e: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 16:24:52,950 - lava_callback - INFO - Received callback for node 68d41b52a5c2c05277bc5d12 INFO: 172.21.0.1:52528 - "POST /node/68d41b52a5c2c05277bc5d12 HTTP/1.0" 202 Accepted 2025-09-26 16:24:52,956 - lava_callback - INFO - Starting async job submit for node 68d41b52a5c2c05277bc5d12 2025-09-26 16:24:52,957 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:24:52,970 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d41b52a5c2c05277bc5d12 HTTP/1.1" 200 None 2025-09-26 16:24:52,970 - lava_callback - DEBUG - Processing node 68d41b52a5c2c05277bc5d12: ltp-ipc 2025-09-26 16:24:52,970 - lava_callback - ERROR - Error processing callback for node 68d41b52a5c2c05277bc5d12: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 16:24:53,706 - lava_callback - INFO - Received callback for node 68d41b53a5c2c05277bc5d15 INFO: 172.21.0.1:52544 - "POST /node/68d41b53a5c2c05277bc5d15 HTTP/1.0" 202 Accepted 2025-09-26 16:24:53,714 - lava_callback - INFO - Starting async job submit for node 68d41b53a5c2c05277bc5d15 2025-09-26 16:24:53,715 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:24:53,733 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d41b53a5c2c05277bc5d15 HTTP/1.1" 200 None 2025-09-26 16:24:53,733 - lava_callback - DEBUG - Processing node 68d41b53a5c2c05277bc5d15: ltp-pty 2025-09-26 16:24:53,733 - lava_callback - ERROR - Error processing callback for node 68d41b53a5c2c05277bc5d15: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:52558 - "GET / HTTP/1.0" 200 OK 2025-09-26 16:25:05,734 - lava_callback - INFO - Received callback for node 68d41b53a5c2c05277bc5d18 INFO: 172.21.0.1:53390 - "POST /node/68d41b53a5c2c05277bc5d18 HTTP/1.0" 202 Accepted 2025-09-26 16:25:05,740 - lava_callback - INFO - Starting async job submit for node 68d41b53a5c2c05277bc5d18 2025-09-26 16:25:05,741 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:25:05,907 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d41b53a5c2c05277bc5d18 HTTP/1.1" 200 None 2025-09-26 16:25:05,907 - lava_callback - DEBUG - Processing node 68d41b53a5c2c05277bc5d18: ltp-sched 2025-09-26 16:25:05,907 - lava_callback - ERROR - Error processing callback for node 68d41b53a5c2c05277bc5d18: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 16:25:06,483 - lava_callback - INFO - Received callback for node 68d41b57a5c2c05277bc5d3d INFO: 172.21.0.1:53404 - "POST /node/68d41b57a5c2c05277bc5d3d HTTP/1.0" 202 Accepted 2025-09-26 16:25:06,489 - lava_callback - INFO - Starting async job submit for node 68d41b57a5c2c05277bc5d3d 2025-09-26 16:25:06,490 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:25:06,503 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d41b57a5c2c05277bc5d3d HTTP/1.1" 200 None 2025-09-26 16:25:06,503 - lava_callback - DEBUG - Processing node 68d41b57a5c2c05277bc5d3d: ltp-smoketest 2025-09-26 16:25:06,503 - lava_callback - ERROR - Error processing callback for node 68d41b57a5c2c05277bc5d3d: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:53420 - "GET / HTTP/1.0" 200 OK 2025-09-26 16:25:07,277 - lava_callback - INFO - Received callback for node 68d41b58a5c2c05277bc5d40 INFO: 172.21.0.1:53428 - "POST /node/68d41b58a5c2c05277bc5d40 HTTP/1.0" 202 Accepted 2025-09-26 16:25:07,283 - lava_callback - INFO - Starting async job submit for node 68d41b58a5c2c05277bc5d40 2025-09-26 16:25:07,284 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:25:07,297 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d41b58a5c2c05277bc5d40 HTTP/1.1" 200 None 2025-09-26 16:25:07,297 - lava_callback - DEBUG - Processing node 68d41b58a5c2c05277bc5d40: ltp-syscalls 2025-09-26 16:25:07,297 - lava_callback - ERROR - Error processing callback for node 68d41b58a5c2c05277bc5d40: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 16:25:08,101 - lava_callback - INFO - Received callback for node 68d41b59a5c2c05277bc5d43 INFO: 172.21.0.1:53436 - "POST /node/68d41b59a5c2c05277bc5d43 HTTP/1.0" 202 Accepted 2025-09-26 16:25:08,107 - lava_callback - INFO - Starting async job submit for node 68d41b59a5c2c05277bc5d43 2025-09-26 16:25:08,108 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:25:08,120 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d41b59a5c2c05277bc5d43 HTTP/1.1" 200 None 2025-09-26 16:25:08,121 - lava_callback - DEBUG - Processing node 68d41b59a5c2c05277bc5d43: ltp-syscalls-ipc 2025-09-26 16:25:08,121 - lava_callback - ERROR - Error processing callback for node 68d41b59a5c2c05277bc5d43: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 16:25:08,851 - lava_callback - INFO - Received callback for node 68d41b59a5c2c05277bc5d46 INFO: 172.21.0.1:53446 - "POST /node/68d41b59a5c2c05277bc5d46 HTTP/1.0" 202 Accepted 2025-09-26 16:25:08,857 - lava_callback - INFO - Starting async job submit for node 68d41b59a5c2c05277bc5d46 2025-09-26 16:25:08,858 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:25:08,873 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d41b59a5c2c05277bc5d46 HTTP/1.1" 200 None 2025-09-26 16:25:08,874 - lava_callback - DEBUG - Processing node 68d41b59a5c2c05277bc5d46: ltp-timers 2025-09-26 16:25:08,874 - lava_callback - ERROR - Error processing callback for node 68d41b59a5c2c05277bc5d46: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 16:25:09,619 - lava_callback - INFO - Received callback for node 68d41b5aa5c2c05277bc5d4b INFO: 172.21.0.1:60870 - "POST /node/68d41b5aa5c2c05277bc5d4b HTTP/1.0" 202 Accepted 2025-09-26 16:25:09,625 - lava_callback - INFO - Starting async job submit for node 68d41b5aa5c2c05277bc5d4b 2025-09-26 16:25:09,626 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:25:09,639 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d41b5aa5c2c05277bc5d4b HTTP/1.1" 200 None 2025-09-26 16:25:09,639 - lava_callback - DEBUG - Processing node 68d41b5aa5c2c05277bc5d4b: ltp-watchqueue 2025-09-26 16:25:09,639 - lava_callback - ERROR - Error processing callback for node 68d41b5aa5c2c05277bc5d4b: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:60882 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60886 - "GET / HTTP/1.0" 200 OK 2025-09-26 16:25:21,813 - lava_callback - INFO - Received callback for node 68d6b99e2f83206171a92b2c INFO: 172.21.0.1:55780 - "POST /node/68d6b99e2f83206171a92b2c HTTP/1.0" 202 Accepted 2025-09-26 16:25:21,819 - lava_callback - INFO - Starting async job submit for node 68d6b99e2f83206171a92b2c 2025-09-26 16:25:21,849 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:25:21,865 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6b99e2f83206171a92b2c HTTP/1.1" 200 None 2025-09-26 16:25:21,865 - lava_callback - DEBUG - Processing node 68d6b99e2f83206171a92b2c: baseline-x86-kcidebug-intel 2025-09-26 16:25:22,110 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 16:25:22,265 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 16:25:22,266 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-kcidebug-intel-68d6b99e2f83206171a92b2c/log.txt.gz 2025-09-26 16:25:22,272 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 16:25:22,430 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 16:25:22,431 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-kcidebug-intel-68d6b99e2f83206171a92b2c/lava_callback.json.gz DEBUG: Setup failed for 68d6b99e2f83206171a92b2c. Transitting job node result: incomplete -> fail 2025-09-26 16:25:22,491 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:25:22,503 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6b99e2f83206171a92b2c HTTP/1.1" 200 None 2025-09-26 16:25:22,504 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:25:22,516 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d649095d233c363a762369 HTTP/1.1" 200 None 2025-09-26 16:25:22,516 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:25:22,541 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6b99e2f83206171a92b2c HTTP/1.1" 200 None 2025-09-26 16:25:22,542 - lava_callback - INFO - Completed processing callback for node 68d6b99e2f83206171a92b2c INFO: 172.21.0.1:55792 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55804 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60050 - "GET / HTTP/1.0" 200 OK 2025-09-26 16:25:38,837 - lava_callback - INFO - Received callback for node 68d6bda13c94e462c6881501 INFO: 172.21.0.1:60056 - "POST /node/68d6bda13c94e462c6881501 HTTP/1.0" 202 Accepted 2025-09-26 16:25:38,843 - lava_callback - INFO - Starting async job submit for node 68d6bda13c94e462c6881501 2025-09-26 16:25:38,862 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:25:38,878 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6bda13c94e462c6881501 HTTP/1.1" 200 None 2025-09-26 16:25:38,878 - lava_callback - DEBUG - Processing node 68d6bda13c94e462c6881501: baseline-x86 2025-09-26 16:25:39,168 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 16:25:39,314 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 16:25:39,315 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-68d6bda13c94e462c6881501/log.txt.gz 2025-09-26 16:25:39,325 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 16:25:39,446 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 16:25:39,447 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-68d6bda13c94e462c6881501/lava_callback.json.gz 2025-09-26 16:25:39,466 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:25:39,478 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6bda13c94e462c6881501 HTTP/1.1" 200 None 2025-09-26 16:25:39,479 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:25:39,490 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6b9f22f83206171a92b66 HTTP/1.1" 200 None 2025-09-26 16:25:39,491 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:25:39,518 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6bda13c94e462c6881501 HTTP/1.1" 200 None 2025-09-26 16:25:39,518 - lava_callback - INFO - Completed processing callback for node 68d6bda13c94e462c6881501 INFO: 172.21.0.1:45666 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60250 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60258 - "GET / HTTP/1.0" 200 OK 2025-09-26 16:26:02,324 - lava_callback - INFO - Received callback for node 68d46e39a5c2c05277bcc42e INFO: 172.21.0.1:60944 - "POST /node/68d46e39a5c2c05277bcc42e HTTP/1.0" 202 Accepted 2025-09-26 16:26:02,330 - lava_callback - INFO - Starting async job submit for node 68d46e39a5c2c05277bcc42e 2025-09-26 16:26:02,351 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:26:02,363 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d46e39a5c2c05277bcc42e HTTP/1.1" 200 None 2025-09-26 16:26:02,363 - lava_callback - DEBUG - Processing node 68d46e39a5c2c05277bcc42e: baseline-arm 2025-09-26 16:26:02,596 - lava_callback - ERROR - Error processing callback for node 68d46e39a5c2c05277bcc42e: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:60954 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40524 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40532 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38186 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38196 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34310 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53838 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55142 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55156 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33874 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59792 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59794 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57326 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57332 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54830 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48908 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45210 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45222 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40922 - "GET / HTTP/1.0" 200 OK 2025-09-26 16:28:06,739 - lava_callback - INFO - Received callback for node 68d46e39a5c2c05277bcc431 INFO: 172.21.0.1:40938 - "POST /node/68d46e39a5c2c05277bcc431 HTTP/1.0" 202 Accepted 2025-09-26 16:28:06,745 - lava_callback - INFO - Starting async job submit for node 68d46e39a5c2c05277bcc431 2025-09-26 16:28:06,767 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:28:06,779 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d46e39a5c2c05277bcc431 HTTP/1.1" 200 None 2025-09-26 16:28:06,780 - lava_callback - DEBUG - Processing node 68d46e39a5c2c05277bcc431: baseline-arm 2025-09-26 16:28:07,022 - lava_callback - ERROR - Error processing callback for node 68d46e39a5c2c05277bcc431: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None 2025-09-26 16:28:08,119 - lava_callback - INFO - Received callback for node 68d6bea83c94e462c6881537 INFO: 172.21.0.1:40950 - "POST /node/68d6bea83c94e462c6881537 HTTP/1.0" 202 Accepted 2025-09-26 16:28:08,125 - lava_callback - INFO - Starting async job submit for node 68d6bea83c94e462c6881537 2025-09-26 16:28:08,146 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:28:08,167 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6bea83c94e462c6881537 HTTP/1.1" 200 None 2025-09-26 16:28:08,167 - lava_callback - DEBUG - Processing node 68d6bea83c94e462c6881537: baseline-riscv-collabora 2025-09-26 16:28:08,385 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 16:28:08,542 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 16:28:08,543 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-riscv-collabora-68d6bea83c94e462c6881537/log.txt.gz 2025-09-26 16:28:08,547 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 16:28:08,682 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 16:28:08,682 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-riscv-collabora-68d6bea83c94e462c6881537/lava_callback.json.gz 2025-09-26 16:28:08,703 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:28:08,718 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6bea83c94e462c6881537 HTTP/1.1" 200 None 2025-09-26 16:28:08,720 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:28:08,733 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6b9ef2f83206171a92b63 HTTP/1.1" 200 None 2025-09-26 16:28:08,734 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:28:08,763 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6bea83c94e462c6881537 HTTP/1.1" 200 None 2025-09-26 16:28:08,764 - lava_callback - INFO - Completed processing callback for node 68d6bea83c94e462c6881537 INFO: 172.21.0.1:56914 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56928 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34154 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34168 - "GET / HTTP/1.0" 200 OK 2025-09-26 16:28:27,623 - lava_callback - INFO - Received callback for node 68d6bea63c94e462c6881534 INFO: 172.21.0.1:34184 - "POST /node/68d6bea63c94e462c6881534 HTTP/1.0" 202 Accepted 2025-09-26 16:28:27,629 - lava_callback - INFO - Starting async job submit for node 68d6bea63c94e462c6881534 2025-09-26 16:28:27,651 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:28:27,663 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6bea63c94e462c6881534 HTTP/1.1" 200 None 2025-09-26 16:28:27,663 - lava_callback - DEBUG - Processing node 68d6bea63c94e462c6881534: baseline-riscv-broonie 2025-09-26 16:28:27,905 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 16:28:28,031 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 16:28:28,032 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-riscv-broonie-68d6bea63c94e462c6881534/log.txt.gz 2025-09-26 16:28:28,038 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 16:28:28,151 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 16:28:28,152 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-riscv-broonie-68d6bea63c94e462c6881534/lava_callback.json.gz 2025-09-26 16:28:28,174 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:28:28,185 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6bea63c94e462c6881534 HTTP/1.1" 200 None 2025-09-26 16:28:28,186 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:28:28,204 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6b9ef2f83206171a92b63 HTTP/1.1" 200 None 2025-09-26 16:28:28,206 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:28:28,234 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6bea63c94e462c6881534 HTTP/1.1" 200 None 2025-09-26 16:28:28,235 - lava_callback - INFO - Completed processing callback for node 68d6bea63c94e462c6881534 INFO: 172.21.0.1:40958 - "GET / HTTP/1.0" 200 OK 2025-09-26 16:28:45,612 - lava_callback - INFO - Received callback for node 68d6bf083c94e462c6881575 INFO: 172.21.0.1:41614 - "POST /node/68d6bf083c94e462c6881575 HTTP/1.0" 202 Accepted 2025-09-26 16:28:45,659 - lava_callback - INFO - Starting async job submit for node 68d6bf083c94e462c6881575 2025-09-26 16:28:45,678 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:28:45,716 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6bf083c94e462c6881575 HTTP/1.1" 200 None 2025-09-26 16:28:45,717 - lava_callback - DEBUG - Processing node 68d6bf083c94e462c6881575: baseline-arm-pengutronix 2025-09-26 16:28:45,825 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 16:28:45,989 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 16:28:45,989 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-68d6bf083c94e462c6881575/log.txt.gz 2025-09-26 16:28:45,993 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 16:28:46,121 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 16:28:46,122 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-68d6bf083c94e462c6881575/lava_callback.json.gz 2025-09-26 16:28:46,141 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:28:46,159 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6bf083c94e462c6881575 HTTP/1.1" 200 None 2025-09-26 16:28:46,160 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:28:46,172 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6b9da2f83206171a92b45 HTTP/1.1" 200 None 2025-09-26 16:28:46,173 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:28:46,222 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6bf083c94e462c6881575 HTTP/1.1" 200 None 2025-09-26 16:28:46,223 - lava_callback - INFO - Completed processing callback for node 68d6bf083c94e462c6881575 INFO: 172.21.0.1:41622 - "GET / HTTP/1.0" 200 OK 2025-09-26 16:28:50,745 - lava_callback - INFO - Received callback for node 68d6bf003c94e462c6881551 INFO: 172.21.0.1:53056 - "POST /node/68d6bf003c94e462c6881551 HTTP/1.0" 202 Accepted 2025-09-26 16:28:50,754 - lava_callback - INFO - Starting async job submit for node 68d6bf003c94e462c6881551 2025-09-26 16:28:50,775 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:28:50,788 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6bf003c94e462c6881551 HTTP/1.1" 200 None 2025-09-26 16:28:50,788 - lava_callback - DEBUG - Processing node 68d6bf003c94e462c6881551: baseline-arm INFO: 172.21.0.1:53060 - "GET / HTTP/1.0" 200 OK 2025-09-26 16:28:51,094 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 16:28:51,235 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 16:28:51,235 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-68d6bf003c94e462c6881551/log.txt.gz 2025-09-26 16:28:51,243 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 16:28:51,379 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 16:28:51,380 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-68d6bf003c94e462c6881551/lava_callback.json.gz 2025-09-26 16:28:51,399 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:28:51,412 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6bf003c94e462c6881551 HTTP/1.1" 200 None 2025-09-26 16:28:51,413 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:28:51,424 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6b9da2f83206171a92b45 HTTP/1.1" 200 None 2025-09-26 16:28:51,425 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:28:51,459 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6bf003c94e462c6881551 HTTP/1.1" 200 None 2025-09-26 16:28:51,460 - lava_callback - INFO - Completed processing callback for node 68d6bf003c94e462c6881551 2025-09-26 16:28:55,276 - lava_callback - INFO - Received callback for node 68d6beff3c94e462c6881545 INFO: 172.21.0.1:53064 - "POST /node/68d6beff3c94e462c6881545 HTTP/1.0" 202 Accepted 2025-09-26 16:28:55,282 - lava_callback - INFO - Starting async job submit for node 68d6beff3c94e462c6881545 2025-09-26 16:28:55,301 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:28:55,316 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6beff3c94e462c6881545 HTTP/1.1" 200 None 2025-09-26 16:28:55,317 - lava_callback - DEBUG - Processing node 68d6beff3c94e462c6881545: baseline-arm 2025-09-26 16:28:55,645 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 16:28:55,779 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 16:28:55,779 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-68d6beff3c94e462c6881545/log.txt.gz 2025-09-26 16:28:55,787 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 16:28:55,906 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 16:28:55,906 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-68d6beff3c94e462c6881545/lava_callback.json.gz 2025-09-26 16:28:55,926 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:28:55,942 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6beff3c94e462c6881545 HTTP/1.1" 200 None 2025-09-26 16:28:55,943 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:28:55,954 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6b9da2f83206171a92b45 HTTP/1.1" 200 None 2025-09-26 16:28:55,955 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:28:55,988 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6beff3c94e462c6881545 HTTP/1.1" 200 None 2025-09-26 16:28:55,989 - lava_callback - INFO - Completed processing callback for node 68d6beff3c94e462c6881545 INFO: 172.21.0.1:53076 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42944 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59686 - "GET /api/metrics HTTP/1.0" 200 OK 2025-09-26 16:29:15,778 - lava_callback - INFO - Received callback for node 68d6beff3c94e462c6881548 INFO: 172.21.0.1:59688 - "POST /node/68d6beff3c94e462c6881548 HTTP/1.0" 202 Accepted 2025-09-26 16:29:15,784 - lava_callback - INFO - Starting async job submit for node 68d6beff3c94e462c6881548 2025-09-26 16:29:15,804 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:29:15,825 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6beff3c94e462c6881548 HTTP/1.1" 200 None 2025-09-26 16:29:15,825 - lava_callback - DEBUG - Processing node 68d6beff3c94e462c6881548: baseline-arm 2025-09-26 16:29:16,039 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 16:29:16,148 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 16:29:16,149 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-68d6beff3c94e462c6881548/log.txt.gz 2025-09-26 16:29:16,155 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 16:29:16,268 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 16:29:16,269 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-68d6beff3c94e462c6881548/lava_callback.json.gz 2025-09-26 16:29:16,290 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:29:16,292 - lava_callback - INFO - Received callback for node 68d6befd3c94e462c688153c 2025-09-26 16:29:16,298 - lava_callback - INFO - Starting async job submit for node 68d6befd3c94e462c688153c INFO: 172.21.0.1:59700 - "POST /node/68d6befd3c94e462c688153c HTTP/1.0" 202 Accepted 2025-09-26 16:29:16,320 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:29:16,331 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6beff3c94e462c6881548 HTTP/1.1" 200 None 2025-09-26 16:29:16,332 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:29:16,344 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6befd3c94e462c688153c HTTP/1.1" 200 None 2025-09-26 16:29:16,345 - lava_callback - DEBUG - Processing node 68d6befd3c94e462c688153c: baseline-arm 2025-09-26 16:29:16,356 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6b9da2f83206171a92b45 HTTP/1.1" 200 None 2025-09-26 16:29:16,372 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:29:16,479 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6beff3c94e462c6881548 HTTP/1.1" 200 None 2025-09-26 16:29:16,480 - lava_callback - INFO - Completed processing callback for node 68d6beff3c94e462c6881548 INFO: 172.21.0.1:59712 - "GET / HTTP/1.0" 200 OK 2025-09-26 16:29:16,622 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 16:29:16,752 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 16:29:16,753 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-68d6befd3c94e462c688153c/log.txt.gz 2025-09-26 16:29:16,758 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 16:29:16,881 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 16:29:16,882 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-68d6befd3c94e462c688153c/lava_callback.json.gz 2025-09-26 16:29:16,903 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:29:16,915 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6befd3c94e462c688153c HTTP/1.1" 200 None 2025-09-26 16:29:16,916 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:29:16,927 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6b9da2f83206171a92b45 HTTP/1.1" 200 None 2025-09-26 16:29:16,928 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:29:16,956 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6befd3c94e462c688153c HTTP/1.1" 200 None 2025-09-26 16:29:16,957 - lava_callback - INFO - Completed processing callback for node 68d6befd3c94e462c688153c INFO: 172.21.0.1:35974 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35982 - "GET / HTTP/1.0" 200 OK 2025-09-26 16:29:28,230 - lava_callback - INFO - Received callback for node 68d6bf023c94e462c688155a INFO: 172.21.0.1:35988 - "POST /node/68d6bf023c94e462c688155a HTTP/1.0" 202 Accepted 2025-09-26 16:29:28,236 - lava_callback - INFO - Starting async job submit for node 68d6bf023c94e462c688155a 2025-09-26 16:29:28,258 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:29:28,271 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6bf023c94e462c688155a HTTP/1.1" 200 None 2025-09-26 16:29:28,271 - lava_callback - DEBUG - Processing node 68d6bf023c94e462c688155a: baseline-arm-broonie 2025-09-26 16:29:28,472 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 16:29:28,604 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 16:29:28,605 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-68d6bf023c94e462c688155a/log.txt.gz 2025-09-26 16:29:28,610 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 16:29:28,734 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 16:29:28,734 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-68d6bf023c94e462c688155a/lava_callback.json.gz 2025-09-26 16:29:28,774 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:29:28,796 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6bf023c94e462c688155a HTTP/1.1" 200 None 2025-09-26 16:29:28,796 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:29:28,808 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6b9da2f83206171a92b45 HTTP/1.1" 200 None 2025-09-26 16:29:28,809 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:29:28,841 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6bf023c94e462c688155a HTTP/1.1" 200 None 2025-09-26 16:29:28,842 - lava_callback - INFO - Completed processing callback for node 68d6bf023c94e462c688155a 2025-09-26 16:29:35,898 - lava_callback - INFO - Received callback for node 68d6bf053c94e462c6881569 INFO: 172.21.0.1:36592 - "POST /node/68d6bf053c94e462c6881569 HTTP/1.0" 202 Accepted 2025-09-26 16:29:35,904 - lava_callback - INFO - Starting async job submit for node 68d6bf053c94e462c6881569 2025-09-26 16:29:35,927 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:29:35,940 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6bf053c94e462c6881569 HTTP/1.1" 200 None 2025-09-26 16:29:35,941 - lava_callback - DEBUG - Processing node 68d6bf053c94e462c6881569: baseline-arm-kontron 2025-09-26 16:29:36,110 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 16:29:36,217 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 16:29:36,218 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-kontron-68d6bf053c94e462c6881569/log.txt.gz 2025-09-26 16:29:36,222 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 16:29:36,338 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 16:29:36,339 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-kontron-68d6bf053c94e462c6881569/lava_callback.json.gz 2025-09-26 16:29:36,362 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:29:36,373 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6bf053c94e462c6881569 HTTP/1.1" 200 None 2025-09-26 16:29:36,374 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:29:36,385 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6b9da2f83206171a92b45 HTTP/1.1" 200 None 2025-09-26 16:29:36,386 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:29:36,411 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6bf053c94e462c6881569 HTTP/1.1" 200 None 2025-09-26 16:29:36,412 - lava_callback - INFO - Completed processing callback for node 68d6bf053c94e462c6881569 INFO: 172.21.0.1:36608 - "GET / HTTP/1.0" 200 OK 2025-09-26 16:29:36,818 - lava_callback - INFO - Received callback for node 68d6bf033c94e462c688155d INFO: 172.21.0.1:36618 - "POST /node/68d6bf033c94e462c688155d HTTP/1.0" 202 Accepted 2025-09-26 16:29:36,824 - lava_callback - INFO - Starting async job submit for node 68d6bf033c94e462c688155d 2025-09-26 16:29:36,847 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:29:36,860 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6bf033c94e462c688155d HTTP/1.1" 200 None 2025-09-26 16:29:36,860 - lava_callback - DEBUG - Processing node 68d6bf033c94e462c688155d: baseline-arm-broonie 2025-09-26 16:29:37,084 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 16:29:37,110 - lava_callback - INFO - Received callback for node 68d6bf053c94e462c688156c INFO: 172.21.0.1:36622 - "POST /node/68d6bf053c94e462c688156c HTTP/1.0" 202 Accepted 2025-09-26 16:29:37,116 - lava_callback - INFO - Starting async job submit for node 68d6bf053c94e462c688156c 2025-09-26 16:29:37,152 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:29:37,164 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6bf053c94e462c688156c HTTP/1.1" 200 None 2025-09-26 16:29:37,165 - lava_callback - DEBUG - Processing node 68d6bf053c94e462c688156c: baseline-arm-kontron 2025-09-26 16:29:37,231 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 16:29:37,272 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-68d6bf033c94e462c688155d/log.txt.gz 2025-09-26 16:29:37,339 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 16:29:37,340 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 16:29:37,468 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 16:29:37,469 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-68d6bf033c94e462c688155d/lava_callback.json.gz 2025-09-26 16:29:37,485 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 16:29:37,493 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:29:37,493 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-kontron-68d6bf053c94e462c688156c/log.txt.gz 2025-09-26 16:29:37,498 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 16:29:37,510 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6bf033c94e462c688155d HTTP/1.1" 200 None 2025-09-26 16:29:37,511 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:29:37,523 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6b9da2f83206171a92b45 HTTP/1.1" 200 None 2025-09-26 16:29:37,524 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:29:37,557 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6bf033c94e462c688155d HTTP/1.1" 200 None 2025-09-26 16:29:37,558 - lava_callback - INFO - Completed processing callback for node 68d6bf033c94e462c688155d 2025-09-26 16:29:37,633 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 16:29:37,634 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-kontron-68d6bf053c94e462c688156c/lava_callback.json.gz 2025-09-26 16:29:37,658 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:29:37,675 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6bf053c94e462c688156c HTTP/1.1" 200 None 2025-09-26 16:29:37,676 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:29:37,689 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6b9da2f83206171a92b45 HTTP/1.1" 200 None 2025-09-26 16:29:37,690 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:29:37,719 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6bf053c94e462c688156c HTTP/1.1" 200 None 2025-09-26 16:29:37,719 - lava_callback - INFO - Completed processing callback for node 68d6bf053c94e462c688156c INFO: 172.21.0.1:45052 - "GET / HTTP/1.0" 200 OK 2025-09-26 16:29:50,411 - lava_callback - INFO - Received callback for node 68d6bf063c94e462c688156f INFO: 172.21.0.1:59762 - "POST /node/68d6bf063c94e462c688156f HTTP/1.0" 202 Accepted 2025-09-26 16:29:50,419 - lava_callback - INFO - Starting async job submit for node 68d6bf063c94e462c688156f 2025-09-26 16:29:50,457 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:29:50,486 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6bf063c94e462c688156f HTTP/1.1" 200 None 2025-09-26 16:29:50,487 - lava_callback - DEBUG - Processing node 68d6bf063c94e462c688156f: baseline-arm-pengutronix 2025-09-26 16:29:50,694 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 16:29:50,862 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 16:29:50,863 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-68d6bf063c94e462c688156f/log.txt.gz 2025-09-26 16:29:50,869 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 16:29:51,046 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 16:29:51,047 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-68d6bf063c94e462c688156f/lava_callback.json.gz 2025-09-26 16:29:51,152 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:29:51,167 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6bf063c94e462c688156f HTTP/1.1" 200 None 2025-09-26 16:29:51,168 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:29:51,208 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6b9da2f83206171a92b45 HTTP/1.1" 200 None 2025-09-26 16:29:51,209 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:29:51,266 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6bf063c94e462c688156f HTTP/1.1" 200 None 2025-09-26 16:29:51,266 - lava_callback - INFO - Completed processing callback for node 68d6bf063c94e462c688156f INFO: 172.21.0.1:59774 - "GET / HTTP/1.0" 200 OK 2025-09-26 16:29:52,166 - lava_callback - INFO - Received callback for node 68d6bf073c94e462c6881572 INFO: 172.21.0.1:59786 - "POST /node/68d6bf073c94e462c6881572 HTTP/1.0" 202 Accepted 2025-09-26 16:29:52,176 - lava_callback - INFO - Starting async job submit for node 68d6bf073c94e462c6881572 2025-09-26 16:29:52,217 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:29:52,245 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6bf073c94e462c6881572 HTTP/1.1" 200 None 2025-09-26 16:29:52,246 - lava_callback - DEBUG - Processing node 68d6bf073c94e462c6881572: baseline-arm-pengutronix 2025-09-26 16:29:52,580 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 16:29:52,900 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 16:29:52,901 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-68d6bf073c94e462c6881572/log.txt.gz 2025-09-26 16:29:52,907 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 16:29:53,065 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 16:29:53,066 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-68d6bf073c94e462c6881572/lava_callback.json.gz 2025-09-26 16:29:53,115 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:29:53,136 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6bf073c94e462c6881572 HTTP/1.1" 200 None 2025-09-26 16:29:53,137 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:29:53,169 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6b9da2f83206171a92b45 HTTP/1.1" 200 None 2025-09-26 16:29:53,170 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:29:53,198 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6bf073c94e462c6881572 HTTP/1.1" 200 None 2025-09-26 16:29:53,199 - lava_callback - INFO - Completed processing callback for node 68d6bf073c94e462c6881572 INFO: 172.21.0.1:59794 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33778 - "GET / HTTP/1.0" 200 OK 2025-09-26 16:30:09,240 - lava_callback - INFO - Received callback for node 68d46efca5c2c05277bcc515 INFO: 172.21.0.1:33786 - "POST /node/68d46efca5c2c05277bcc515 HTTP/1.0" 202 Accepted 2025-09-26 16:30:09,247 - lava_callback - INFO - Starting async job submit for node 68d46efca5c2c05277bcc515 2025-09-26 16:30:09,280 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:30:09,293 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d46efca5c2c05277bcc515 HTTP/1.1" 200 None 2025-09-26 16:30:09,294 - lava_callback - DEBUG - Processing node 68d46efca5c2c05277bcc515: baseline-arm 2025-09-26 16:30:09,521 - lava_callback - ERROR - Error processing callback for node 68d46efca5c2c05277bcc515: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:41088 - "GET /api/metrics HTTP/1.0" 200 OK 2025-09-26 16:30:15,511 - lava_callback - INFO - Received callback for node 68d41ae6a5c2c05277bc5c33 INFO: 172.21.0.1:41090 - "POST /node/68d41ae6a5c2c05277bc5c33 HTTP/1.0" 202 Accepted 2025-09-26 16:30:15,519 - lava_callback - INFO - Starting async job submit for node 68d41ae6a5c2c05277bc5c33 2025-09-26 16:30:15,620 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:30:15,633 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d41ae6a5c2c05277bc5c33 HTTP/1.1" 200 None 2025-09-26 16:30:15,634 - lava_callback - DEBUG - Processing node 68d41ae6a5c2c05277bc5c33: ltp-containers 2025-09-26 16:30:16,436 - lava_callback - ERROR - Error processing callback for node 68d41ae6a5c2c05277bc5c33: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:41094 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43752 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43760 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45150 - "GET / HTTP/1.0" 200 OK 2025-09-26 16:30:45,535 - lava_callback - INFO - Received callback for node 68d6bf803c94e462c68815d8 INFO: 172.21.0.1:43338 - "POST /node/68d6bf803c94e462c68815d8 HTTP/1.0" 202 Accepted 2025-09-26 16:30:45,541 - lava_callback - INFO - Starting async job submit for node 68d6bf803c94e462c68815d8 2025-09-26 16:30:45,569 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:30:45,584 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6bf803c94e462c68815d8 HTTP/1.1" 200 None 2025-09-26 16:30:45,585 - lava_callback - DEBUG - Processing node 68d6bf803c94e462c68815d8: baseline-arm-pengutronix 2025-09-26 16:30:45,742 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 16:30:45,907 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 16:30:45,908 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-68d6bf803c94e462c68815d8/log.txt.gz 2025-09-26 16:30:45,913 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 16:30:46,059 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 16:30:46,059 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-68d6bf803c94e462c68815d8/lava_callback.json.gz 2025-09-26 16:30:46,079 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:30:46,097 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6bf803c94e462c68815d8 HTTP/1.1" 200 None 2025-09-26 16:30:46,098 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:30:46,109 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6b9da2f83206171a92b45 HTTP/1.1" 200 None 2025-09-26 16:30:46,110 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:30:46,142 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6bf803c94e462c68815d8 HTTP/1.1" 200 None 2025-09-26 16:30:46,143 - lava_callback - INFO - Completed processing callback for node 68d6bf803c94e462c68815d8 INFO: 172.21.0.1:43350 - "GET / HTTP/1.0" 200 OK 2025-09-26 16:30:46,805 - lava_callback - INFO - Received callback for node 68d6bf833c94e462c68815e1 INFO: 172.21.0.1:43354 - "POST /node/68d6bf833c94e462c68815e1 HTTP/1.0" 202 Accepted 2025-09-26 16:30:46,810 - lava_callback - INFO - Starting async job submit for node 68d6bf833c94e462c68815e1 2025-09-26 16:30:46,829 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:30:46,843 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6bf833c94e462c68815e1 HTTP/1.1" 200 None 2025-09-26 16:30:46,843 - lava_callback - DEBUG - Processing node 68d6bf833c94e462c68815e1: baseline-arm-pengutronix 2025-09-26 16:30:47,010 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 16:30:47,125 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 16:30:47,126 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-68d6bf833c94e462c68815e1/log.txt.gz 2025-09-26 16:30:47,131 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 16:30:47,241 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 16:30:47,241 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-68d6bf833c94e462c68815e1/lava_callback.json.gz 2025-09-26 16:30:47,264 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:30:47,280 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6bf833c94e462c68815e1 HTTP/1.1" 200 None 2025-09-26 16:30:47,281 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:30:47,293 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6b9da2f83206171a92b45 HTTP/1.1" 200 None 2025-09-26 16:30:47,294 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:30:47,325 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6bf833c94e462c68815e1 HTTP/1.1" 200 None 2025-09-26 16:30:47,326 - lava_callback - INFO - Completed processing callback for node 68d6bf833c94e462c68815e1 INFO: 172.21.0.1:37626 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37630 - "GET / HTTP/1.0" 200 OK 2025-09-26 16:30:58,782 - lava_callback - INFO - Received callback for node 68d6bf0b3c94e462c6881584 INFO: 172.21.0.1:37642 - "POST /node/68d6bf0b3c94e462c6881584 HTTP/1.0" 202 Accepted 2025-09-26 16:30:58,789 - lava_callback - INFO - Starting async job submit for node 68d6bf0b3c94e462c6881584 2025-09-26 16:30:58,822 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:30:58,835 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6bf0b3c94e462c6881584 HTTP/1.1" 200 None 2025-09-26 16:30:58,835 - lava_callback - DEBUG - Processing node 68d6bf0b3c94e462c6881584: ltp-dio 2025-09-26 16:30:59,253 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 16:30:59,360 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 16:30:59,361 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-dio-68d6bf0b3c94e462c6881584/log.txt.gz 2025-09-26 16:30:59,371 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 16:30:59,501 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 16:30:59,501 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-dio-68d6bf0b3c94e462c6881584/lava_callback.json.gz 2025-09-26 16:30:59,549 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:30:59,563 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6bf0b3c94e462c6881584 HTTP/1.1" 200 None 2025-09-26 16:30:59,564 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:30:59,574 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6b9da2f83206171a92b45 HTTP/1.1" 200 None 2025-09-26 16:30:59,576 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:30:59,619 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6bf0b3c94e462c6881584 HTTP/1.1" 200 None 2025-09-26 16:30:59,622 - lava_callback - INFO - Completed processing callback for node 68d6bf0b3c94e462c6881584 INFO: 172.21.0.1:46074 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46150 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46152 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42986 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42998 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40320 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43566 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36954 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36962 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42908 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33774 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33784 - "GET / HTTP/1.0" 200 OK 2025-09-26 16:32:23,240 - lava_callback - INFO - Received callback for node 68d46efda5c2c05277bcc518 INFO: 172.21.0.1:37146 - "POST /node/68d46efda5c2c05277bcc518 HTTP/1.0" 202 Accepted 2025-09-26 16:32:23,254 - lava_callback - INFO - Starting async job submit for node 68d46efda5c2c05277bcc518 2025-09-26 16:32:23,295 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:32:23,332 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d46efda5c2c05277bcc518 HTTP/1.1" 200 None 2025-09-26 16:32:23,333 - lava_callback - DEBUG - Processing node 68d46efda5c2c05277bcc518: baseline-arm 2025-09-26 16:32:23,796 - lava_callback - ERROR - Error processing callback for node 68d46efda5c2c05277bcc518: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:37158 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37170 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51902 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55914 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42578 - "GET / HTTP/1.0" 200 OK 2025-09-26 16:32:53,478 - lava_callback - INFO - Received callback for node 68d41cd9a5c2c05277bc5e6b INFO: 172.21.0.1:42580 - "POST /node/68d41cd9a5c2c05277bc5e6b HTTP/1.0" 202 Accepted 2025-09-26 16:32:53,486 - lava_callback - INFO - Starting async job submit for node 68d41cd9a5c2c05277bc5e6b 2025-09-26 16:32:53,526 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:32:53,540 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d41cd9a5c2c05277bc5e6b HTTP/1.1" 200 None 2025-09-26 16:32:53,540 - lava_callback - DEBUG - Processing node 68d41cd9a5c2c05277bc5e6b: baseline-arm64 2025-09-26 16:32:54,481 - lava_callback - ERROR - Error processing callback for node 68d41cd9a5c2c05277bc5e6b: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:42586 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37224 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35626 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35628 - "GET / HTTP/1.0" 200 OK 2025-09-26 16:33:21,678 - lava_callback - INFO - Received callback for node 68d41cdca5c2c05277bc5e6e INFO: 172.21.0.1:58688 - "POST /node/68d41cdca5c2c05277bc5e6e HTTP/1.0" 202 Accepted 2025-09-26 16:33:21,690 - lava_callback - INFO - Starting async job submit for node 68d41cdca5c2c05277bc5e6e 2025-09-26 16:33:21,771 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:33:21,793 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d41cdca5c2c05277bc5e6e HTTP/1.1" 200 None 2025-09-26 16:33:21,794 - lava_callback - DEBUG - Processing node 68d41cdca5c2c05277bc5e6e: baseline-arm64 2025-09-26 16:33:23,416 - lava_callback - ERROR - Error processing callback for node 68d41cdca5c2c05277bc5e6e: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:58696 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58700 - "GET / HTTP/1.0" 200 OK 2025-09-26 16:33:33,830 - lava_callback - INFO - Received callback for node 68d6ba0e2f83206171a92b7b INFO: 172.21.0.1:43708 - "POST /node/68d6ba0e2f83206171a92b7b HTTP/1.0" 202 Accepted 2025-09-26 16:33:33,842 - lava_callback - INFO - Starting async job submit for node 68d6ba0e2f83206171a92b7b 2025-09-26 16:33:33,933 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:33:33,964 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6ba0e2f83206171a92b7b HTTP/1.1" 200 None 2025-09-26 16:33:33,964 - lava_callback - DEBUG - Processing node 68d6ba0e2f83206171a92b7b: baseline-arm64 2025-09-26 16:33:36,081 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.21.0.1:43722 - "GET / HTTP/1.0" 200 OK 2025-09-26 16:33:36,546 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 16:33:36,547 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6ba0e2f83206171a92b7b/log.txt.gz 2025-09-26 16:33:36,581 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 16:33:36,917 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 16:33:36,918 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6ba0e2f83206171a92b7b/lava_callback.json.gz DEBUG: Setup failed for 68d6ba0e2f83206171a92b7b. Transitting job node result: incomplete -> fail 2025-09-26 16:33:37,066 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:33:37,085 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6ba0e2f83206171a92b7b HTTP/1.1" 200 None 2025-09-26 16:33:37,087 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:33:37,103 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d651ddcca86fb17472aa24 HTTP/1.1" 200 None 2025-09-26 16:33:37,104 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:33:37,149 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6ba0e2f83206171a92b7b HTTP/1.1" 200 None 2025-09-26 16:33:37,149 - lava_callback - INFO - Completed processing callback for node 68d6ba0e2f83206171a92b7b INFO: 172.21.0.1:49898 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41546 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41554 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33978 - "GET / HTTP/1.0" 200 OK 2025-09-26 16:34:14,327 - lava_callback - INFO - Received callback for node 68d41d17a5c2c05277bc5e88 INFO: 172.21.0.1:53072 - "POST /node/68d41d17a5c2c05277bc5e88 HTTP/1.0" 202 Accepted 2025-09-26 16:34:14,339 - lava_callback - INFO - Starting async job submit for node 68d41d17a5c2c05277bc5e88 2025-09-26 16:34:14,453 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:34:14,469 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d41d17a5c2c05277bc5e88 HTTP/1.1" 200 None 2025-09-26 16:34:14,469 - lava_callback - DEBUG - Processing node 68d41d17a5c2c05277bc5e88: baseline-arm64 INFO: 172.21.0.1:53074 - "GET /api/metrics HTTP/1.0" 200 OK 2025-09-26 16:34:15,999 - lava_callback - ERROR - Error processing callback for node 68d41d17a5c2c05277bc5e88: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:53076 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35362 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35370 - "GET / HTTP/1.0" 200 OK 2025-09-26 16:34:27,632 - lava_callback - INFO - Received callback for node 68d46f01a5c2c05277bcc51b INFO: 172.21.0.1:35376 - "POST /node/68d46f01a5c2c05277bcc51b HTTP/1.0" 202 Accepted 2025-09-26 16:34:27,638 - lava_callback - INFO - Starting async job submit for node 68d46f01a5c2c05277bcc51b 2025-09-26 16:34:27,659 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:34:27,671 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d46f01a5c2c05277bcc51b HTTP/1.1" 200 None 2025-09-26 16:34:27,671 - lava_callback - DEBUG - Processing node 68d46f01a5c2c05277bcc51b: baseline-arm 2025-09-26 16:34:27,900 - lava_callback - ERROR - Error processing callback for node 68d46f01a5c2c05277bcc51b: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None 2025-09-26 16:34:33,961 - lava_callback - INFO - Received callback for node 68d41d1ba5c2c05277bc5e8b INFO: 172.21.0.1:41546 - "POST /node/68d41d1ba5c2c05277bc5e8b HTTP/1.0" 202 Accepted 2025-09-26 16:34:33,968 - lava_callback - INFO - Starting async job submit for node 68d41d1ba5c2c05277bc5e8b 2025-09-26 16:34:34,009 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:34:34,026 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d41d1ba5c2c05277bc5e8b HTTP/1.1" 200 None 2025-09-26 16:34:34,026 - lava_callback - DEBUG - Processing node 68d41d1ba5c2c05277bc5e8b: baseline-arm64 2025-09-26 16:34:34,907 - lava_callback - ERROR - Error processing callback for node 68d41d1ba5c2c05277bc5e8b: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:41550 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41248 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57304 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57308 - "GET / HTTP/1.0" 200 OK 2025-09-26 16:34:57,826 - lava_callback - INFO - Received callback for node 68d6ba0e2f83206171a92b7e INFO: 172.21.0.1:57320 - "POST /node/68d6ba0e2f83206171a92b7e HTTP/1.0" 202 Accepted 2025-09-26 16:34:57,833 - lava_callback - INFO - Starting async job submit for node 68d6ba0e2f83206171a92b7e 2025-09-26 16:34:57,871 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:34:57,884 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6ba0e2f83206171a92b7e HTTP/1.1" 200 None 2025-09-26 16:34:57,884 - lava_callback - DEBUG - Processing node 68d6ba0e2f83206171a92b7e: baseline-arm64 2025-09-26 16:34:58,850 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 16:34:58,995 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 16:34:58,996 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6ba0e2f83206171a92b7e/log.txt.gz 2025-09-26 16:34:59,016 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 16:34:59,194 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 16:34:59,195 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6ba0e2f83206171a92b7e/lava_callback.json.gz DEBUG: Setup failed for 68d6ba0e2f83206171a92b7e. Transitting job node result: incomplete -> fail 2025-09-26 16:34:59,271 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:34:59,283 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6ba0e2f83206171a92b7e HTTP/1.1" 200 None 2025-09-26 16:34:59,284 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:34:59,300 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d651ddcca86fb17472aa24 HTTP/1.1" 200 None 2025-09-26 16:34:59,301 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:34:59,327 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6ba0e2f83206171a92b7e HTTP/1.1" 200 None 2025-09-26 16:34:59,328 - lava_callback - INFO - Completed processing callback for node 68d6ba0e2f83206171a92b7e INFO: 172.21.0.1:58568 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42226 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42242 - "GET / HTTP/1.0" 200 OK 2025-09-26 16:35:16,921 - lava_callback - INFO - Received callback for node 68d6bb872f83206171a92e7a INFO: 172.21.0.1:42244 - "POST /node/68d6bb872f83206171a92e7a HTTP/1.0" 202 Accepted 2025-09-26 16:35:16,929 - lava_callback - INFO - Starting async job submit for node 68d6bb872f83206171a92e7a 2025-09-26 16:35:16,966 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:35:16,979 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6bb872f83206171a92e7a HTTP/1.1" 200 None 2025-09-26 16:35:16,979 - lava_callback - DEBUG - Processing node 68d6bb872f83206171a92e7a: baseline-arm64 2025-09-26 16:35:17,993 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 16:35:18,107 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 16:35:18,107 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6bb872f83206171a92e7a/log.txt.gz 2025-09-26 16:35:18,127 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 16:35:18,295 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 16:35:18,296 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6bb872f83206171a92e7a/lava_callback.json.gz DEBUG: Setup failed for 68d6bb872f83206171a92e7a. Transitting job node result: incomplete -> fail 2025-09-26 16:35:18,371 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:35:18,384 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6bb872f83206171a92e7a HTTP/1.1" 200 None 2025-09-26 16:35:18,385 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:35:18,397 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d651ddcca86fb17472aa24 HTTP/1.1" 200 None 2025-09-26 16:35:18,398 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:35:18,425 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6bb872f83206171a92e7a HTTP/1.1" 200 None 2025-09-26 16:35:18,426 - lava_callback - INFO - Completed processing callback for node 68d6bb872f83206171a92e7a INFO: 172.21.0.1:38686 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38688 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37486 - "GET / HTTP/1.0" 200 OK 2025-09-26 16:35:44,407 - lava_callback - INFO - Received callback for node 68d41dd9a5c2c05277bc5f0a INFO: 172.21.0.1:43444 - "POST /node/68d41dd9a5c2c05277bc5f0a HTTP/1.0" 202 Accepted 2025-09-26 16:35:44,413 - lava_callback - INFO - Starting async job submit for node 68d41dd9a5c2c05277bc5f0a 2025-09-26 16:35:44,434 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:35:44,446 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d41dd9a5c2c05277bc5f0a HTTP/1.1" 200 None 2025-09-26 16:35:44,447 - lava_callback - DEBUG - Processing node 68d41dd9a5c2c05277bc5f0a: baseline-arm64 2025-09-26 16:35:44,758 - lava_callback - ERROR - Error processing callback for node 68d41dd9a5c2c05277bc5f0a: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:43452 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60694 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60698 - "GET / HTTP/1.0" 200 OK 2025-09-26 16:36:02,311 - lava_callback - INFO - Received callback for node 68d41ddda5c2c05277bc5f0d INFO: 172.21.0.1:52430 - "POST /node/68d41ddda5c2c05277bc5f0d HTTP/1.0" 202 Accepted 2025-09-26 16:36:02,317 - lava_callback - INFO - Starting async job submit for node 68d41ddda5c2c05277bc5f0d 2025-09-26 16:36:02,337 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:36:02,350 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d41ddda5c2c05277bc5f0d HTTP/1.1" 200 None 2025-09-26 16:36:02,350 - lava_callback - DEBUG - Processing node 68d41ddda5c2c05277bc5f0d: baseline-arm64 2025-09-26 16:36:02,645 - lava_callback - ERROR - Error processing callback for node 68d41ddda5c2c05277bc5f0d: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:52432 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46554 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46558 - "GET / HTTP/1.0" 200 OK 2025-09-26 16:36:18,994 - lava_callback - INFO - Received callback for node 68d6bdaa3c94e462c688150a INFO: 172.21.0.1:46566 - "POST /node/68d6bdaa3c94e462c688150a HTTP/1.0" 202 Accepted 2025-09-26 16:36:19,000 - lava_callback - INFO - Starting async job submit for node 68d6bdaa3c94e462c688150a 2025-09-26 16:36:19,025 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:36:19,039 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6bdaa3c94e462c688150a HTTP/1.1" 200 None 2025-09-26 16:36:19,039 - lava_callback - DEBUG - Processing node 68d6bdaa3c94e462c688150a: baseline-x86 2025-09-26 16:36:19,178 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 16:36:19,333 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 16:36:19,334 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-68d6bdaa3c94e462c688150a/log.txt.gz 2025-09-26 16:36:19,337 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 16:36:19,473 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 16:36:19,474 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-68d6bdaa3c94e462c688150a/lava_callback.json.gz 2025-09-26 16:36:19,499 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:36:19,512 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6bdaa3c94e462c688150a HTTP/1.1" 200 None 2025-09-26 16:36:19,513 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:36:19,524 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6b9f22f83206171a92b66 HTTP/1.1" 200 None 2025-09-26 16:36:19,525 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:36:19,551 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6bdaa3c94e462c688150a HTTP/1.1" 200 None 2025-09-26 16:36:19,552 - lava_callback - INFO - Completed processing callback for node 68d6bdaa3c94e462c688150a INFO: 172.21.0.1:44572 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44574 - "GET / HTTP/1.0" 200 OK 2025-09-26 16:36:35,463 - lava_callback - INFO - Received callback for node 68d46f02a5c2c05277bcc51e INFO: 172.21.0.1:39478 - "POST /node/68d46f02a5c2c05277bcc51e HTTP/1.0" 202 Accepted 2025-09-26 16:36:35,469 - lava_callback - INFO - Starting async job submit for node 68d46f02a5c2c05277bcc51e 2025-09-26 16:36:35,490 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:36:35,502 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d46f02a5c2c05277bcc51e HTTP/1.1" 200 None 2025-09-26 16:36:35,502 - lava_callback - DEBUG - Processing node 68d46f02a5c2c05277bcc51e: baseline-arm 2025-09-26 16:36:35,728 - lava_callback - ERROR - Error processing callback for node 68d46f02a5c2c05277bcc51e: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:39484 - "GET / HTTP/1.0" 200 OK 2025-09-26 16:36:39,283 - lava_callback - INFO - Received callback for node 68d6b03e2f83206171a929ae INFO: 172.21.0.1:39498 - "POST /node/68d6b03e2f83206171a929ae HTTP/1.0" 202 Accepted 2025-09-26 16:36:39,295 - lava_callback - INFO - Starting async job submit for node 68d6b03e2f83206171a929ae 2025-09-26 16:36:39,384 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:36:39,398 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6b03e2f83206171a929ae HTTP/1.1" 200 None 2025-09-26 16:36:39,399 - lava_callback - DEBUG - Processing node 68d6b03e2f83206171a929ae: ltp-smoketest 2025-09-26 16:36:40,234 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 16:36:40,421 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 16:36:40,422 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoketest-68d6b03e2f83206171a929ae/log.txt.gz 2025-09-26 16:36:40,437 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 16:36:40,594 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 16:36:40,595 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoketest-68d6b03e2f83206171a929ae/lava_callback.json.gz 2025-09-26 16:36:40,804 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:36:40,820 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6b03e2f83206171a929ae HTTP/1.1" 200 None 2025-09-26 16:36:40,821 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:36:40,848 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d64d07a5c3ab168ecc7a34 HTTP/1.1" 200 None 2025-09-26 16:36:40,850 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:36:40,887 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6b03e2f83206171a929ae HTTP/1.1" 200 None 2025-09-26 16:36:40,888 - lava_callback - INFO - Completed processing callback for node 68d6b03e2f83206171a929ae INFO: 172.21.0.1:45398 - "GET / HTTP/1.0" 200 OK 2025-09-26 16:36:50,669 - lava_callback - INFO - Received callback for node 68d6bb872f83206171a92e7d INFO: 172.21.0.1:60812 - "POST /node/68d6bb872f83206171a92e7d HTTP/1.0" 202 Accepted 2025-09-26 16:36:50,677 - lava_callback - INFO - Starting async job submit for node 68d6bb872f83206171a92e7d 2025-09-26 16:36:50,715 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:36:50,736 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6bb872f83206171a92e7d HTTP/1.1" 200 None 2025-09-26 16:36:50,737 - lava_callback - DEBUG - Processing node 68d6bb872f83206171a92e7d: baseline-arm64 2025-09-26 16:36:51,858 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 16:36:51,995 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 16:36:51,996 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6bb872f83206171a92e7d/log.txt.gz 2025-09-26 16:36:52,017 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 16:36:52,166 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 16:36:52,167 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6bb872f83206171a92e7d/lava_callback.json.gz DEBUG: Setup failed for 68d6bb872f83206171a92e7d. Transitting job node result: incomplete -> fail 2025-09-26 16:36:52,242 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:36:52,254 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6bb872f83206171a92e7d HTTP/1.1" 200 None 2025-09-26 16:36:52,255 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:36:52,266 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d651ddcca86fb17472aa24 HTTP/1.1" 200 None 2025-09-26 16:36:52,267 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:36:52,297 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6bb872f83206171a92e7d HTTP/1.1" 200 None 2025-09-26 16:36:52,298 - lava_callback - INFO - Completed processing callback for node 68d6bb872f83206171a92e7d INFO: 172.21.0.1:60828 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60840 - "GET / HTTP/1.0" 200 OK 2025-09-26 16:37:02,839 - lava_callback - INFO - Received callback for node 68d41e1ea5c2c05277bc5f69 INFO: 172.21.0.1:56690 - "POST /node/68d41e1ea5c2c05277bc5f69 HTTP/1.0" 202 Accepted 2025-09-26 16:37:02,845 - lava_callback - INFO - Starting async job submit for node 68d41e1ea5c2c05277bc5f69 2025-09-26 16:37:02,865 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:37:02,886 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d41e1ea5c2c05277bc5f69 HTTP/1.1" 200 None 2025-09-26 16:37:02,886 - lava_callback - DEBUG - Processing node 68d41e1ea5c2c05277bc5f69: baseline-arm64 2025-09-26 16:37:03,209 - lava_callback - ERROR - Error processing callback for node 68d41e1ea5c2c05277bc5f69: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:56706 - "GET / HTTP/1.0" 200 OK 2025-09-26 16:37:10,440 - lava_callback - INFO - Received callback for node 68d6bd4f3c94e462c68814ab INFO: 172.21.0.1:54982 - "POST /node/68d6bd4f3c94e462c68814ab HTTP/1.0" 202 Accepted 2025-09-26 16:37:10,447 - lava_callback - INFO - Starting async job submit for node 68d6bd4f3c94e462c68814ab 2025-09-26 16:37:10,484 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:37:10,496 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6bd4f3c94e462c68814ab HTTP/1.1" 200 None 2025-09-26 16:37:10,497 - lava_callback - DEBUG - Processing node 68d6bd4f3c94e462c68814ab: baseline-arm64 2025-09-26 16:37:11,477 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 16:37:11,607 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 16:37:11,608 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6bd4f3c94e462c68814ab/log.txt.gz 2025-09-26 16:37:11,627 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 16:37:11,780 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 16:37:11,781 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6bd4f3c94e462c68814ab/lava_callback.json.gz DEBUG: Setup failed for 68d6bd4f3c94e462c68814ab. Transitting job node result: incomplete -> fail 2025-09-26 16:37:11,854 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:37:11,870 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6bd4f3c94e462c68814ab HTTP/1.1" 200 None 2025-09-26 16:37:11,871 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:37:11,882 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d651ddcca86fb17472aa24 HTTP/1.1" 200 None 2025-09-26 16:37:11,883 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:37:11,910 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6bd4f3c94e462c68814ab HTTP/1.1" 200 None 2025-09-26 16:37:11,911 - lava_callback - INFO - Completed processing callback for node 68d6bd4f3c94e462c68814ab INFO: 172.21.0.1:54986 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55002 - "GET / HTTP/1.0" 200 OK 2025-09-26 16:37:24,099 - lava_callback - INFO - Received callback for node 68d41e20a5c2c05277bc5f6c INFO: 172.21.0.1:38210 - "POST /node/68d41e20a5c2c05277bc5f6c HTTP/1.0" 202 Accepted 2025-09-26 16:37:24,104 - lava_callback - INFO - Starting async job submit for node 68d41e20a5c2c05277bc5f6c 2025-09-26 16:37:24,125 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:37:24,144 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d41e20a5c2c05277bc5f6c HTTP/1.1" 200 None 2025-09-26 16:37:24,145 - lava_callback - DEBUG - Processing node 68d41e20a5c2c05277bc5f6c: baseline-arm64 2025-09-26 16:37:24,452 - lava_callback - ERROR - Error processing callback for node 68d41e20a5c2c05277bc5f6c: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:38224 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38232 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41020 - "GET / HTTP/1.0" 200 OK 2025-09-26 16:37:39,928 - lava_callback - INFO - Received callback for node 68d41e4ea5c2c05277bc5faa INFO: 172.21.0.1:41168 - "POST /node/68d41e4ea5c2c05277bc5faa HTTP/1.0" 202 Accepted 2025-09-26 16:37:39,933 - lava_callback - INFO - Starting async job submit for node 68d41e4ea5c2c05277bc5faa 2025-09-26 16:37:39,934 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:37:39,955 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d41e4ea5c2c05277bc5faa HTTP/1.1" 200 None 2025-09-26 16:37:39,955 - lava_callback - DEBUG - Processing node 68d41e4ea5c2c05277bc5faa: baseline-arm64-baylibre 2025-09-26 16:37:39,955 - lava_callback - ERROR - Error processing callback for node 68d41e4ea5c2c05277bc5faa: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 16:37:41,936 - lava_callback - INFO - Received callback for node 68d41ae9a5c2c05277bc5c36 INFO: 172.21.0.1:41176 - "POST /node/68d41ae9a5c2c05277bc5c36 HTTP/1.0" 202 Accepted 2025-09-26 16:37:41,948 - lava_callback - INFO - Starting async job submit for node 68d41ae9a5c2c05277bc5c36 2025-09-26 16:37:42,313 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:37:42,344 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d41ae9a5c2c05277bc5c36 HTTP/1.1" 200 None 2025-09-26 16:37:42,345 - lava_callback - DEBUG - Processing node 68d41ae9a5c2c05277bc5c36: ltp-controllers 2025-09-26 16:37:43,761 - lava_callback - INFO - Received callback for node 68d6a74f2f83206171a924f1 INFO: 172.21.0.1:41186 - "POST /node/68d6a74f2f83206171a924f1 HTTP/1.0" 202 Accepted 2025-09-26 16:37:43,783 - lava_callback - INFO - Starting async job submit for node 68d6a74f2f83206171a924f1 2025-09-26 16:37:43,831 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:37:43,915 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6a74f2f83206171a924f1 HTTP/1.1" 200 None 2025-09-26 16:37:43,925 - lava_callback - DEBUG - Processing node 68d6a74f2f83206171a924f1: baseline-arm64-broonie 2025-09-26 16:37:44,506 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 16:37:44,730 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 16:37:44,771 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-68d6a74f2f83206171a924f1/log.txt.gz 2025-09-26 16:37:44,869 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 16:37:45,116 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 16:37:45,173 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-68d6a74f2f83206171a924f1/lava_callback.json.gz 2025-09-26 16:37:45,217 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:37:45,311 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6a74f2f83206171a924f1 HTTP/1.1" 200 None 2025-09-26 16:37:45,333 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:37:45,418 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d648ed5d233c363a762351 HTTP/1.1" 200 None 2025-09-26 16:37:45,563 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:37:45,677 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6a74f2f83206171a924f1 HTTP/1.1" 200 None 2025-09-26 16:37:45,678 - lava_callback - INFO - Completed processing callback for node 68d6a74f2f83206171a924f1 2025-09-26 16:37:45,758 - lava_callback - ERROR - Error processing callback for node 68d41ae9a5c2c05277bc5c36: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:41198 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51920 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51932 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50022 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49566 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49568 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37964 - "GET / HTTP/1.0" 200 OK 2025-09-26 16:38:26,886 - lava_callback - INFO - Received callback for node 68d41ebaa5c2c05277bc65c2 INFO: 172.21.0.1:37978 - "POST /node/68d41ebaa5c2c05277bc65c2 HTTP/1.0" 202 Accepted 2025-09-26 16:38:26,896 - lava_callback - INFO - Starting async job submit for node 68d41ebaa5c2c05277bc65c2 2025-09-26 16:38:26,929 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:38:26,947 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d41ebaa5c2c05277bc65c2 HTTP/1.1" 200 None 2025-09-26 16:38:26,947 - lava_callback - DEBUG - Processing node 68d41ebaa5c2c05277bc65c2: baseline-arm64 2025-09-26 16:38:27,443 - lava_callback - ERROR - Error processing callback for node 68d41ebaa5c2c05277bc65c2: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:37992 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34344 - "GET / HTTP/1.0" 200 OK 2025-09-26 16:38:39,946 - lava_callback - INFO - Received callback for node 68d41e85a5c2c05277bc6538 INFO: 172.21.0.1:52334 - "POST /node/68d41e85a5c2c05277bc6538 HTTP/1.0" 202 Accepted 2025-09-26 16:38:39,953 - lava_callback - INFO - Starting async job submit for node 68d41e85a5c2c05277bc6538 2025-09-26 16:38:39,954 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:38:39,968 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d41e85a5c2c05277bc6538 HTTP/1.1" 200 None 2025-09-26 16:38:39,968 - lava_callback - DEBUG - Processing node 68d41e85a5c2c05277bc6538: baseline-arm64-baylibre 2025-09-26 16:38:39,968 - lava_callback - ERROR - Error processing callback for node 68d41e85a5c2c05277bc6538: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 16:38:40,353 - lava_callback - INFO - Received callback for node 68d41ec7a5c2c05277bc6605 INFO: 172.21.0.1:52342 - "POST /node/68d41ec7a5c2c05277bc6605 HTTP/1.0" 202 Accepted 2025-09-26 16:38:40,361 - lava_callback - INFO - Starting async job submit for node 68d41ec7a5c2c05277bc6605 2025-09-26 16:38:40,385 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:38:40,404 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d41ec7a5c2c05277bc6605 HTTP/1.1" 200 None 2025-09-26 16:38:40,404 - lava_callback - DEBUG - Processing node 68d41ec7a5c2c05277bc6605: baseline-arm64 2025-09-26 16:38:40,907 - lava_callback - ERROR - Error processing callback for node 68d41ec7a5c2c05277bc6605: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None 2025-09-26 16:38:44,109 - lava_callback - INFO - Received callback for node 68d46f1ca5c2c05277bcc545 INFO: 172.21.0.1:52350 - "POST /node/68d46f1ca5c2c05277bcc545 HTTP/1.0" 202 Accepted 2025-09-26 16:38:44,118 - lava_callback - INFO - Starting async job submit for node 68d46f1ca5c2c05277bcc545 2025-09-26 16:38:44,166 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:38:44,206 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d46f1ca5c2c05277bcc545 HTTP/1.1" 200 None 2025-09-26 16:38:44,207 - lava_callback - DEBUG - Processing node 68d46f1ca5c2c05277bcc545: baseline-arm 2025-09-26 16:38:44,696 - lava_callback - ERROR - Error processing callback for node 68d46f1ca5c2c05277bcc545: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:52364 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33836 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33846 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49652 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54856 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54872 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49442 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49456 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51556 - "GET / HTTP/1.0" 200 OK 2025-09-26 16:39:41,194 - lava_callback - INFO - Received callback for node 68d420b2a5c2c05277bc6905 INFO: 172.21.0.1:49164 - "POST /node/68d420b2a5c2c05277bc6905 HTTP/1.0" 202 Accepted 2025-09-26 16:39:41,204 - lava_callback - INFO - Starting async job submit for node 68d420b2a5c2c05277bc6905 2025-09-26 16:39:41,245 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:39:41,261 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d420b2a5c2c05277bc6905 HTTP/1.1" 200 None 2025-09-26 16:39:41,261 - lava_callback - DEBUG - Processing node 68d420b2a5c2c05277bc6905: baseline-arm64 2025-09-26 16:39:41,773 - lava_callback - ERROR - Error processing callback for node 68d420b2a5c2c05277bc6905: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:49168 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57524 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57526 - "GET / HTTP/1.0" 200 OK 2025-09-26 16:40:05,522 - lava_callback - INFO - Received callback for node 68d420b4a5c2c05277bc6908 INFO: 172.21.0.1:48794 - "POST /node/68d420b4a5c2c05277bc6908 HTTP/1.0" 202 Accepted 2025-09-26 16:40:05,532 - lava_callback - INFO - Starting async job submit for node 68d420b4a5c2c05277bc6908 2025-09-26 16:40:05,574 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:40:05,598 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d420b4a5c2c05277bc6908 HTTP/1.1" 200 None 2025-09-26 16:40:05,598 - lava_callback - DEBUG - Processing node 68d420b4a5c2c05277bc6908: baseline-arm64 2025-09-26 16:40:06,148 - lava_callback - ERROR - Error processing callback for node 68d420b4a5c2c05277bc6908: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:48796 - "GET / HTTP/1.0" 200 OK 2025-09-26 16:40:08,984 - lava_callback - INFO - Received callback for node 68d41edfa5c2c05277bc6654 INFO: 172.21.0.1:48806 - "POST /node/68d41edfa5c2c05277bc6654 HTTP/1.0" 202 Accepted 2025-09-26 16:40:08,993 - lava_callback - INFO - Starting async job submit for node 68d41edfa5c2c05277bc6654 2025-09-26 16:40:08,994 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:40:09,013 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d41edfa5c2c05277bc6654 HTTP/1.1" 200 None 2025-09-26 16:40:09,014 - lava_callback - DEBUG - Processing node 68d41edfa5c2c05277bc6654: baseline-arm64-clabbe 2025-09-26 16:40:09,014 - lava_callback - ERROR - Error processing callback for node 68d41edfa5c2c05277bc6654: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:33666 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33668 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52914 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52916 - "GET / HTTP/1.0" 200 OK 2025-09-26 16:40:29,015 - lava_callback - INFO - Received callback for node 68d41eefa5c2c05277bc666c INFO: 172.21.0.1:52918 - "POST /node/68d41eefa5c2c05277bc666c HTTP/1.0" 202 Accepted 2025-09-26 16:40:29,023 - lava_callback - INFO - Starting async job submit for node 68d41eefa5c2c05277bc666c 2025-09-26 16:40:29,024 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:40:29,036 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d41eefa5c2c05277bc666c HTTP/1.1" 200 None 2025-09-26 16:40:29,036 - lava_callback - DEBUG - Processing node 68d41eefa5c2c05277bc666c: baseline-arm64-clabbe 2025-09-26 16:40:29,036 - lava_callback - ERROR - Error processing callback for node 68d41eefa5c2c05277bc666c: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 16:40:29,673 - lava_callback - INFO - Received callback for node 68d41ef2a5c2c05277bc666f INFO: 172.21.0.1:53484 - "POST /node/68d41ef2a5c2c05277bc666f HTTP/1.0" 202 Accepted 2025-09-26 16:40:29,684 - lava_callback - INFO - Starting async job submit for node 68d41ef2a5c2c05277bc666f 2025-09-26 16:40:29,687 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:40:29,709 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d41ef2a5c2c05277bc666f HTTP/1.1" 200 None 2025-09-26 16:40:29,710 - lava_callback - DEBUG - Processing node 68d41ef2a5c2c05277bc666f: baseline-arm64-clabbe 2025-09-26 16:40:29,710 - lava_callback - ERROR - Error processing callback for node 68d41ef2a5c2c05277bc666f: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 16:40:30,336 - lava_callback - INFO - Received callback for node 68d41ef8a5c2c05277bc6672 INFO: 172.21.0.1:53500 - "POST /node/68d41ef8a5c2c05277bc6672 HTTP/1.0" 202 Accepted 2025-09-26 16:40:30,344 - lava_callback - INFO - Starting async job submit for node 68d41ef8a5c2c05277bc6672 2025-09-26 16:40:30,345 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:40:30,361 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d41ef8a5c2c05277bc6672 HTTP/1.1" 200 None 2025-09-26 16:40:30,362 - lava_callback - DEBUG - Processing node 68d41ef8a5c2c05277bc6672: baseline-arm64-clabbe 2025-09-26 16:40:30,362 - lava_callback - ERROR - Error processing callback for node 68d41ef8a5c2c05277bc6672: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:53504 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39906 - "GET / HTTP/1.0" 200 OK 2025-09-26 16:40:48,995 - lava_callback - INFO - Received callback for node 68d41efba5c2c05277bc6675 INFO: 172.21.0.1:39922 - "POST /node/68d41efba5c2c05277bc6675 HTTP/1.0" 202 Accepted 2025-09-26 16:40:49,004 - lava_callback - INFO - Starting async job submit for node 68d41efba5c2c05277bc6675 2025-09-26 16:40:49,005 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:40:49,036 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d41efba5c2c05277bc6675 HTTP/1.1" 200 None 2025-09-26 16:40:49,037 - lava_callback - DEBUG - Processing node 68d41efba5c2c05277bc6675: baseline-arm64-clabbe 2025-09-26 16:40:49,037 - lava_callback - ERROR - Error processing callback for node 68d41efba5c2c05277bc6675: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 16:40:49,151 - lava_callback - INFO - Received callback for node 68d46f1da5c2c05277bcc548 INFO: 172.21.0.1:39936 - "POST /node/68d46f1da5c2c05277bcc548 HTTP/1.0" 202 Accepted 2025-09-26 16:40:49,161 - lava_callback - INFO - Starting async job submit for node 68d46f1da5c2c05277bcc548 2025-09-26 16:40:49,213 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:40:49,233 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d46f1da5c2c05277bcc548 HTTP/1.1" 200 None 2025-09-26 16:40:49,233 - lava_callback - DEBUG - Processing node 68d46f1da5c2c05277bcc548: baseline-arm 2025-09-26 16:40:49,795 - lava_callback - ERROR - Error processing callback for node 68d46f1da5c2c05277bcc548: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:33154 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33160 - "GET / HTTP/1.0" 200 OK 2025-09-26 16:41:01,006 - lava_callback - INFO - Received callback for node 68d420e5a5c2c05277bc6c4e INFO: 172.21.0.1:45566 - "POST /node/68d420e5a5c2c05277bc6c4e HTTP/1.0" 202 Accepted 2025-09-26 16:41:01,016 - lava_callback - INFO - Starting async job submit for node 68d420e5a5c2c05277bc6c4e 2025-09-26 16:41:01,048 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:41:01,061 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d420e5a5c2c05277bc6c4e HTTP/1.1" 200 None 2025-09-26 16:41:01,061 - lava_callback - DEBUG - Processing node 68d420e5a5c2c05277bc6c4e: baseline-arm64 2025-09-26 16:41:01,492 - lava_callback - ERROR - Error processing callback for node 68d420e5a5c2c05277bc6c4e: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:45580 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39468 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39478 - "GET / HTTP/1.0" 200 OK 2025-09-26 16:41:18,880 - lava_callback - INFO - Received callback for node 68d6c1ec3c94e462c6881686 INFO: 172.21.0.1:39486 - "POST /node/68d6c1ec3c94e462c6881686 HTTP/1.0" 202 Accepted 2025-09-26 16:41:18,888 - lava_callback - INFO - Starting async job submit for node 68d6c1ec3c94e462c6881686 2025-09-26 16:41:18,917 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:41:18,966 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6c1ec3c94e462c6881686 HTTP/1.1" 200 None 2025-09-26 16:41:18,966 - lava_callback - DEBUG - Processing node 68d6c1ec3c94e462c6881686: baseline-arm 2025-09-26 16:41:19,400 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 16:41:19,579 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 16:41:19,580 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-68d6c1ec3c94e462c6881686/log.txt.gz 2025-09-26 16:41:19,589 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 16:41:19,797 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 16:41:19,798 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-68d6c1ec3c94e462c6881686/lava_callback.json.gz 2025-09-26 16:41:19,820 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:41:19,839 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6c1ec3c94e462c6881686 HTTP/1.1" 200 None 2025-09-26 16:41:19,840 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:41:19,854 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6bd793c94e462c68814e2 HTTP/1.1" 200 None 2025-09-26 16:41:19,856 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:41:19,918 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6c1ec3c94e462c6881686 HTTP/1.1" 200 None 2025-09-26 16:41:19,919 - lava_callback - INFO - Completed processing callback for node 68d6c1ec3c94e462c6881686 INFO: 172.21.0.1:59600 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59616 - "GET / HTTP/1.0" 200 OK 2025-09-26 16:41:28,423 - lava_callback - INFO - Received callback for node 68d41f23a5c2c05277bc66b2 INFO: 172.21.0.1:59630 - "POST /node/68d41f23a5c2c05277bc66b2 HTTP/1.0" 202 Accepted 2025-09-26 16:41:28,439 - lava_callback - INFO - Starting async job submit for node 68d41f23a5c2c05277bc66b2 2025-09-26 16:41:28,442 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:41:28,459 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d41f23a5c2c05277bc66b2 HTTP/1.1" 200 None 2025-09-26 16:41:28,460 - lava_callback - DEBUG - Processing node 68d41f23a5c2c05277bc66b2: baseline-arm 2025-09-26 16:41:28,460 - lava_callback - ERROR - Error processing callback for node 68d41f23a5c2c05277bc66b2: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 16:41:29,114 - lava_callback - INFO - Received callback for node 68d41f26a5c2c05277bc66b5 INFO: 172.21.0.1:59640 - "POST /node/68d41f26a5c2c05277bc66b5 HTTP/1.0" 202 Accepted 2025-09-26 16:41:29,123 - lava_callback - INFO - Starting async job submit for node 68d41f26a5c2c05277bc66b5 2025-09-26 16:41:29,124 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:41:29,143 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d41f26a5c2c05277bc66b5 HTTP/1.1" 200 None 2025-09-26 16:41:29,144 - lava_callback - DEBUG - Processing node 68d41f26a5c2c05277bc66b5: baseline-arm 2025-09-26 16:41:29,144 - lava_callback - ERROR - Error processing callback for node 68d41f26a5c2c05277bc66b5: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 16:41:29,862 - lava_callback - INFO - Received callback for node 68d6c2023c94e462c6881705 INFO: 172.21.0.1:32828 - "POST /node/68d6c2023c94e462c6881705 HTTP/1.0" 202 Accepted 2025-09-26 16:41:29,913 - lava_callback - INFO - Starting async job submit for node 68d6c2023c94e462c6881705 2025-09-26 16:41:29,954 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:41:29,970 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6c2023c94e462c6881705 HTTP/1.1" 200 None 2025-09-26 16:41:29,971 - lava_callback - DEBUG - Processing node 68d6c2023c94e462c6881705: baseline-arm-pengutronix 2025-09-26 16:41:30,225 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 16:41:30,656 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 16:41:30,657 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-68d6c2023c94e462c6881705/log.txt.gz 2025-09-26 16:41:30,661 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 16:41:31,255 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 16:41:31,256 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-68d6c2023c94e462c6881705/lava_callback.json.gz 2025-09-26 16:41:31,297 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:41:31,313 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6c2023c94e462c6881705 HTTP/1.1" 200 None 2025-09-26 16:41:31,314 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:41:31,328 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6bd793c94e462c68814e2 HTTP/1.1" 200 None 2025-09-26 16:41:31,330 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:41:31,381 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6c2023c94e462c6881705 HTTP/1.1" 200 None 2025-09-26 16:41:31,381 - lava_callback - INFO - Completed processing callback for node 68d6c2023c94e462c6881705 2025-09-26 16:41:31,936 - lava_callback - INFO - Received callback for node 68d6c1ee3c94e462c6881692 INFO: 172.21.0.1:32832 - "POST /node/68d6c1ee3c94e462c6881692 HTTP/1.0" 202 Accepted 2025-09-26 16:41:31,953 - lava_callback - INFO - Starting async job submit for node 68d6c1ee3c94e462c6881692 2025-09-26 16:41:31,994 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:41:32,035 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6c1ee3c94e462c6881692 HTTP/1.1" 200 None 2025-09-26 16:41:32,035 - lava_callback - DEBUG - Processing node 68d6c1ee3c94e462c6881692: baseline-arm 2025-09-26 16:41:32,590 - lava_callback - INFO - Received callback for node 68d6c2003c94e462c68816fc INFO: 172.21.0.1:32840 - "POST /node/68d6c2003c94e462c68816fc HTTP/1.0" 202 Accepted 2025-09-26 16:41:32,599 - lava_callback - INFO - Starting async job submit for node 68d6c2003c94e462c68816fc 2025-09-26 16:41:32,690 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 16:41:32,693 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:41:32,716 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6c2003c94e462c68816fc HTTP/1.1" 200 None 2025-09-26 16:41:32,717 - lava_callback - DEBUG - Processing node 68d6c2003c94e462c68816fc: baseline-arm-pengutronix 2025-09-26 16:41:33,008 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 16:41:33,152 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 16:41:33,153 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-68d6c1ee3c94e462c6881692/log.txt.gz 2025-09-26 16:41:33,166 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 16:41:33,333 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 16:41:33,334 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-68d6c2003c94e462c68816fc/log.txt.gz 2025-09-26 16:41:33,340 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 16:41:33,736 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 16:41:33,736 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-68d6c1ee3c94e462c6881692/lava_callback.json.gz 2025-09-26 16:41:33,776 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:41:33,807 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6c1ee3c94e462c6881692 HTTP/1.1" 200 None 2025-09-26 16:41:33,809 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:41:33,809 - lava_callback - INFO - Received callback for node 68d6c2023c94e462c6881702 2025-09-26 16:41:33,822 - lava_callback - INFO - Starting async job submit for node 68d6c2023c94e462c6881702 INFO: 172.21.0.1:32846 - "POST /node/68d6c2023c94e462c6881702 HTTP/1.0" 202 Accepted 2025-09-26 16:41:33,859 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6bd793c94e462c68814e2 HTTP/1.1" 200 None 2025-09-26 16:41:33,863 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:41:33,866 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:41:33,889 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6c2023c94e462c6881702 HTTP/1.1" 200 None 2025-09-26 16:41:33,889 - lava_callback - DEBUG - Processing node 68d6c2023c94e462c6881702: baseline-arm-pengutronix 2025-09-26 16:41:33,926 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6c1ee3c94e462c6881692 HTTP/1.1" 200 None 2025-09-26 16:41:33,927 - lava_callback - INFO - Completed processing callback for node 68d6c1ee3c94e462c6881692 2025-09-26 16:41:34,065 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 16:41:34,066 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-68d6c2003c94e462c68816fc/lava_callback.json.gz 2025-09-26 16:41:34,132 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:41:34,221 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6c2003c94e462c68816fc HTTP/1.1" 200 None 2025-09-26 16:41:34,228 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:41:34,276 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6bd793c94e462c68814e2 HTTP/1.1" 200 None 2025-09-26 16:41:34,277 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:41:34,311 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 16:41:34,314 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6c2003c94e462c68816fc HTTP/1.1" 200 None 2025-09-26 16:41:34,315 - lava_callback - INFO - Completed processing callback for node 68d6c2003c94e462c68816fc 2025-09-26 16:41:35,009 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 16:41:35,010 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-68d6c2023c94e462c6881702/log.txt.gz 2025-09-26 16:41:35,015 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 16:41:35,250 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 16:41:35,251 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-68d6c2023c94e462c6881702/lava_callback.json.gz 2025-09-26 16:41:35,287 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:41:35,322 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6c2023c94e462c6881702 HTTP/1.1" 200 None 2025-09-26 16:41:35,323 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:41:35,338 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6bd793c94e462c68814e2 HTTP/1.1" 200 None 2025-09-26 16:41:35,339 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:41:35,371 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6c2023c94e462c6881702 HTTP/1.1" 200 None 2025-09-26 16:41:35,371 - lava_callback - INFO - Completed processing callback for node 68d6c2023c94e462c6881702 INFO: 172.21.0.1:32848 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34132 - "GET / HTTP/1.0" 200 OK 2025-09-26 16:41:51,986 - lava_callback - INFO - Received callback for node 68d6c1f13c94e462c688169b INFO: 172.21.0.1:38932 - "POST /node/68d6c1f13c94e462c688169b HTTP/1.0" 202 Accepted 2025-09-26 16:41:51,995 - lava_callback - INFO - Starting async job submit for node 68d6c1f13c94e462c688169b 2025-09-26 16:41:52,058 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:41:52,078 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6c1f13c94e462c688169b HTTP/1.1" 200 None 2025-09-26 16:41:52,079 - lava_callback - DEBUG - Processing node 68d6c1f13c94e462c688169b: baseline-arm-baylibre 2025-09-26 16:41:52,444 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 16:41:52,749 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 16:41:52,750 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-baylibre-68d6c1f13c94e462c688169b/log.txt.gz 2025-09-26 16:41:52,757 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 16:41:53,267 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 16:41:53,268 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-baylibre-68d6c1f13c94e462c688169b/lava_callback.json.gz 2025-09-26 16:41:53,310 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:41:53,338 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6c1f13c94e462c688169b HTTP/1.1" 200 None 2025-09-26 16:41:53,339 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:41:53,380 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6bd793c94e462c68814e2 HTTP/1.1" 200 None 2025-09-26 16:41:53,382 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:41:53,453 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6c1f13c94e462c688169b HTTP/1.1" 200 None 2025-09-26 16:41:53,454 - lava_callback - INFO - Completed processing callback for node 68d6c1f13c94e462c688169b 2025-09-26 16:41:54,215 - lava_callback - INFO - Received callback for node 68d6c1f53c94e462c68816b9 INFO: 172.21.0.1:38946 - "POST /node/68d6c1f53c94e462c68816b9 HTTP/1.0" 202 Accepted 2025-09-26 16:41:54,225 - lava_callback - INFO - Starting async job submit for node 68d6c1f53c94e462c68816b9 2025-09-26 16:41:54,271 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:41:54,295 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6c1f53c94e462c68816b9 HTTP/1.1" 200 None 2025-09-26 16:41:54,295 - lava_callback - DEBUG - Processing node 68d6c1f53c94e462c68816b9: baseline-arm-broonie 2025-09-26 16:41:54,689 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.21.0.1:38948 - "GET / HTTP/1.0" 200 OK 2025-09-26 16:41:55,089 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 16:41:55,090 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-68d6c1f53c94e462c68816b9/log.txt.gz 2025-09-26 16:41:55,097 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 16:41:55,473 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 16:41:55,474 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-68d6c1f53c94e462c68816b9/lava_callback.json.gz 2025-09-26 16:41:55,508 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:41:55,525 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6c1f53c94e462c68816b9 HTTP/1.1" 200 None 2025-09-26 16:41:55,526 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:41:55,542 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6bd793c94e462c68814e2 HTTP/1.1" 200 None 2025-09-26 16:41:55,543 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:41:55,615 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6c1f53c94e462c68816b9 HTTP/1.1" 200 None 2025-09-26 16:41:55,616 - lava_callback - INFO - Completed processing callback for node 68d6c1f53c94e462c68816b9 2025-09-26 16:41:55,668 - lava_callback - INFO - Received callback for node 68d6c1ef3c94e462c6881698 INFO: 172.21.0.1:38956 - "POST /node/68d6c1ef3c94e462c6881698 HTTP/1.0" 202 Accepted 2025-09-26 16:41:55,679 - lava_callback - INFO - Starting async job submit for node 68d6c1ef3c94e462c6881698 2025-09-26 16:41:55,751 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:41:55,781 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6c1ef3c94e462c6881698 HTTP/1.1" 200 None 2025-09-26 16:41:55,782 - lava_callback - DEBUG - Processing node 68d6c1ef3c94e462c6881698: baseline-arm-baylibre 2025-09-26 16:41:56,156 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.21.0.1:38958 - "GET / HTTP/1.0" 200 OK 2025-09-26 16:41:56,627 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 16:41:56,628 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-baylibre-68d6c1ef3c94e462c6881698/log.txt.gz 2025-09-26 16:41:56,635 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 16:41:57,089 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 16:41:57,089 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-baylibre-68d6c1ef3c94e462c6881698/lava_callback.json.gz 2025-09-26 16:41:57,125 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:41:57,154 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6c1ef3c94e462c6881698 HTTP/1.1" 200 None 2025-09-26 16:41:57,155 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:41:57,171 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6bd793c94e462c68814e2 HTTP/1.1" 200 None 2025-09-26 16:41:57,173 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:41:57,225 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6c1ef3c94e462c6881698 HTTP/1.1" 200 None 2025-09-26 16:41:57,225 - lava_callback - INFO - Completed processing callback for node 68d6c1ef3c94e462c6881698 2025-09-26 16:42:05,914 - lava_callback - INFO - Received callback for node 68d6c1f23c94e462c68816a1 INFO: 172.21.0.1:35740 - "POST /node/68d6c1f23c94e462c68816a1 HTTP/1.0" 202 Accepted 2025-09-26 16:42:05,923 - lava_callback - INFO - Starting async job submit for node 68d6c1f23c94e462c68816a1 2025-09-26 16:42:05,966 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:42:05,981 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6c1f23c94e462c68816a1 HTTP/1.1" 200 None 2025-09-26 16:42:05,982 - lava_callback - DEBUG - Processing node 68d6c1f23c94e462c68816a1: baseline-arm-baylibre 2025-09-26 16:42:06,309 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.21.0.1:35748 - "GET / HTTP/1.0" 200 OK 2025-09-26 16:42:06,740 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 16:42:06,741 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-baylibre-68d6c1f23c94e462c68816a1/log.txt.gz 2025-09-26 16:42:06,748 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 16:42:07,109 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 16:42:07,110 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-baylibre-68d6c1f23c94e462c68816a1/lava_callback.json.gz 2025-09-26 16:42:07,184 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:42:07,203 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6c1f23c94e462c68816a1 HTTP/1.1" 200 None 2025-09-26 16:42:07,204 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:42:07,219 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6bd793c94e462c68814e2 HTTP/1.1" 200 None 2025-09-26 16:42:07,221 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:42:07,258 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6c1f23c94e462c68816a1 HTTP/1.1" 200 None 2025-09-26 16:42:07,259 - lava_callback - INFO - Completed processing callback for node 68d6c1f23c94e462c68816a1 2025-09-26 16:42:07,704 - lava_callback - INFO - Received callback for node 68d6c1fb3c94e462c68816d7 INFO: 172.21.0.1:35760 - "POST /node/68d6c1fb3c94e462c68816d7 HTTP/1.0" 202 Accepted 2025-09-26 16:42:07,712 - lava_callback - INFO - Starting async job submit for node 68d6c1fb3c94e462c68816d7 2025-09-26 16:42:07,759 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:42:07,783 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6c1fb3c94e462c68816d7 HTTP/1.1" 200 None 2025-09-26 16:42:07,784 - lava_callback - DEBUG - Processing node 68d6c1fb3c94e462c68816d7: baseline-arm-clabbe 2025-09-26 16:42:08,149 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 16:42:08,692 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 16:42:08,692 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-68d6c1fb3c94e462c68816d7/log.txt.gz 2025-09-26 16:42:08,699 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 16:42:09,038 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 16:42:09,039 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-68d6c1fb3c94e462c68816d7/lava_callback.json.gz 2025-09-26 16:42:09,080 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:42:09,096 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6c1fb3c94e462c68816d7 HTTP/1.1" 200 None 2025-09-26 16:42:09,097 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:42:09,112 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6bd793c94e462c68814e2 HTTP/1.1" 200 None 2025-09-26 16:42:09,114 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:42:09,153 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6c1fb3c94e462c68816d7 HTTP/1.1" 200 None 2025-09-26 16:42:09,154 - lava_callback - INFO - Completed processing callback for node 68d6c1fb3c94e462c68816d7 2025-09-26 16:42:09,468 - lava_callback - INFO - Received callback for node 68d6c1f33c94e462c68816aa INFO: 172.21.0.1:60672 - "POST /node/68d6c1f33c94e462c68816aa HTTP/1.0" 202 Accepted 2025-09-26 16:42:09,478 - lava_callback - INFO - Starting async job submit for node 68d6c1f33c94e462c68816aa 2025-09-26 16:42:09,526 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:42:09,557 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6c1f33c94e462c68816aa HTTP/1.1" 200 None 2025-09-26 16:42:09,557 - lava_callback - DEBUG - Processing node 68d6c1f33c94e462c68816aa: baseline-arm-baylibre 2025-09-26 16:42:09,914 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 16:42:10,227 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 16:42:10,228 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-baylibre-68d6c1f33c94e462c68816aa/log.txt.gz 2025-09-26 16:42:10,234 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 16:42:10,646 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 16:42:10,646 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-baylibre-68d6c1f33c94e462c68816aa/lava_callback.json.gz 2025-09-26 16:42:10,695 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:42:10,711 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6c1f33c94e462c68816aa HTTP/1.1" 200 None 2025-09-26 16:42:10,713 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:42:10,728 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6bd793c94e462c68814e2 HTTP/1.1" 200 None 2025-09-26 16:42:10,730 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:42:10,778 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6c1f33c94e462c68816aa HTTP/1.1" 200 None 2025-09-26 16:42:10,779 - lava_callback - INFO - Completed processing callback for node 68d6c1f33c94e462c68816aa 2025-09-26 16:42:13,466 - lava_callback - INFO - Received callback for node 68d6c1f63c94e462c68816bc INFO: 172.21.0.1:60678 - "POST /node/68d6c1f63c94e462c68816bc HTTP/1.0" 202 Accepted 2025-09-26 16:42:13,475 - lava_callback - INFO - Starting async job submit for node 68d6c1f63c94e462c68816bc 2025-09-26 16:42:13,520 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:42:13,546 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6c1f63c94e462c68816bc HTTP/1.1" 200 None 2025-09-26 16:42:13,546 - lava_callback - DEBUG - Processing node 68d6c1f63c94e462c68816bc: baseline-arm-broonie 2025-09-26 16:42:13,919 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 16:42:14,187 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 16:42:14,188 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-68d6c1f63c94e462c68816bc/log.txt.gz 2025-09-26 16:42:14,195 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 16:42:14,803 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 16:42:14,804 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-68d6c1f63c94e462c68816bc/lava_callback.json.gz 2025-09-26 16:42:14,844 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:42:14,872 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6c1f63c94e462c68816bc HTTP/1.1" 200 None 2025-09-26 16:42:14,873 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:42:14,889 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6bd793c94e462c68814e2 HTTP/1.1" 200 None 2025-09-26 16:42:14,890 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:42:14,938 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6c1f63c94e462c68816bc HTTP/1.1" 200 None 2025-09-26 16:42:14,939 - lava_callback - INFO - Completed processing callback for node 68d6c1f63c94e462c68816bc INFO: 172.21.0.1:60682 - "GET /api/metrics HTTP/1.0" 200 OK 2025-09-26 16:42:15,549 - lava_callback - INFO - Received callback for node 68d6c1f43c94e462c68816b0 INFO: 172.21.0.1:60686 - "POST /node/68d6c1f43c94e462c68816b0 HTTP/1.0" 202 Accepted 2025-09-26 16:42:15,559 - lava_callback - INFO - Starting async job submit for node 68d6c1f43c94e462c68816b0 2025-09-26 16:42:15,600 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:42:15,615 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6c1f43c94e462c68816b0 HTTP/1.1" 200 None 2025-09-26 16:42:15,615 - lava_callback - DEBUG - Processing node 68d6c1f43c94e462c68816b0: baseline-arm-baylibre 2025-09-26 16:42:16,073 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 16:42:16,444 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 16:42:16,445 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-baylibre-68d6c1f43c94e462c68816b0/log.txt.gz 2025-09-26 16:42:16,452 - lava_callback - INFO - Received callback for node 68d6c1ff3c94e462c68816f2 2025-09-26 16:42:16,461 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.21.0.1:60692 - "POST /node/68d6c1ff3c94e462c68816f2 HTTP/1.0" 202 Accepted 2025-09-26 16:42:16,462 - lava_callback - INFO - Starting async job submit for node 68d6c1ff3c94e462c68816f2 2025-09-26 16:42:16,511 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.21.0.1:60704 - "GET / HTTP/1.0" 200 OK 2025-09-26 16:42:16,540 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6c1ff3c94e462c68816f2 HTTP/1.1" 200 None 2025-09-26 16:42:16,541 - lava_callback - DEBUG - Processing node 68d6c1ff3c94e462c68816f2: baseline-arm-kontron 2025-09-26 16:42:16,853 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 16:42:17,186 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 16:42:17,187 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-baylibre-68d6c1f43c94e462c68816b0/lava_callback.json.gz 2025-09-26 16:42:17,227 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:42:17,249 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6c1f43c94e462c68816b0 HTTP/1.1" 200 None 2025-09-26 16:42:17,251 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:42:17,261 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 16:42:17,262 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-kontron-68d6c1ff3c94e462c68816f2/log.txt.gz 2025-09-26 16:42:17,267 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6bd793c94e462c68814e2 HTTP/1.1" 200 None 2025-09-26 16:42:17,269 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:42:17,270 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 16:42:17,316 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6c1f43c94e462c68816b0 HTTP/1.1" 200 None 2025-09-26 16:42:17,316 - lava_callback - INFO - Completed processing callback for node 68d6c1f43c94e462c68816b0 2025-09-26 16:42:17,712 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 16:42:17,712 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-kontron-68d6c1ff3c94e462c68816f2/lava_callback.json.gz 2025-09-26 16:42:17,761 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:42:17,782 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6c1ff3c94e462c68816f2 HTTP/1.1" 200 None 2025-09-26 16:42:17,783 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:42:17,802 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6bd793c94e462c68814e2 HTTP/1.1" 200 None 2025-09-26 16:42:17,803 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:42:17,833 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6c1ff3c94e462c68816f2 HTTP/1.1" 200 None 2025-09-26 16:42:17,834 - lava_callback - INFO - Completed processing callback for node 68d6c1ff3c94e462c68816f2 2025-09-26 16:42:20,017 - lava_callback - INFO - Received callback for node 68d6c1ea3c94e462c688167d INFO: 172.21.0.1:53296 - "POST /node/68d6c1ea3c94e462c688167d HTTP/1.0" 202 Accepted 2025-09-26 16:42:20,024 - lava_callback - INFO - Starting async job submit for node 68d6c1ea3c94e462c688167d 2025-09-26 16:42:20,049 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:42:20,065 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6c1ea3c94e462c688167d HTTP/1.1" 200 None 2025-09-26 16:42:20,065 - lava_callback - DEBUG - Processing node 68d6c1ea3c94e462c688167d: baseline-arm 2025-09-26 16:42:20,289 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 16:42:20,621 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 16:42:20,622 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-68d6c1ea3c94e462c688167d/log.txt.gz 2025-09-26 16:42:20,628 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 16:42:20,757 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 16:42:20,758 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-68d6c1ea3c94e462c688167d/lava_callback.json.gz 2025-09-26 16:42:20,781 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:42:20,796 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6c1ea3c94e462c688167d HTTP/1.1" 200 None 2025-09-26 16:42:20,797 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:42:20,814 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6bd793c94e462c68814e2 HTTP/1.1" 200 None 2025-09-26 16:42:20,815 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:42:20,863 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6c1ea3c94e462c688167d HTTP/1.1" 200 None 2025-09-26 16:42:20,864 - lava_callback - INFO - Completed processing callback for node 68d6c1ea3c94e462c688167d 2025-09-26 16:42:24,381 - lava_callback - INFO - Received callback for node 68d6bdae3c94e462c6881510 INFO: 172.21.0.1:53312 - "POST /node/68d6bdae3c94e462c6881510 HTTP/1.0" 202 Accepted 2025-09-26 16:42:24,392 - lava_callback - INFO - Starting async job submit for node 68d6bdae3c94e462c6881510 2025-09-26 16:42:24,455 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:42:24,469 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6bdae3c94e462c6881510 HTTP/1.1" 200 None 2025-09-26 16:42:24,469 - lava_callback - DEBUG - Processing node 68d6bdae3c94e462c6881510: baseline-x86 2025-09-26 16:42:25,330 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 16:42:25,595 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 16:42:25,596 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-68d6bdae3c94e462c6881510/log.txt.gz 2025-09-26 16:42:25,617 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 16:42:25,663 - lava_callback - INFO - Received callback for node 68d6c1ec3c94e462c6881689 INFO: 172.21.0.1:53316 - "POST /node/68d6c1ec3c94e462c6881689 HTTP/1.0" 202 Accepted 2025-09-26 16:42:25,672 - lava_callback - INFO - Starting async job submit for node 68d6c1ec3c94e462c6881689 2025-09-26 16:42:25,708 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:42:25,727 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6c1ec3c94e462c6881689 HTTP/1.1" 200 None 2025-09-26 16:42:25,728 - lava_callback - DEBUG - Processing node 68d6c1ec3c94e462c6881689: baseline-arm 2025-09-26 16:42:25,918 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 16:42:25,919 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-68d6bdae3c94e462c6881510/lava_callback.json.gz DEBUG: Setup failed for 68d6bdae3c94e462c6881510. Transitting job node result: incomplete -> fail 2025-09-26 16:42:26,254 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:42:26,294 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6bdae3c94e462c6881510 HTTP/1.1" 200 None 2025-09-26 16:42:26,296 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:42:26,299 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 16:42:26,314 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6b9f22f83206171a92b66 HTTP/1.1" 200 None 2025-09-26 16:42:26,315 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:42:26,359 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6bdae3c94e462c6881510 HTTP/1.1" 200 None 2025-09-26 16:42:26,360 - lava_callback - INFO - Completed processing callback for node 68d6bdae3c94e462c6881510 INFO: 172.21.0.1:53324 - "GET / HTTP/1.0" 200 OK 2025-09-26 16:42:26,723 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 16:42:26,724 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-68d6c1ec3c94e462c6881689/log.txt.gz 2025-09-26 16:42:26,731 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 16:42:26,945 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 16:42:26,946 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-68d6c1ec3c94e462c6881689/lava_callback.json.gz 2025-09-26 16:42:26,985 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:42:27,011 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6c1ec3c94e462c6881689 HTTP/1.1" 200 None 2025-09-26 16:42:27,013 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:42:27,037 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6bd793c94e462c68814e2 HTTP/1.1" 200 None 2025-09-26 16:42:27,039 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:42:27,090 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6c1ec3c94e462c6881689 HTTP/1.1" 200 None 2025-09-26 16:42:27,091 - lava_callback - INFO - Completed processing callback for node 68d6c1ec3c94e462c6881689 INFO: 172.21.0.1:53330 - "GET / HTTP/1.0" 200 OK 2025-09-26 16:42:32,887 - lava_callback - INFO - Received callback for node 68d6c1f13c94e462c688169e INFO: 172.21.0.1:60708 - "POST /node/68d6c1f13c94e462c688169e HTTP/1.0" 202 Accepted 2025-09-26 16:42:32,901 - lava_callback - INFO - Starting async job submit for node 68d6c1f13c94e462c688169e 2025-09-26 16:42:32,951 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:42:32,971 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6c1f13c94e462c688169e HTTP/1.1" 200 None 2025-09-26 16:42:32,972 - lava_callback - DEBUG - Processing node 68d6c1f13c94e462c688169e: baseline-arm-baylibre 2025-09-26 16:42:33,370 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 16:42:33,773 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 16:42:33,774 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-baylibre-68d6c1f13c94e462c688169e/log.txt.gz 2025-09-26 16:42:33,781 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 16:42:34,161 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 16:42:34,162 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-baylibre-68d6c1f13c94e462c688169e/lava_callback.json.gz 2025-09-26 16:42:34,203 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:42:34,219 - lava_callback - INFO - Received callback for node 68d485e8a5c2c05277bcd717 INFO: 172.21.0.1:60724 - "POST /node/68d485e8a5c2c05277bcd717 HTTP/1.0" 202 Accepted 2025-09-26 16:42:34,229 - lava_callback - INFO - Starting async job submit for node 68d485e8a5c2c05277bcd717 2025-09-26 16:42:34,231 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6c1f13c94e462c688169e HTTP/1.1" 200 None 2025-09-26 16:42:34,239 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:42:34,261 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:42:34,270 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6bd793c94e462c68814e2 HTTP/1.1" 200 None 2025-09-26 16:42:34,272 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:42:34,286 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d485e8a5c2c05277bcd717 HTTP/1.1" 200 None 2025-09-26 16:42:34,286 - lava_callback - DEBUG - Processing node 68d485e8a5c2c05277bcd717: baseline-arm64-broonie 2025-09-26 16:42:34,332 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6c1f13c94e462c688169e HTTP/1.1" 200 None 2025-09-26 16:42:34,341 - lava_callback - INFO - Completed processing callback for node 68d6c1f13c94e462c688169e 2025-09-26 16:42:34,817 - lava_callback - ERROR - Error processing callback for node 68d485e8a5c2c05277bcd717: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:60726 - "GET / HTTP/1.0" 200 OK 2025-09-26 16:42:36,607 - lava_callback - INFO - Received callback for node 68d6c2003c94e462c68816f9 INFO: 172.21.0.1:60730 - "POST /node/68d6c2003c94e462c68816f9 HTTP/1.0" 202 Accepted 2025-09-26 16:42:36,616 - lava_callback - INFO - Starting async job submit for node 68d6c2003c94e462c68816f9 2025-09-26 16:42:36,655 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:42:36,695 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6c2003c94e462c68816f9 HTTP/1.1" 200 None 2025-09-26 16:42:36,696 - lava_callback - DEBUG - Processing node 68d6c2003c94e462c68816f9: baseline-arm-kontron 2025-09-26 16:42:36,986 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 16:42:37,153 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 16:42:37,154 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-kontron-68d6c2003c94e462c68816f9/log.txt.gz 2025-09-26 16:42:37,159 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 16:42:37,300 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 16:42:37,301 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-kontron-68d6c2003c94e462c68816f9/lava_callback.json.gz 2025-09-26 16:42:37,325 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:42:37,337 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6c2003c94e462c68816f9 HTTP/1.1" 200 None 2025-09-26 16:42:37,338 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:42:37,355 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6bd793c94e462c68814e2 HTTP/1.1" 200 None 2025-09-26 16:42:37,356 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:42:37,405 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6c2003c94e462c68816f9 HTTP/1.1" 200 None 2025-09-26 16:42:37,406 - lava_callback - INFO - Completed processing callback for node 68d6c2003c94e462c68816f9 INFO: 172.21.0.1:51338 - "GET / HTTP/1.0" 200 OK 2025-09-26 16:42:54,529 - lava_callback - INFO - Received callback for node 68d6a85d2f83206171a9263f INFO: 172.21.0.1:33378 - "POST /node/68d6a85d2f83206171a9263f HTTP/1.0" 202 Accepted 2025-09-26 16:42:54,538 - lava_callback - INFO - Starting async job submit for node 68d6a85d2f83206171a9263f 2025-09-26 16:42:54,575 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:42:54,599 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6a85d2f83206171a9263f HTTP/1.1" 200 None 2025-09-26 16:42:54,600 - lava_callback - DEBUG - Processing node 68d6a85d2f83206171a9263f: baseline-arm64-broonie 2025-09-26 16:42:54,979 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.21.0.1:33390 - "GET / HTTP/1.0" 200 OK 2025-09-26 16:42:55,215 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 16:42:55,216 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-68d6a85d2f83206171a9263f/log.txt.gz 2025-09-26 16:42:55,224 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 16:42:55,392 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 16:42:55,393 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-68d6a85d2f83206171a9263f/lava_callback.json.gz 2025-09-26 16:42:55,418 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:42:55,434 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6a85d2f83206171a9263f HTTP/1.1" 200 None 2025-09-26 16:42:55,436 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:42:55,451 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d64d12a5c3ab168ecc7a3d HTTP/1.1" 200 None 2025-09-26 16:42:55,453 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:42:55,522 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6a85d2f83206171a9263f HTTP/1.1" 200 None 2025-09-26 16:42:55,523 - lava_callback - INFO - Completed processing callback for node 68d6a85d2f83206171a9263f INFO: 172.21.0.1:33394 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56892 - "GET / HTTP/1.0" 200 OK 2025-09-26 16:43:07,394 - lava_callback - INFO - Received callback for node 68d46f21a5c2c05277bcc54b INFO: 172.21.0.1:56900 - "POST /node/68d46f21a5c2c05277bcc54b HTTP/1.0" 202 Accepted 2025-09-26 16:43:07,401 - lava_callback - INFO - Starting async job submit for node 68d46f21a5c2c05277bcc54b 2025-09-26 16:43:07,426 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:43:07,438 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d46f21a5c2c05277bcc54b HTTP/1.1" 200 None 2025-09-26 16:43:07,438 - lava_callback - DEBUG - Processing node 68d46f21a5c2c05277bcc54b: baseline-arm 2025-09-26 16:43:07,715 - lava_callback - ERROR - Error processing callback for node 68d46f21a5c2c05277bcc54b: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:37922 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37934 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33272 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33274 - "GET / HTTP/1.0" 200 OK 2025-09-26 16:43:30,902 - lava_callback - INFO - Received callback for node 68d41b3da5c2c05277bc5cbb INFO: 172.21.0.1:38512 - "POST /node/68d41b3da5c2c05277bc5cbb HTTP/1.0" 202 Accepted 2025-09-26 16:43:30,908 - lava_callback - INFO - Starting async job submit for node 68d41b3da5c2c05277bc5cbb 2025-09-26 16:43:30,937 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:43:30,950 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d41b3da5c2c05277bc5cbb HTTP/1.1" 200 None 2025-09-26 16:43:30,950 - lava_callback - DEBUG - Processing node 68d41b3da5c2c05277bc5cbb: baseline-arm 2025-09-26 16:43:31,209 - lava_callback - ERROR - Error processing callback for node 68d41b3da5c2c05277bc5cbb: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:38516 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54028 - "GET / HTTP/1.0" 200 OK 2025-09-26 16:43:46,786 - lava_callback - INFO - Received callback for node 68d6c1fd3c94e462c68816e8 INFO: 172.21.0.1:54036 - "POST /node/68d6c1fd3c94e462c68816e8 HTTP/1.0" 202 Accepted 2025-09-26 16:43:46,792 - lava_callback - INFO - Starting async job submit for node 68d6c1fd3c94e462c68816e8 2025-09-26 16:43:46,813 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:43:46,825 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6c1fd3c94e462c68816e8 HTTP/1.1" 200 None 2025-09-26 16:43:46,826 - lava_callback - DEBUG - Processing node 68d6c1fd3c94e462c68816e8: baseline-arm-clabbe 2025-09-26 16:43:47,035 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 16:43:47,181 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 16:43:47,182 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-68d6c1fd3c94e462c68816e8/log.txt.gz 2025-09-26 16:43:47,188 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 16:43:47,322 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 16:43:47,323 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-68d6c1fd3c94e462c68816e8/lava_callback.json.gz 2025-09-26 16:43:47,344 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:43:47,358 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6c1fd3c94e462c68816e8 HTTP/1.1" 200 None 2025-09-26 16:43:47,359 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:43:47,371 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6bd793c94e462c68814e2 HTTP/1.1" 200 None 2025-09-26 16:43:47,372 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:43:47,398 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6c1fd3c94e462c68816e8 HTTP/1.1" 200 None 2025-09-26 16:43:47,399 - lava_callback - INFO - Completed processing callback for node 68d6c1fd3c94e462c68816e8 INFO: 172.21.0.1:60806 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60808 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45832 - "GET / HTTP/1.0" 200 OK 2025-09-26 16:44:14,622 - lava_callback - INFO - Received callback for node 68d6c2073c94e462c6881717 INFO: 172.21.0.1:54866 - "POST /node/68d6c2073c94e462c6881717 HTTP/1.0" 202 Accepted 2025-09-26 16:44:14,629 - lava_callback - INFO - Starting async job submit for node 68d6c2073c94e462c6881717 2025-09-26 16:44:14,680 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:44:14,692 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6c2073c94e462c6881717 HTTP/1.1" 200 None 2025-09-26 16:44:14,693 - lava_callback - DEBUG - Processing node 68d6c2073c94e462c6881717: ltp-dio 2025-09-26 16:44:15,065 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.21.0.1:54872 - "GET /api/metrics HTTP/1.0" 200 OK 2025-09-26 16:44:15,187 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 16:44:15,188 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-dio-68d6c2073c94e462c6881717/log.txt.gz 2025-09-26 16:44:15,197 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 16:44:15,341 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 16:44:15,342 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-dio-68d6c2073c94e462c6881717/lava_callback.json.gz 2025-09-26 16:44:15,391 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:44:15,406 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6c2073c94e462c6881717 HTTP/1.1" 200 None 2025-09-26 16:44:15,407 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:44:15,419 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6bd793c94e462c68814e2 HTTP/1.1" 200 None 2025-09-26 16:44:15,420 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:44:15,460 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6c2073c94e462c6881717 HTTP/1.1" 200 None 2025-09-26 16:44:15,461 - lava_callback - INFO - Completed processing callback for node 68d6c2073c94e462c6881717 INFO: 172.21.0.1:54874 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57072 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57074 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55440 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36176 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52202 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52206 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57108 - "GET / HTTP/1.0" 200 OK 2025-09-26 16:45:13,815 - lava_callback - INFO - Received callback for node 68d6ae3e2f83206171a92896 INFO: 172.21.0.1:41554 - "POST /node/68d6ae3e2f83206171a92896 HTTP/1.0" 202 Accepted 2025-09-26 16:45:13,821 - lava_callback - INFO - Starting async job submit for node 68d6ae3e2f83206171a92896 2025-09-26 16:45:13,843 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:45:13,860 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6ae3e2f83206171a92896 HTTP/1.1" 200 None 2025-09-26 16:45:13,860 - lava_callback - DEBUG - Processing node 68d6ae3e2f83206171a92896: baseline-arm64-broonie 2025-09-26 16:45:14,078 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 16:45:14,197 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 16:45:14,198 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-68d6ae3e2f83206171a92896/log.txt.gz 2025-09-26 16:45:14,204 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 16:45:14,346 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 16:45:14,346 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-68d6ae3e2f83206171a92896/lava_callback.json.gz 2025-09-26 16:45:14,368 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:45:14,381 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6ae3e2f83206171a92896 HTTP/1.1" 200 None 2025-09-26 16:45:14,381 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:45:14,393 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d651ddcca86fb17472aa24 HTTP/1.1" 200 None 2025-09-26 16:45:14,394 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:45:14,423 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6ae3e2f83206171a92896 HTTP/1.1" 200 None 2025-09-26 16:45:14,424 - lava_callback - INFO - Completed processing callback for node 68d6ae3e2f83206171a92896 INFO: 172.21.0.1:41558 - "GET /api/metrics HTTP/1.0" 200 OK 2025-09-26 16:45:16,060 - lava_callback - INFO - Received callback for node 68d46f22a5c2c05277bcc54e INFO: 172.21.0.1:41564 - "POST /node/68d46f22a5c2c05277bcc54e HTTP/1.0" 202 Accepted 2025-09-26 16:45:16,068 - lava_callback - INFO - Starting async job submit for node 68d46f22a5c2c05277bcc54e 2025-09-26 16:45:16,102 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:45:16,117 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d46f22a5c2c05277bcc54e HTTP/1.1" 200 None 2025-09-26 16:45:16,117 - lava_callback - DEBUG - Processing node 68d46f22a5c2c05277bcc54e: baseline-arm INFO: 172.21.0.1:41574 - "GET / HTTP/1.0" 200 OK 2025-09-26 16:45:16,522 - lava_callback - ERROR - Error processing callback for node 68d46f22a5c2c05277bcc54e: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None 2025-09-26 16:45:25,846 - lava_callback - INFO - Received callback for node 68d6c1fe3c94e462c68816ec INFO: 172.21.0.1:60634 - "POST /node/68d6c1fe3c94e462c68816ec HTTP/1.0" 202 Accepted 2025-09-26 16:45:25,852 - lava_callback - INFO - Starting async job submit for node 68d6c1fe3c94e462c68816ec 2025-09-26 16:45:25,875 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:45:25,913 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6c1fe3c94e462c68816ec HTTP/1.1" 200 None 2025-09-26 16:45:25,913 - lava_callback - DEBUG - Processing node 68d6c1fe3c94e462c68816ec: baseline-arm-clabbe 2025-09-26 16:45:26,141 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 16:45:26,273 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 16:45:26,273 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-68d6c1fe3c94e462c68816ec/log.txt.gz 2025-09-26 16:45:26,280 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 16:45:26,411 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 16:45:26,412 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-68d6c1fe3c94e462c68816ec/lava_callback.json.gz 2025-09-26 16:45:26,436 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:45:26,448 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6c1fe3c94e462c68816ec HTTP/1.1" 200 None 2025-09-26 16:45:26,449 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:45:26,469 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6bd793c94e462c68814e2 HTTP/1.1" 200 None 2025-09-26 16:45:26,470 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:45:26,498 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6c1fe3c94e462c68816ec HTTP/1.1" 200 None 2025-09-26 16:45:26,499 - lava_callback - INFO - Completed processing callback for node 68d6c1fe3c94e462c68816ec INFO: 172.21.0.1:60646 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60662 - "GET / HTTP/1.0" 200 OK 2025-09-26 16:45:32,531 - lava_callback - INFO - Received callback for node 68d6bdb23c94e462c688151d INFO: 172.21.0.1:43806 - "POST /node/68d6bdb23c94e462c688151d HTTP/1.0" 202 Accepted 2025-09-26 16:45:32,539 - lava_callback - INFO - Starting async job submit for node 68d6bdb23c94e462c688151d 2025-09-26 16:45:32,560 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:45:32,572 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6bdb23c94e462c688151d HTTP/1.1" 200 None 2025-09-26 16:45:32,573 - lava_callback - DEBUG - Processing node 68d6bdb23c94e462c688151d: blktests-ddp-x86 2025-09-26 16:45:33,354 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 16:45:33,494 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 16:45:33,495 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/blktests-ddp-x86-68d6bdb23c94e462c688151d/log.txt.gz 2025-09-26 16:45:33,514 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 16:45:33,706 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 16:45:33,707 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/blktests-ddp-x86-68d6bdb23c94e462c688151d/lava_callback.json.gz 2025-09-26 16:45:33,749 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:45:33,761 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6bdb23c94e462c688151d HTTP/1.1" 200 None 2025-09-26 16:45:33,762 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:45:33,773 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6b9f22f83206171a92b66 HTTP/1.1" 200 None 2025-09-26 16:45:33,774 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:45:33,837 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6bdb23c94e462c688151d HTTP/1.1" 200 None 2025-09-26 16:45:33,837 - lava_callback - INFO - Completed processing callback for node 68d6bdb23c94e462c688151d INFO: 172.21.0.1:43810 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52428 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49776 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49780 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52330 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51842 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51854 - "GET / HTTP/1.0" 200 OK 2025-09-26 16:46:17,586 - lava_callback - INFO - Received callback for node 68d6bf013c94e462c6881554 INFO: 172.21.0.1:51864 - "POST /node/68d6bf013c94e462c6881554 HTTP/1.0" 202 Accepted 2025-09-26 16:46:17,592 - lava_callback - INFO - Starting async job submit for node 68d6bf013c94e462c6881554 2025-09-26 16:46:17,623 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:46:17,646 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6bf013c94e462c6881554 HTTP/1.1" 200 None 2025-09-26 16:46:17,646 - lava_callback - DEBUG - Processing node 68d6bf013c94e462c6881554: baseline-arm 2025-09-26 16:46:17,962 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 16:46:18,127 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 16:46:18,127 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-68d6bf013c94e462c6881554/log.txt.gz 2025-09-26 16:46:18,136 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 16:46:18,261 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 16:46:18,262 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-68d6bf013c94e462c6881554/lava_callback.json.gz DEBUG: Setup failed for 68d6bf013c94e462c6881554. Transitting job node result: incomplete -> fail 2025-09-26 16:46:18,324 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:46:18,339 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6bf013c94e462c6881554 HTTP/1.1" 200 None 2025-09-26 16:46:18,340 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:46:18,352 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6b9da2f83206171a92b45 HTTP/1.1" 200 None 2025-09-26 16:46:18,353 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:46:18,405 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6bf013c94e462c6881554 HTTP/1.1" 200 None 2025-09-26 16:46:18,406 - lava_callback - INFO - Completed processing callback for node 68d6bf013c94e462c6881554 2025-09-26 16:46:21,743 - lava_callback - INFO - Received callback for node 68d42378a5c2c05277bc6ed9 INFO: 172.21.0.1:45870 - "POST /node/68d42378a5c2c05277bc6ed9 HTTP/1.0" 202 Accepted 2025-09-26 16:46:21,749 - lava_callback - INFO - Starting async job submit for node 68d42378a5c2c05277bc6ed9 2025-09-26 16:46:21,771 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:46:21,788 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d42378a5c2c05277bc6ed9 HTTP/1.1" 200 None 2025-09-26 16:46:21,788 - lava_callback - DEBUG - Processing node 68d42378a5c2c05277bc6ed9: baseline-arm-broonie 2025-09-26 16:46:21,993 - lava_callback - ERROR - Error processing callback for node 68d42378a5c2c05277bc6ed9: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:45884 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45886 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55680 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42878 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48268 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48284 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58154 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36722 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36728 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55040 - "GET / HTTP/1.0" 200 OK 2025-09-26 16:47:26,864 - lava_callback - INFO - Received callback for node 68d46fbaa5c2c05277bcc623 INFO: 172.21.0.1:55056 - "POST /node/68d46fbaa5c2c05277bcc623 HTTP/1.0" 202 Accepted 2025-09-26 16:47:26,878 - lava_callback - INFO - Starting async job submit for node 68d46fbaa5c2c05277bcc623 2025-09-26 16:47:26,928 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:47:26,947 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d46fbaa5c2c05277bcc623 HTTP/1.1" 200 None 2025-09-26 16:47:26,948 - lava_callback - DEBUG - Processing node 68d46fbaa5c2c05277bcc623: baseline-arm 2025-09-26 16:47:27,367 - lava_callback - ERROR - Error processing callback for node 68d46fbaa5c2c05277bcc623: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:55072 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34156 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44610 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48332 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48340 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48426 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50876 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50878 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51146 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51156 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52754 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33070 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33442 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33458 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49910 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44136 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44138 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41294 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41308 - "GET / HTTP/1.0" 200 OK 2025-09-26 16:49:29,120 - lava_callback - INFO - Received callback for node 68d46fbba5c2c05277bcc626 INFO: 172.21.0.1:41316 - "POST /node/68d46fbba5c2c05277bcc626 HTTP/1.0" 202 Accepted 2025-09-26 16:49:29,130 - lava_callback - INFO - Starting async job submit for node 68d46fbba5c2c05277bcc626 2025-09-26 16:49:29,184 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:49:29,224 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d46fbba5c2c05277bcc626 HTTP/1.1" 200 None 2025-09-26 16:49:29,225 - lava_callback - DEBUG - Processing node 68d46fbba5c2c05277bcc626: baseline-arm 2025-09-26 16:49:29,631 - lava_callback - ERROR - Error processing callback for node 68d46fbba5c2c05277bcc626: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:37730 - "GET / HTTP/1.0" 200 OK 2025-09-26 16:49:37,423 - lava_callback - INFO - Received callback for node 68d6bd563c94e462c68814b4 INFO: 172.21.0.1:37738 - "POST /node/68d6bd563c94e462c68814b4 HTTP/1.0" 202 Accepted 2025-09-26 16:49:37,442 - lava_callback - INFO - Starting async job submit for node 68d6bd563c94e462c68814b4 2025-09-26 16:49:37,556 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:49:37,592 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6bd563c94e462c68814b4 HTTP/1.1" 200 None 2025-09-26 16:49:37,593 - lava_callback - DEBUG - Processing node 68d6bd563c94e462c68814b4: baseline-arm64 2025-09-26 16:49:39,489 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 16:49:40,019 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 16:49:40,019 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6bd563c94e462c68814b4/log.txt.gz 2025-09-26 16:49:40,044 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 16:49:40,441 - lava_callback - INFO - Received callback for node 68d4211ca5c2c05277bc6c9e INFO: 172.21.0.1:53444 - "POST /node/68d4211ca5c2c05277bc6c9e HTTP/1.0" 202 Accepted 2025-09-26 16:49:40,450 - lava_callback - INFO - Starting async job submit for node 68d4211ca5c2c05277bc6c9e 2025-09-26 16:49:40,453 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:49:40,500 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4211ca5c2c05277bc6c9e HTTP/1.1" 200 None 2025-09-26 16:49:40,501 - lava_callback - DEBUG - Processing node 68d4211ca5c2c05277bc6c9e: baseline-arm64-baylibre 2025-09-26 16:49:40,501 - lava_callback - ERROR - Error processing callback for node 68d4211ca5c2c05277bc6c9e: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 16:49:40,534 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 16:49:40,535 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6bd563c94e462c68814b4/lava_callback.json.gz DEBUG: Setup failed for 68d6bd563c94e462c68814b4. Transitting job node result: incomplete -> fail 2025-09-26 16:49:40,678 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:49:40,695 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6bd563c94e462c68814b4 HTTP/1.1" 200 None 2025-09-26 16:49:40,697 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:49:40,713 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d651ddcca86fb17472aa24 HTTP/1.1" 200 None 2025-09-26 16:49:40,714 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:49:40,749 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6bd563c94e462c68814b4 HTTP/1.1" 200 None 2025-09-26 16:49:40,750 - lava_callback - INFO - Completed processing callback for node 68d6bd563c94e462c68814b4 INFO: 172.21.0.1:53456 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59996 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60004 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40630 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40992 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41006 - "GET / HTTP/1.0" 200 OK 2025-09-26 16:50:20,465 - lava_callback - INFO - Received callback for node 68d42143a5c2c05277bc6cd4 INFO: 172.21.0.1:34252 - "POST /node/68d42143a5c2c05277bc6cd4 HTTP/1.0" 202 Accepted 2025-09-26 16:50:20,473 - lava_callback - INFO - Starting async job submit for node 68d42143a5c2c05277bc6cd4 2025-09-26 16:50:20,477 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:50:20,493 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d42143a5c2c05277bc6cd4 HTTP/1.1" 200 None 2025-09-26 16:50:20,493 - lava_callback - DEBUG - Processing node 68d42143a5c2c05277bc6cd4: baseline-arm64-baylibre 2025-09-26 16:50:20,494 - lava_callback - ERROR - Error processing callback for node 68d42143a5c2c05277bc6cd4: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:34268 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34276 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55168 - "GET / HTTP/1.0" 200 OK 2025-09-26 16:50:39,132 - lava_callback - INFO - Received callback for node 68d6bd563c94e462c68814b7 INFO: 172.21.0.1:55176 - "POST /node/68d6bd563c94e462c68814b7 HTTP/1.0" 202 Accepted 2025-09-26 16:50:39,149 - lava_callback - INFO - Starting async job submit for node 68d6bd563c94e462c68814b7 2025-09-26 16:50:39,236 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:50:39,252 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6bd563c94e462c68814b7 HTTP/1.1" 200 None 2025-09-26 16:50:39,252 - lava_callback - DEBUG - Processing node 68d6bd563c94e462c68814b7: baseline-arm64 2025-09-26 16:50:41,203 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 16:50:41,802 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 16:50:41,803 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6bd563c94e462c68814b7/log.txt.gz 2025-09-26 16:50:41,834 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 16:50:42,053 - lava_callback - INFO - Received callback for node 68d4238aa5c2c05277bc6f2d INFO: 172.21.0.1:44400 - "POST /node/68d4238aa5c2c05277bc6f2d HTTP/1.0" 202 Accepted 2025-09-26 16:50:42,072 - lava_callback - INFO - Starting async job submit for node 68d4238aa5c2c05277bc6f2d 2025-09-26 16:50:42,126 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:50:42,150 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4238aa5c2c05277bc6f2d HTTP/1.1" 200 None 2025-09-26 16:50:42,150 - lava_callback - DEBUG - Processing node 68d4238aa5c2c05277bc6f2d: ltp-cap-bounds 2025-09-26 16:50:42,524 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 16:50:42,525 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6bd563c94e462c68814b7/lava_callback.json.gz DEBUG: Setup failed for 68d6bd563c94e462c68814b7. Transitting job node result: incomplete -> fail 2025-09-26 16:50:42,870 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:50:42,918 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6bd563c94e462c68814b7 HTTP/1.1" 200 None 2025-09-26 16:50:42,925 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:50:42,973 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d651ddcca86fb17472aa24 HTTP/1.1" 200 None 2025-09-26 16:50:42,975 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:50:42,978 - lava_callback - ERROR - Error processing callback for node 68d4238aa5c2c05277bc6f2d: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None 2025-09-26 16:50:43,022 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6bd563c94e462c68814b7 HTTP/1.1" 200 None 2025-09-26 16:50:43,023 - lava_callback - INFO - Completed processing callback for node 68d6bd563c94e462c68814b7 INFO: 172.21.0.1:44416 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50808 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50812 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58666 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33308 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33316 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41038 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41042 - "GET / HTTP/1.0" 200 OK 2025-09-26 16:51:29,499 - lava_callback - INFO - Received callback for node 68d42185a5c2c05277bc6d20 INFO: 172.21.0.1:39614 - "POST /node/68d42185a5c2c05277bc6d20 HTTP/1.0" 202 Accepted 2025-09-26 16:51:29,507 - lava_callback - INFO - Starting async job submit for node 68d42185a5c2c05277bc6d20 2025-09-26 16:51:29,509 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:51:29,524 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d42185a5c2c05277bc6d20 HTTP/1.1" 200 None 2025-09-26 16:51:29,525 - lava_callback - DEBUG - Processing node 68d42185a5c2c05277bc6d20: baseline-arm64-clabbe 2025-09-26 16:51:29,525 - lava_callback - ERROR - Error processing callback for node 68d42185a5c2c05277bc6d20: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 16:51:31,398 - lava_callback - INFO - Received callback for node 68d6bd643c94e462c68814c3 INFO: 172.21.0.1:39622 - "POST /node/68d6bd643c94e462c68814c3 HTTP/1.0" 202 Accepted 2025-09-26 16:51:31,412 - lava_callback - INFO - Starting async job submit for node 68d6bd643c94e462c68814c3 2025-09-26 16:51:31,497 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:51:31,522 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6bd643c94e462c68814c3 HTTP/1.1" 200 None 2025-09-26 16:51:31,523 - lava_callback - DEBUG - Processing node 68d6bd643c94e462c68814c3: baseline-arm64 2025-09-26 16:51:33,500 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 16:51:33,891 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 16:51:33,892 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6bd643c94e462c68814c3/log.txt.gz 2025-09-26 16:51:33,917 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 16:51:34,335 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 16:51:34,335 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6bd643c94e462c68814c3/lava_callback.json.gz DEBUG: Setup failed for 68d6bd643c94e462c68814c3. Transitting job node result: incomplete -> fail 2025-09-26 16:51:34,484 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:51:34,534 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6bd643c94e462c68814c3 HTTP/1.1" 200 None 2025-09-26 16:51:34,535 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:51:34,549 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d651ddcca86fb17472aa24 HTTP/1.1" 200 None 2025-09-26 16:51:34,551 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:51:34,593 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6bd643c94e462c68814c3 HTTP/1.1" 200 None 2025-09-26 16:51:34,594 - lava_callback - INFO - Completed processing callback for node 68d6bd643c94e462c68814c3 INFO: 172.21.0.1:39636 - "GET / HTTP/1.0" 200 OK 2025-09-26 16:51:46,407 - lava_callback - INFO - Received callback for node 68d46fcba5c2c05277bcc649 INFO: 172.21.0.1:33512 - "POST /node/68d46fcba5c2c05277bcc649 HTTP/1.0" 202 Accepted 2025-09-26 16:51:46,419 - lava_callback - INFO - Starting async job submit for node 68d46fcba5c2c05277bcc649 2025-09-26 16:51:46,462 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:51:46,493 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d46fcba5c2c05277bcc649 HTTP/1.1" 200 None 2025-09-26 16:51:46,494 - lava_callback - DEBUG - Processing node 68d46fcba5c2c05277bcc649: baseline-arm INFO: 172.21.0.1:33522 - "GET / HTTP/1.0" 200 OK 2025-09-26 16:51:46,923 - lava_callback - ERROR - Error processing callback for node 68d46fcba5c2c05277bcc649: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None 2025-09-26 16:51:49,477 - lava_callback - INFO - Received callback for node 68d42191a5c2c05277bc6d29 INFO: 172.21.0.1:36166 - "POST /node/68d42191a5c2c05277bc6d29 HTTP/1.0" 202 Accepted 2025-09-26 16:51:49,490 - lava_callback - INFO - Starting async job submit for node 68d42191a5c2c05277bc6d29 2025-09-26 16:51:49,491 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:51:49,524 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d42191a5c2c05277bc6d29 HTTP/1.1" 200 None 2025-09-26 16:51:49,525 - lava_callback - DEBUG - Processing node 68d42191a5c2c05277bc6d29: baseline-arm64-clabbe 2025-09-26 16:51:49,525 - lava_callback - ERROR - Error processing callback for node 68d42191a5c2c05277bc6d29: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 16:51:50,157 - lava_callback - INFO - Received callback for node 68d42194a5c2c05277bc6d2c INFO: 172.21.0.1:36180 - "POST /node/68d42194a5c2c05277bc6d2c HTTP/1.0" 202 Accepted 2025-09-26 16:51:50,165 - lava_callback - INFO - Starting async job submit for node 68d42194a5c2c05277bc6d2c 2025-09-26 16:51:50,167 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:51:50,227 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d42194a5c2c05277bc6d2c HTTP/1.1" 200 None 2025-09-26 16:51:50,227 - lava_callback - DEBUG - Processing node 68d42194a5c2c05277bc6d2c: baseline-arm64-clabbe 2025-09-26 16:51:50,227 - lava_callback - ERROR - Error processing callback for node 68d42194a5c2c05277bc6d2c: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 16:51:50,810 - lava_callback - INFO - Received callback for node 68d4219aa5c2c05277bc6d2f INFO: 172.21.0.1:36196 - "POST /node/68d4219aa5c2c05277bc6d2f HTTP/1.0" 202 Accepted 2025-09-26 16:51:50,819 - lava_callback - INFO - Starting async job submit for node 68d4219aa5c2c05277bc6d2f 2025-09-26 16:51:50,820 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:51:50,855 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4219aa5c2c05277bc6d2f HTTP/1.1" 200 None 2025-09-26 16:51:50,855 - lava_callback - DEBUG - Processing node 68d4219aa5c2c05277bc6d2f: baseline-arm64-clabbe 2025-09-26 16:51:50,855 - lava_callback - ERROR - Error processing callback for node 68d4219aa5c2c05277bc6d2f: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 16:51:51,241 - lava_callback - INFO - Received callback for node 68d6bd643c94e462c68814c9 INFO: 172.21.0.1:36204 - "POST /node/68d6bd643c94e462c68814c9 HTTP/1.0" 202 Accepted 2025-09-26 16:51:51,257 - lava_callback - INFO - Starting async job submit for node 68d6bd643c94e462c68814c9 2025-09-26 16:51:51,330 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:51:51,356 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6bd643c94e462c68814c9 HTTP/1.1" 200 None 2025-09-26 16:51:51,357 - lava_callback - DEBUG - Processing node 68d6bd643c94e462c68814c9: baseline-arm64 2025-09-26 16:51:51,474 - lava_callback - INFO - Received callback for node 68d4219fa5c2c05277bc6d32 INFO: 172.21.0.1:36216 - "POST /node/68d4219fa5c2c05277bc6d32 HTTP/1.0" 202 Accepted 2025-09-26 16:51:51,483 - lava_callback - INFO - Starting async job submit for node 68d4219fa5c2c05277bc6d32 2025-09-26 16:51:51,511 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:51:51,569 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4219fa5c2c05277bc6d32 HTTP/1.1" 200 None 2025-09-26 16:51:51,575 - lava_callback - DEBUG - Processing node 68d4219fa5c2c05277bc6d32: baseline-arm64-clabbe 2025-09-26 16:51:51,580 - lava_callback - ERROR - Error processing callback for node 68d4219fa5c2c05277bc6d32: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 16:51:53,147 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 16:51:53,687 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 16:51:53,688 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6bd643c94e462c68814c9/log.txt.gz 2025-09-26 16:51:53,710 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 16:51:54,208 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 16:51:54,209 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6bd643c94e462c68814c9/lava_callback.json.gz DEBUG: Setup failed for 68d6bd643c94e462c68814c9. Transitting job node result: incomplete -> fail 2025-09-26 16:51:54,341 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:51:54,370 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6bd643c94e462c68814c9 HTTP/1.1" 200 None 2025-09-26 16:51:54,371 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:51:54,393 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d651ddcca86fb17472aa24 HTTP/1.1" 200 None 2025-09-26 16:51:54,395 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:51:54,446 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6bd643c94e462c68814c9 HTTP/1.1" 200 None 2025-09-26 16:51:54,447 - lava_callback - INFO - Completed processing callback for node 68d6bd643c94e462c68814c9 INFO: 172.21.0.1:36230 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36236 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47036 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53920 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53926 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53442 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53452 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42558 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47424 - "GET / HTTP/1.0" 200 OK 2025-09-26 16:52:53,721 - lava_callback - INFO - Received callback for node 68d69a652f83206171a92229 INFO: 172.21.0.1:59338 - "POST /node/68d69a652f83206171a92229 HTTP/1.0" 202 Accepted 2025-09-26 16:52:53,728 - lava_callback - INFO - Starting async job submit for node 68d69a652f83206171a92229 2025-09-26 16:52:53,769 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:52:53,808 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d69a652f83206171a92229 HTTP/1.1" 200 None 2025-09-26 16:52:53,809 - lava_callback - DEBUG - Processing node 68d69a652f83206171a92229: baseline-arm64 2025-09-26 16:52:54,820 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 16:52:54,964 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 16:52:54,965 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d69a652f83206171a92229/log.txt.gz 2025-09-26 16:52:54,985 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 16:52:55,162 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 16:52:55,163 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d69a652f83206171a92229/lava_callback.json.gz DEBUG: Setup failed for 68d69a652f83206171a92229. Transitting job node result: incomplete -> fail 2025-09-26 16:52:55,250 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:52:55,263 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d69a652f83206171a92229 HTTP/1.1" 200 None 2025-09-26 16:52:55,264 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:52:55,285 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d62c44c1bc82a30b37a916 HTTP/1.1" 200 None 2025-09-26 16:52:55,286 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:52:55,316 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d69a652f83206171a92229 HTTP/1.1" 200 None 2025-09-26 16:52:55,317 - lava_callback - INFO - Completed processing callback for node 68d69a652f83206171a92229 INFO: 172.21.0.1:59354 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59366 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59224 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59104 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59114 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36586 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36592 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48670 - "GET / HTTP/1.0" 200 OK 2025-09-26 16:53:42,349 - lava_callback - INFO - Received callback for node 68d4238ba5c2c05277bc6f32 INFO: 172.21.0.1:46798 - "POST /node/68d4238ba5c2c05277bc6f32 HTTP/1.0" 202 Accepted 2025-09-26 16:53:42,357 - lava_callback - INFO - Starting async job submit for node 68d4238ba5c2c05277bc6f32 2025-09-26 16:53:42,460 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:53:42,474 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4238ba5c2c05277bc6f32 HTTP/1.1" 200 None 2025-09-26 16:53:42,474 - lava_callback - DEBUG - Processing node 68d4238ba5c2c05277bc6f32: ltp-containers 2025-09-26 16:53:43,257 - lava_callback - ERROR - Error processing callback for node 68d4238ba5c2c05277bc6f32: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:46802 - "GET / HTTP/1.0" 200 OK 2025-09-26 16:53:48,691 - lava_callback - INFO - Received callback for node 68d46fcda5c2c05277bcc64c INFO: 172.21.0.1:46812 - "POST /node/68d46fcda5c2c05277bcc64c HTTP/1.0" 202 Accepted 2025-09-26 16:53:48,697 - lava_callback - INFO - Starting async job submit for node 68d46fcda5c2c05277bcc64c 2025-09-26 16:53:48,717 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:53:48,764 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d46fcda5c2c05277bcc64c HTTP/1.1" 200 None 2025-09-26 16:53:48,765 - lava_callback - DEBUG - Processing node 68d46fcda5c2c05277bcc64c: baseline-arm 2025-09-26 16:53:49,006 - lava_callback - ERROR - Error processing callback for node 68d46fcda5c2c05277bcc64c: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:38878 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38888 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48334 - "GET / HTTP/1.0" 200 OK 2025-09-26 16:54:13,980 - lava_callback - INFO - Received callback for node 68d6b5182f83206171a92a70 INFO: 172.21.0.1:33572 - "POST /node/68d6b5182f83206171a92a70 HTTP/1.0" 202 Accepted 2025-09-26 16:54:13,993 - lava_callback - INFO - Starting async job submit for node 68d6b5182f83206171a92a70 2025-09-26 16:54:14,101 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:54:14,117 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6b5182f83206171a92a70 HTTP/1.1" 200 None 2025-09-26 16:54:14,118 - lava_callback - DEBUG - Processing node 68d6b5182f83206171a92a70: ltp-smoketest 2025-09-26 16:54:14,637 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 16:54:14,759 - lava_callback - INFO - Received callback for node 68d6bd513c94e462c68814ae INFO: 172.21.0.1:33580 - "POST /node/68d6bd513c94e462c68814ae HTTP/1.0" 202 Accepted 2025-09-26 16:54:14,767 - lava_callback - INFO - Starting async job submit for node 68d6bd513c94e462c68814ae 2025-09-26 16:54:14,778 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 16:54:14,810 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:54:14,810 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoketest-68d6b5182f83206171a92a70/log.txt.gz 2025-09-26 16:54:14,822 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 16:54:14,822 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6bd513c94e462c68814ae HTTP/1.1" 200 None 2025-09-26 16:54:14,823 - lava_callback - DEBUG - Processing node 68d6bd513c94e462c68814ae: baseline-arm64 2025-09-26 16:54:15,031 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 16:54:15,067 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoketest-68d6b5182f83206171a92a70/lava_callback.json.gz INFO: 172.21.0.1:33584 - "GET /api/metrics HTTP/1.0" 200 OK 2025-09-26 16:54:15,350 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:54:15,418 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6b5182f83206171a92a70 HTTP/1.1" 200 None 2025-09-26 16:54:15,440 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:54:15,509 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d651d5cca86fb17472aa18 HTTP/1.1" 200 None 2025-09-26 16:54:15,530 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:54:15,640 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6b5182f83206171a92a70 HTTP/1.1" 200 None 2025-09-26 16:54:15,641 - lava_callback - INFO - Completed processing callback for node 68d6b5182f83206171a92a70 2025-09-26 16:54:16,231 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 16:54:16,351 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 16:54:16,352 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6bd513c94e462c68814ae/log.txt.gz 2025-09-26 16:54:16,375 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 16:54:16,532 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 16:54:16,532 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6bd513c94e462c68814ae/lava_callback.json.gz INFO: 172.21.0.1:33586 - "GET / HTTP/1.0" 200 OK DEBUG: Setup failed for 68d6bd513c94e462c68814ae. Transitting job node result: incomplete -> fail 2025-09-26 16:54:16,613 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:54:16,625 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6bd513c94e462c68814ae HTTP/1.1" 200 None 2025-09-26 16:54:16,626 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:54:16,637 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d651ddcca86fb17472aa24 HTTP/1.1" 200 None 2025-09-26 16:54:16,638 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:54:16,662 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6bd513c94e462c68814ae HTTP/1.1" 200 None 2025-09-26 16:54:16,662 - lava_callback - INFO - Completed processing callback for node 68d6bd513c94e462c68814ae INFO: 172.21.0.1:58430 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58446 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48238 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54334 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55672 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55676 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38860 - "GET / HTTP/1.0" 200 OK 2025-09-26 16:55:07,372 - lava_callback - INFO - Received callback for node 68d42426a5c2c05277bc7004 INFO: 172.21.0.1:38876 - "POST /node/68d42426a5c2c05277bc7004 HTTP/1.0" 202 Accepted 2025-09-26 16:55:07,381 - lava_callback - INFO - Starting async job submit for node 68d42426a5c2c05277bc7004 2025-09-26 16:55:07,432 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:55:07,692 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d42426a5c2c05277bc7004 HTTP/1.1" 200 None 2025-09-26 16:55:07,692 - lava_callback - DEBUG - Processing node 68d42426a5c2c05277bc7004: baseline-arm64 2025-09-26 16:55:08,274 - lava_callback - ERROR - Error processing callback for node 68d42426a5c2c05277bc7004: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:47162 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47176 - "GET / HTTP/1.0" 200 OK 2025-09-26 16:55:25,366 - lava_callback - INFO - Received callback for node 68d42424a5c2c05277bc6ffc INFO: 172.21.0.1:44108 - "POST /node/68d42424a5c2c05277bc6ffc HTTP/1.0" 202 Accepted 2025-09-26 16:55:25,376 - lava_callback - INFO - Starting async job submit for node 68d42424a5c2c05277bc6ffc 2025-09-26 16:55:25,403 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:55:25,439 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d42424a5c2c05277bc6ffc HTTP/1.1" 200 None 2025-09-26 16:55:25,439 - lava_callback - DEBUG - Processing node 68d42424a5c2c05277bc6ffc: baseline-arm64 2025-09-26 16:55:25,941 - lava_callback - ERROR - Error processing callback for node 68d42424a5c2c05277bc6ffc: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:44110 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44118 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53376 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51442 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50668 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50674 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37592 - "GET / HTTP/1.0" 200 OK 2025-09-26 16:56:08,445 - lava_callback - INFO - Received callback for node 68d69aad2f83206171a92244 INFO: 172.21.0.1:37608 - "POST /node/68d69aad2f83206171a92244 HTTP/1.0" 202 Accepted 2025-09-26 16:56:08,455 - lava_callback - INFO - Starting async job submit for node 68d69aad2f83206171a92244 2025-09-26 16:56:08,496 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:56:08,538 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d69aad2f83206171a92244 HTTP/1.1" 200 None 2025-09-26 16:56:08,538 - lava_callback - DEBUG - Processing node 68d69aad2f83206171a92244: baseline-arm64 2025-09-26 16:56:09,087 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 16:56:09,344 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 16:56:09,345 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d69aad2f83206171a92244/log.txt.gz 2025-09-26 16:56:09,353 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 16:56:09,478 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 16:56:09,479 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d69aad2f83206171a92244/lava_callback.json.gz 2025-09-26 16:56:09,511 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:56:09,526 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d69aad2f83206171a92244 HTTP/1.1" 200 None 2025-09-26 16:56:09,528 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:56:09,545 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d5dc8530c58dac7d9410cb HTTP/1.1" 200 None 2025-09-26 16:56:09,547 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:56:09,612 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d69aad2f83206171a92244 HTTP/1.1" 200 None 2025-09-26 16:56:09,612 - lava_callback - INFO - Completed processing callback for node 68d69aad2f83206171a92244 2025-09-26 16:56:12,821 - lava_callback - INFO - Received callback for node 68d42428a5c2c05277bc700a INFO: 172.21.0.1:35842 - "POST /node/68d42428a5c2c05277bc700a HTTP/1.0" 202 Accepted 2025-09-26 16:56:12,831 - lava_callback - INFO - Starting async job submit for node 68d42428a5c2c05277bc700a 2025-09-26 16:56:12,865 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:56:12,882 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d42428a5c2c05277bc700a HTTP/1.1" 200 None 2025-09-26 16:56:12,883 - lava_callback - DEBUG - Processing node 68d42428a5c2c05277bc700a: baseline-arm64 2025-09-26 16:56:13,403 - lava_callback - ERROR - Error processing callback for node 68d42428a5c2c05277bc700a: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:35846 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35862 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39888 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39894 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33778 - "GET / HTTP/1.0" 200 OK 2025-09-26 16:56:40,212 - lava_callback - INFO - Received callback for node 68d4242da5c2c05277bc701d INFO: 172.21.0.1:57066 - "POST /node/68d4242da5c2c05277bc701d HTTP/1.0" 202 Accepted 2025-09-26 16:56:40,218 - lava_callback - INFO - Starting async job submit for node 68d4242da5c2c05277bc701d 2025-09-26 16:56:40,239 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:56:40,273 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4242da5c2c05277bc701d HTTP/1.1" 200 None 2025-09-26 16:56:40,273 - lava_callback - DEBUG - Processing node 68d4242da5c2c05277bc701d: baseline-arm64 2025-09-26 16:56:40,610 - lava_callback - ERROR - Error processing callback for node 68d4242da5c2c05277bc701d: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:57074 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52258 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52274 - "GET / HTTP/1.0" 200 OK 2025-09-26 16:57:01,184 - lava_callback - INFO - Received callback for node 68d69a652f83206171a9222c INFO: 172.21.0.1:37664 - "POST /node/68d69a652f83206171a9222c HTTP/1.0" 202 Accepted 2025-09-26 16:57:01,192 - lava_callback - INFO - Starting async job submit for node 68d69a652f83206171a9222c 2025-09-26 16:57:01,233 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:57:01,264 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d69a652f83206171a9222c HTTP/1.1" 200 None 2025-09-26 16:57:01,265 - lava_callback - DEBUG - Processing node 68d69a652f83206171a9222c: baseline-arm64 2025-09-26 16:57:02,274 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 16:57:02,403 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 16:57:02,404 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d69a652f83206171a9222c/log.txt.gz 2025-09-26 16:57:02,424 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 16:57:02,615 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 16:57:02,616 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d69a652f83206171a9222c/lava_callback.json.gz DEBUG: Setup failed for 68d69a652f83206171a9222c. Transitting job node result: incomplete -> fail 2025-09-26 16:57:02,714 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:57:02,726 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d69a652f83206171a9222c HTTP/1.1" 200 None 2025-09-26 16:57:02,727 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:57:02,749 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d62c44c1bc82a30b37a916 HTTP/1.1" 200 None 2025-09-26 16:57:02,750 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:57:02,777 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d69a652f83206171a9222c HTTP/1.1" 200 None 2025-09-26 16:57:02,778 - lava_callback - INFO - Completed processing callback for node 68d69a652f83206171a9222c INFO: 172.21.0.1:37680 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60252 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60264 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50994 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51006 - "GET / HTTP/1.0" 200 OK 2025-09-26 16:57:31,831 - lava_callback - INFO - Received callback for node 68d42463a5c2c05277bc7118 INFO: 172.21.0.1:48694 - "POST /node/68d42463a5c2c05277bc7118 HTTP/1.0" 202 Accepted 2025-09-26 16:57:31,838 - lava_callback - INFO - Starting async job submit for node 68d42463a5c2c05277bc7118 2025-09-26 16:57:31,935 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:57:31,971 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d42463a5c2c05277bc7118 HTTP/1.1" 200 None 2025-09-26 16:57:31,971 - lava_callback - DEBUG - Processing node 68d42463a5c2c05277bc7118: baseline-arm64 2025-09-26 16:57:32,579 - lava_callback - ERROR - Error processing callback for node 68d42463a5c2c05277bc7118: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:48710 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50544 - "GET / HTTP/1.0" 200 OK 2025-09-26 16:57:48,456 - lava_callback - INFO - Received callback for node 68d69ac92f83206171a92263 INFO: 172.21.0.1:50558 - "POST /node/68d69ac92f83206171a92263 HTTP/1.0" 202 Accepted 2025-09-26 16:57:48,462 - lava_callback - INFO - Starting async job submit for node 68d69ac92f83206171a92263 2025-09-26 16:57:48,483 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:57:48,498 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d69ac92f83206171a92263 HTTP/1.1" 200 None 2025-09-26 16:57:48,498 - lava_callback - DEBUG - Processing node 68d69ac92f83206171a92263: baseline-arm64 2025-09-26 16:57:48,871 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 16:57:49,009 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 16:57:49,010 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d69ac92f83206171a92263/log.txt.gz 2025-09-26 16:57:49,018 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 16:57:49,130 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 16:57:49,131 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d69ac92f83206171a92263/lava_callback.json.gz 2025-09-26 16:57:49,151 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:57:49,170 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d69ac92f83206171a92263 HTTP/1.1" 200 None 2025-09-26 16:57:49,171 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:57:49,181 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d5dc8530c58dac7d9410cb HTTP/1.1" 200 None 2025-09-26 16:57:49,182 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:57:49,240 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d69ac92f83206171a92263 HTTP/1.1" 200 None 2025-09-26 16:57:49,241 - lava_callback - INFO - Completed processing callback for node 68d69ac92f83206171a92263 INFO: 172.21.0.1:54038 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54048 - "GET / HTTP/1.0" 200 OK 2025-09-26 16:58:05,218 - lava_callback - INFO - Received callback for node 68d42464a5c2c05277bc711b INFO: 172.21.0.1:41770 - "POST /node/68d42464a5c2c05277bc711b HTTP/1.0" 202 Accepted 2025-09-26 16:58:05,224 - lava_callback - INFO - Starting async job submit for node 68d42464a5c2c05277bc711b 2025-09-26 16:58:05,245 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:58:05,261 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d42464a5c2c05277bc711b HTTP/1.1" 200 None 2025-09-26 16:58:05,262 - lava_callback - DEBUG - Processing node 68d42464a5c2c05277bc711b: baseline-arm64 2025-09-26 16:58:05,564 - lava_callback - ERROR - Error processing callback for node 68d42464a5c2c05277bc711b: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:41774 - "GET / HTTP/1.0" 200 OK 2025-09-26 16:58:14,265 - lava_callback - INFO - Received callback for node 68d69aca2f83206171a92266 INFO: 172.21.0.1:38928 - "POST /node/68d69aca2f83206171a92266 HTTP/1.0" 202 Accepted 2025-09-26 16:58:14,271 - lava_callback - INFO - Starting async job submit for node 68d69aca2f83206171a92266 2025-09-26 16:58:14,292 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:58:14,338 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d69aca2f83206171a92266 HTTP/1.1" 200 None 2025-09-26 16:58:14,338 - lava_callback - DEBUG - Processing node 68d69aca2f83206171a92266: baseline-arm64 2025-09-26 16:58:14,718 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 16:58:14,850 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 16:58:14,851 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d69aca2f83206171a92266/log.txt.gz 2025-09-26 16:58:14,859 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 16:58:14,974 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 16:58:14,975 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d69aca2f83206171a92266/lava_callback.json.gz 2025-09-26 16:58:14,996 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:58:15,009 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d69aca2f83206171a92266 HTTP/1.1" 200 None 2025-09-26 16:58:15,010 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:58:15,021 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d5dc8530c58dac7d9410cb HTTP/1.1" 200 None 2025-09-26 16:58:15,022 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:58:15,054 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d69aca2f83206171a92266 HTTP/1.1" 200 None 2025-09-26 16:58:15,055 - lava_callback - INFO - Completed processing callback for node 68d69aca2f83206171a92266 INFO: 172.21.0.1:38934 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38936 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60188 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60202 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41906 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34580 - "GET / HTTP/1.0" 200 OK 2025-09-26 16:58:56,293 - lava_callback - INFO - Received callback for node 68d6c1ef3c94e462c6881695 INFO: 172.21.0.1:50526 - "POST /node/68d6c1ef3c94e462c6881695 HTTP/1.0" 202 Accepted 2025-09-26 16:58:56,300 - lava_callback - INFO - Starting async job submit for node 68d6c1ef3c94e462c6881695 2025-09-26 16:58:56,331 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:58:56,367 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6c1ef3c94e462c6881695 HTTP/1.1" 200 None 2025-09-26 16:58:56,368 - lava_callback - DEBUG - Processing node 68d6c1ef3c94e462c6881695: baseline-arm INFO: 172.21.0.1:50528 - "GET / HTTP/1.0" 200 OK 2025-09-26 16:58:56,690 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 16:58:56,845 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 16:58:56,846 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-68d6c1ef3c94e462c6881695/log.txt.gz 2025-09-26 16:58:56,854 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 16:58:56,978 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 16:58:56,979 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-68d6c1ef3c94e462c6881695/lava_callback.json.gz DEBUG: Setup failed for 68d6c1ef3c94e462c6881695. Transitting job node result: incomplete -> fail 2025-09-26 16:58:57,040 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:58:57,052 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6c1ef3c94e462c6881695 HTTP/1.1" 200 None 2025-09-26 16:58:57,053 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:58:57,074 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6bd793c94e462c68814e2 HTTP/1.1" 200 None 2025-09-26 16:58:57,075 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:58:57,103 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6c1ef3c94e462c6881695 HTTP/1.1" 200 None 2025-09-26 16:58:57,104 - lava_callback - INFO - Completed processing callback for node 68d6c1ef3c94e462c6881695 INFO: 172.21.0.1:33844 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33850 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35642 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35644 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33568 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33576 - "GET / HTTP/1.0" 200 OK 2025-09-26 16:59:29,278 - lava_callback - INFO - Received callback for node 68d4236ea5c2c05277bc6eb8 INFO: 172.21.0.1:33584 - "POST /node/68d4236ea5c2c05277bc6eb8 HTTP/1.0" 202 Accepted 2025-09-26 16:59:29,283 - lava_callback - INFO - Starting async job submit for node 68d4236ea5c2c05277bc6eb8 2025-09-26 16:59:29,284 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:59:29,324 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4236ea5c2c05277bc6eb8 HTTP/1.1" 200 None 2025-09-26 16:59:29,324 - lava_callback - DEBUG - Processing node 68d4236ea5c2c05277bc6eb8: baseline-arm 2025-09-26 16:59:29,324 - lava_callback - ERROR - Error processing callback for node 68d4236ea5c2c05277bc6eb8: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 16:59:36,255 - lava_callback - INFO - Received callback for node 68d69ca62f83206171a92295 INFO: 172.21.0.1:54048 - "POST /node/68d69ca62f83206171a92295 HTTP/1.0" 202 Accepted 2025-09-26 16:59:36,261 - lava_callback - INFO - Starting async job submit for node 68d69ca62f83206171a92295 2025-09-26 16:59:36,281 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:59:36,293 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d69ca62f83206171a92295 HTTP/1.1" 200 None 2025-09-26 16:59:36,294 - lava_callback - DEBUG - Processing node 68d69ca62f83206171a92295: baseline-arm64 INFO: 172.21.0.1:54056 - "GET / HTTP/1.0" 200 OK 2025-09-26 16:59:36,680 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 16:59:36,815 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 16:59:36,816 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d69ca62f83206171a92295/log.txt.gz 2025-09-26 16:59:36,824 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 16:59:36,959 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 16:59:36,959 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d69ca62f83206171a92295/lava_callback.json.gz 2025-09-26 16:59:36,980 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:59:37,001 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d69ca62f83206171a92295 HTTP/1.1" 200 None 2025-09-26 16:59:37,002 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:59:37,013 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d5dc8530c58dac7d9410cb HTTP/1.1" 200 None 2025-09-26 16:59:37,014 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:59:37,038 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d69ca62f83206171a92295 HTTP/1.1" 200 None 2025-09-26 16:59:37,039 - lava_callback - INFO - Completed processing callback for node 68d69ca62f83206171a92295 INFO: 172.21.0.1:44072 - "GET / HTTP/1.0" 200 OK 2025-09-26 16:59:47,373 - lava_callback - INFO - Received callback for node 68d4237aa5c2c05277bc6ee5 INFO: 172.21.0.1:44084 - "POST /node/68d4237aa5c2c05277bc6ee5 HTTP/1.0" 202 Accepted 2025-09-26 16:59:47,379 - lava_callback - INFO - Starting async job submit for node 68d4237aa5c2c05277bc6ee5 2025-09-26 16:59:47,379 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:59:47,400 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4237aa5c2c05277bc6ee5 HTTP/1.1" 200 None 2025-09-26 16:59:47,400 - lava_callback - DEBUG - Processing node 68d4237aa5c2c05277bc6ee5: baseline-arm-broonie 2025-09-26 16:59:47,400 - lava_callback - ERROR - Error processing callback for node 68d4237aa5c2c05277bc6ee5: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 16:59:50,004 - lava_callback - INFO - Received callback for node 68d4237fa5c2c05277bc6efa INFO: 172.21.0.1:59652 - "POST /node/68d4237fa5c2c05277bc6efa HTTP/1.0" 202 Accepted 2025-09-26 16:59:50,009 - lava_callback - INFO - Starting async job submit for node 68d4237fa5c2c05277bc6efa 2025-09-26 16:59:50,010 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:59:50,022 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4237fa5c2c05277bc6efa HTTP/1.1" 200 None 2025-09-26 16:59:50,022 - lava_callback - DEBUG - Processing node 68d4237fa5c2c05277bc6efa: baseline-arm-clabbe 2025-09-26 16:59:50,023 - lava_callback - ERROR - Error processing callback for node 68d4237fa5c2c05277bc6efa: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 16:59:50,673 - lava_callback - INFO - Received callback for node 68d42380a5c2c05277bc6f03 INFO: 172.21.0.1:59664 - "POST /node/68d42380a5c2c05277bc6f03 HTTP/1.0" 202 Accepted 2025-09-26 16:59:50,678 - lava_callback - INFO - Starting async job submit for node 68d42380a5c2c05277bc6f03 2025-09-26 16:59:50,679 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:59:50,710 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d42380a5c2c05277bc6f03 HTTP/1.1" 200 None 2025-09-26 16:59:50,710 - lava_callback - DEBUG - Processing node 68d42380a5c2c05277bc6f03: baseline-arm-clabbe 2025-09-26 16:59:50,711 - lava_callback - ERROR - Error processing callback for node 68d42380a5c2c05277bc6f03: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 16:59:51,321 - lava_callback - INFO - Received callback for node 68d42384a5c2c05277bc6f15 INFO: 172.21.0.1:59666 - "POST /node/68d42384a5c2c05277bc6f15 HTTP/1.0" 202 Accepted 2025-09-26 16:59:51,327 - lava_callback - INFO - Starting async job submit for node 68d42384a5c2c05277bc6f15 2025-09-26 16:59:51,328 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:59:51,340 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d42384a5c2c05277bc6f15 HTTP/1.1" 200 None 2025-09-26 16:59:51,340 - lava_callback - DEBUG - Processing node 68d42384a5c2c05277bc6f15: baseline-arm-clabbe 2025-09-26 16:59:51,340 - lava_callback - ERROR - Error processing callback for node 68d42384a5c2c05277bc6f15: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 16:59:51,957 - lava_callback - INFO - Received callback for node 68d42381a5c2c05277bc6f06 INFO: 172.21.0.1:59678 - "POST /node/68d42381a5c2c05277bc6f06 HTTP/1.0" 202 Accepted 2025-09-26 16:59:51,962 - lava_callback - INFO - Starting async job submit for node 68d42381a5c2c05277bc6f06 2025-09-26 16:59:51,963 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 16:59:51,977 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d42381a5c2c05277bc6f06 HTTP/1.1" 200 None 2025-09-26 16:59:51,977 - lava_callback - DEBUG - Processing node 68d42381a5c2c05277bc6f06: baseline-arm-clabbe 2025-09-26 16:59:51,977 - lava_callback - ERROR - Error processing callback for node 68d42381a5c2c05277bc6f06: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:59690 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52866 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52868 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:00:07,391 - lava_callback - INFO - Received callback for node 68d42391a5c2c05277bc6f48 INFO: 172.21.0.1:52880 - "POST /node/68d42391a5c2c05277bc6f48 HTTP/1.0" 202 Accepted 2025-09-26 17:00:07,397 - lava_callback - INFO - Starting async job submit for node 68d42391a5c2c05277bc6f48 2025-09-26 17:00:07,397 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:00:07,410 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d42391a5c2c05277bc6f48 HTTP/1.1" 200 None 2025-09-26 17:00:07,411 - lava_callback - DEBUG - Processing node 68d42391a5c2c05277bc6f48: ltp-fsx 2025-09-26 17:00:07,411 - lava_callback - ERROR - Error processing callback for node 68d42391a5c2c05277bc6f48: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 17:00:08,131 - lava_callback - INFO - Received callback for node 68d42391a5c2c05277bc6f4d INFO: 172.21.0.1:52892 - "POST /node/68d42391a5c2c05277bc6f4d HTTP/1.0" 202 Accepted 2025-09-26 17:00:08,137 - lava_callback - INFO - Starting async job submit for node 68d42391a5c2c05277bc6f4d 2025-09-26 17:00:08,137 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:00:08,150 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d42391a5c2c05277bc6f4d HTTP/1.1" 200 None 2025-09-26 17:00:08,150 - lava_callback - DEBUG - Processing node 68d42391a5c2c05277bc6f4d: ltp-input 2025-09-26 17:00:08,150 - lava_callback - ERROR - Error processing callback for node 68d42391a5c2c05277bc6f4d: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 17:00:08,971 - lava_callback - INFO - Received callback for node 68d42392a5c2c05277bc6f51 INFO: 172.21.0.1:52902 - "POST /node/68d42392a5c2c05277bc6f51 HTTP/1.0" 202 Accepted 2025-09-26 17:00:08,977 - lava_callback - INFO - Starting async job submit for node 68d42392a5c2c05277bc6f51 2025-09-26 17:00:08,978 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:00:08,990 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d42392a5c2c05277bc6f51 HTTP/1.1" 200 None 2025-09-26 17:00:08,991 - lava_callback - DEBUG - Processing node 68d42392a5c2c05277bc6f51: ltp-ipc 2025-09-26 17:00:08,991 - lava_callback - ERROR - Error processing callback for node 68d42392a5c2c05277bc6f51: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 17:00:09,742 - lava_callback - INFO - Received callback for node 68d42393a5c2c05277bc6f54 INFO: 172.21.0.1:41842 - "POST /node/68d42393a5c2c05277bc6f54 HTTP/1.0" 202 Accepted 2025-09-26 17:00:09,748 - lava_callback - INFO - Starting async job submit for node 68d42393a5c2c05277bc6f54 2025-09-26 17:00:09,749 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:00:09,761 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d42393a5c2c05277bc6f54 HTTP/1.1" 200 None 2025-09-26 17:00:09,761 - lava_callback - DEBUG - Processing node 68d42393a5c2c05277bc6f54: ltp-pty 2025-09-26 17:00:09,761 - lava_callback - ERROR - Error processing callback for node 68d42393a5c2c05277bc6f54: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 17:00:10,474 - lava_callback - INFO - Received callback for node 68d42396a5c2c05277bc6f5f INFO: 172.21.0.1:41846 - "POST /node/68d42396a5c2c05277bc6f5f HTTP/1.0" 202 Accepted 2025-09-26 17:00:10,480 - lava_callback - INFO - Starting async job submit for node 68d42396a5c2c05277bc6f5f 2025-09-26 17:00:10,481 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:00:10,494 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d42396a5c2c05277bc6f5f HTTP/1.1" 200 None 2025-09-26 17:00:10,494 - lava_callback - DEBUG - Processing node 68d42396a5c2c05277bc6f5f: ltp-sched 2025-09-26 17:00:10,494 - lava_callback - ERROR - Error processing callback for node 68d42396a5c2c05277bc6f5f: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 17:00:11,215 - lava_callback - INFO - Received callback for node 68d42397a5c2c05277bc6f70 INFO: 172.21.0.1:41862 - "POST /node/68d42397a5c2c05277bc6f70 HTTP/1.0" 202 Accepted 2025-09-26 17:00:11,221 - lava_callback - INFO - Starting async job submit for node 68d42397a5c2c05277bc6f70 2025-09-26 17:00:11,222 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:00:11,253 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d42397a5c2c05277bc6f70 HTTP/1.1" 200 None 2025-09-26 17:00:11,253 - lava_callback - DEBUG - Processing node 68d42397a5c2c05277bc6f70: ltp-smoketest 2025-09-26 17:00:11,253 - lava_callback - ERROR - Error processing callback for node 68d42397a5c2c05277bc6f70: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 17:00:11,476 - lava_callback - INFO - Received callback for node 68d6c29b3c94e462c6881796 INFO: 172.21.0.1:41868 - "POST /node/68d6c29b3c94e462c6881796 HTTP/1.0" 202 Accepted 2025-09-26 17:00:11,482 - lava_callback - INFO - Starting async job submit for node 68d6c29b3c94e462c6881796 2025-09-26 17:00:11,526 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:00:11,538 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6c29b3c94e462c6881796 HTTP/1.1" 200 None 2025-09-26 17:00:11,539 - lava_callback - DEBUG - Processing node 68d6c29b3c94e462c6881796: baseline-x86 2025-09-26 17:00:12,012 - lava_callback - INFO - Received callback for node 68d42398a5c2c05277bc6f7c INFO: 172.21.0.1:41884 - "POST /node/68d42398a5c2c05277bc6f7c HTTP/1.0" 202 Accepted 2025-09-26 17:00:12,034 - lava_callback - INFO - Starting async job submit for node 68d42398a5c2c05277bc6f7c 2025-09-26 17:00:12,034 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:00:12,097 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d42398a5c2c05277bc6f7c HTTP/1.1" 200 None 2025-09-26 17:00:12,108 - lava_callback - DEBUG - Processing node 68d42398a5c2c05277bc6f7c: ltp-syscalls 2025-09-26 17:00:12,108 - lava_callback - ERROR - Error processing callback for node 68d42398a5c2c05277bc6f7c: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 17:00:12,175 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 17:00:12,295 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 17:00:12,296 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-68d6c29b3c94e462c6881796/log.txt.gz 2025-09-26 17:00:12,312 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 17:00:12,502 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 17:00:12,503 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-68d6c29b3c94e462c6881796/lava_callback.json.gz DEBUG: Setup failed for 68d6c29b3c94e462c6881796. Transitting job node result: incomplete -> fail 2025-09-26 17:00:12,590 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:00:12,603 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6c29b3c94e462c6881796 HTTP/1.1" 200 None 2025-09-26 17:00:12,604 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:00:12,618 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6b9f22f83206171a92b66 HTTP/1.1" 200 None 2025-09-26 17:00:12,619 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:00:12,647 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6c29b3c94e462c6881796 HTTP/1.1" 200 None 2025-09-26 17:00:12,647 - lava_callback - INFO - Completed processing callback for node 68d6c29b3c94e462c6881796 2025-09-26 17:00:12,818 - lava_callback - INFO - Received callback for node 68d42399a5c2c05277bc6f7f INFO: 172.21.0.1:41896 - "POST /node/68d42399a5c2c05277bc6f7f HTTP/1.0" 202 Accepted 2025-09-26 17:00:12,825 - lava_callback - INFO - Starting async job submit for node 68d42399a5c2c05277bc6f7f 2025-09-26 17:00:12,826 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:00:12,837 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d42399a5c2c05277bc6f7f HTTP/1.1" 200 None 2025-09-26 17:00:12,838 - lava_callback - DEBUG - Processing node 68d42399a5c2c05277bc6f7f: ltp-syscalls-ipc 2025-09-26 17:00:12,838 - lava_callback - ERROR - Error processing callback for node 68d42399a5c2c05277bc6f7f: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 17:00:13,550 - lava_callback - INFO - Received callback for node 68d42399a5c2c05277bc6f82 INFO: 172.21.0.1:41898 - "POST /node/68d42399a5c2c05277bc6f82 HTTP/1.0" 202 Accepted 2025-09-26 17:00:13,558 - lava_callback - INFO - Starting async job submit for node 68d42399a5c2c05277bc6f82 2025-09-26 17:00:13,559 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:00:13,571 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d42399a5c2c05277bc6f82 HTTP/1.1" 200 None 2025-09-26 17:00:13,572 - lava_callback - DEBUG - Processing node 68d42399a5c2c05277bc6f82: ltp-timers 2025-09-26 17:00:13,572 - lava_callback - ERROR - Error processing callback for node 68d42399a5c2c05277bc6f82: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 17:00:14,549 - lava_callback - INFO - Received callback for node 68d4239aa5c2c05277bc6f87 INFO: 172.21.0.1:41910 - "POST /node/68d4239aa5c2c05277bc6f87 HTTP/1.0" 202 Accepted 2025-09-26 17:00:14,557 - lava_callback - INFO - Starting async job submit for node 68d4239aa5c2c05277bc6f87 2025-09-26 17:00:14,558 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:00:14,570 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4239aa5c2c05277bc6f87 HTTP/1.1" 200 None 2025-09-26 17:00:14,570 - lava_callback - DEBUG - Processing node 68d4239aa5c2c05277bc6f87: ltp-watchqueue 2025-09-26 17:00:14,570 - lava_callback - ERROR - Error processing callback for node 68d4239aa5c2c05277bc6f87: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:41926 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41936 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34654 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34668 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:00:30,041 - lava_callback - INFO - Received callback for node 68d423a9a5c2c05277bc6f94 INFO: 172.21.0.1:54372 - "POST /node/68d423a9a5c2c05277bc6f94 HTTP/1.0" 202 Accepted 2025-09-26 17:00:30,047 - lava_callback - INFO - Starting async job submit for node 68d423a9a5c2c05277bc6f94 2025-09-26 17:00:30,048 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:00:30,083 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d423a9a5c2c05277bc6f94 HTTP/1.1" 200 None 2025-09-26 17:00:30,083 - lava_callback - DEBUG - Processing node 68d423a9a5c2c05277bc6f94: baseline-arm 2025-09-26 17:00:30,083 - lava_callback - ERROR - Error processing callback for node 68d423a9a5c2c05277bc6f94: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 17:00:30,760 - lava_callback - INFO - Received callback for node 68d423aba5c2c05277bc6f97 INFO: 172.21.0.1:54380 - "POST /node/68d423aba5c2c05277bc6f97 HTTP/1.0" 202 Accepted 2025-09-26 17:00:30,765 - lava_callback - INFO - Starting async job submit for node 68d423aba5c2c05277bc6f97 2025-09-26 17:00:30,766 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:00:30,783 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d423aba5c2c05277bc6f97 HTTP/1.1" 200 None 2025-09-26 17:00:30,783 - lava_callback - DEBUG - Processing node 68d423aba5c2c05277bc6f97: baseline-arm 2025-09-26 17:00:30,783 - lava_callback - ERROR - Error processing callback for node 68d423aba5c2c05277bc6f97: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:54386 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:00:44,408 - lava_callback - INFO - Received callback for node 68d477f8a5c2c05277bccbe6 INFO: 172.21.0.1:38910 - "POST /node/68d477f8a5c2c05277bccbe6 HTTP/1.0" 202 Accepted 2025-09-26 17:00:44,414 - lava_callback - INFO - Starting async job submit for node 68d477f8a5c2c05277bccbe6 2025-09-26 17:00:44,453 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:00:44,471 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d477f8a5c2c05277bccbe6 HTTP/1.1" 200 None 2025-09-26 17:00:44,471 - lava_callback - DEBUG - Processing node 68d477f8a5c2c05277bccbe6: baseline-arm 2025-09-26 17:00:44,698 - lava_callback - ERROR - Error processing callback for node 68d477f8a5c2c05277bccbe6: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:38924 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:00:56,388 - lava_callback - INFO - Received callback for node 68d69ca72f83206171a92298 INFO: 172.21.0.1:58712 - "POST /node/68d69ca72f83206171a92298 HTTP/1.0" 202 Accepted 2025-09-26 17:00:56,394 - lava_callback - INFO - Starting async job submit for node 68d69ca72f83206171a92298 2025-09-26 17:00:56,415 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:00:56,427 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d69ca72f83206171a92298 HTTP/1.1" 200 None 2025-09-26 17:00:56,428 - lava_callback - DEBUG - Processing node 68d69ca72f83206171a92298: baseline-arm64 INFO: 172.21.0.1:58724 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:00:56,784 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 17:00:56,957 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 17:00:56,958 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d69ca72f83206171a92298/log.txt.gz 2025-09-26 17:00:56,967 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 17:00:57,082 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 17:00:57,083 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d69ca72f83206171a92298/lava_callback.json.gz 2025-09-26 17:00:57,104 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:00:57,116 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d69ca72f83206171a92298 HTTP/1.1" 200 None 2025-09-26 17:00:57,117 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:00:57,138 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d5dc8530c58dac7d9410cb HTTP/1.1" 200 None 2025-09-26 17:00:57,139 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:00:57,166 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d69ca72f83206171a92298 HTTP/1.1" 200 None 2025-09-26 17:00:57,167 - lava_callback - INFO - Completed processing callback for node 68d69ca72f83206171a92298 INFO: 172.21.0.1:35690 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35700 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:01:07,287 - lava_callback - INFO - Received callback for node 68d423c8a5c2c05277bc6f9b INFO: 172.21.0.1:35710 - "POST /node/68d423c8a5c2c05277bc6f9b HTTP/1.0" 202 Accepted 2025-09-26 17:01:07,294 - lava_callback - INFO - Starting async job submit for node 68d423c8a5c2c05277bc6f9b 2025-09-26 17:01:07,294 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:01:07,309 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d423c8a5c2c05277bc6f9b HTTP/1.1" 200 None 2025-09-26 17:01:07,309 - lava_callback - DEBUG - Processing node 68d423c8a5c2c05277bc6f9b: ltp-fsx 2025-09-26 17:01:07,309 - lava_callback - ERROR - Error processing callback for node 68d423c8a5c2c05277bc6f9b: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:41744 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41760 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43666 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43676 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53946 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54704 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37144 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59050 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59056 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37610 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37612 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41144 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41150 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41118 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:02:40,560 - lava_callback - INFO - Received callback for node 68d477fba5c2c05277bccbe9 INFO: 172.21.0.1:45836 - "POST /node/68d477fba5c2c05277bccbe9 HTTP/1.0" 202 Accepted 2025-09-26 17:02:40,568 - lava_callback - INFO - Starting async job submit for node 68d477fba5c2c05277bccbe9 2025-09-26 17:02:40,590 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:02:40,604 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d477fba5c2c05277bccbe9 HTTP/1.1" 200 None 2025-09-26 17:02:40,604 - lava_callback - DEBUG - Processing node 68d477fba5c2c05277bccbe9: baseline-arm 2025-09-26 17:02:40,946 - lava_callback - ERROR - Error processing callback for node 68d477fba5c2c05277bccbe9: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:45838 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60672 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49540 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:03:01,186 - lava_callback - INFO - Received callback for node 68d42430a5c2c05277bc7035 INFO: 172.21.0.1:49544 - "POST /node/68d42430a5c2c05277bc7035 HTTP/1.0" 202 Accepted 2025-09-26 17:03:01,192 - lava_callback - INFO - Starting async job submit for node 68d42430a5c2c05277bc7035 2025-09-26 17:03:01,193 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:03:01,208 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d42430a5c2c05277bc7035 HTTP/1.1" 200 None 2025-09-26 17:03:01,208 - lava_callback - DEBUG - Processing node 68d42430a5c2c05277bc7035: baseline-arm64-baylibre 2025-09-26 17:03:01,208 - lava_callback - ERROR - Error processing callback for node 68d42430a5c2c05277bc7035: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 17:03:01,843 - lava_callback - INFO - Received callback for node 68d4243ca5c2c05277bc705c INFO: 172.21.0.1:49558 - "POST /node/68d4243ca5c2c05277bc705c HTTP/1.0" 202 Accepted 2025-09-26 17:03:01,852 - lava_callback - INFO - Starting async job submit for node 68d4243ca5c2c05277bc705c 2025-09-26 17:03:01,853 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:03:01,878 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4243ca5c2c05277bc705c HTTP/1.1" 200 None 2025-09-26 17:03:01,878 - lava_callback - DEBUG - Processing node 68d4243ca5c2c05277bc705c: baseline-arm64-baylibre 2025-09-26 17:03:01,879 - lava_callback - ERROR - Error processing callback for node 68d4243ca5c2c05277bc705c: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:49574 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:03:10,711 - lava_callback - INFO - Received callback for node 68d42368a5c2c05277bc6eac INFO: 172.21.0.1:45818 - "POST /node/68d42368a5c2c05277bc6eac HTTP/1.0" 202 Accepted 2025-09-26 17:03:10,717 - lava_callback - INFO - Starting async job submit for node 68d42368a5c2c05277bc6eac 2025-09-26 17:03:10,763 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:03:10,796 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d42368a5c2c05277bc6eac HTTP/1.1" 200 None 2025-09-26 17:03:10,797 - lava_callback - DEBUG - Processing node 68d42368a5c2c05277bc6eac: baseline-arm 2025-09-26 17:03:11,145 - lava_callback - ERROR - Error processing callback for node 68d42368a5c2c05277bc6eac: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:45826 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45828 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39920 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39936 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:03:29,962 - lava_callback - INFO - Received callback for node 68d4244fa5c2c05277bc709e INFO: 172.21.0.1:53408 - "POST /node/68d4244fa5c2c05277bc709e HTTP/1.0" 202 Accepted 2025-09-26 17:03:29,969 - lava_callback - INFO - Starting async job submit for node 68d4244fa5c2c05277bc709e 2025-09-26 17:03:29,970 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:03:29,981 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4244fa5c2c05277bc709e HTTP/1.1" 200 None 2025-09-26 17:03:29,982 - lava_callback - DEBUG - Processing node 68d4244fa5c2c05277bc709e: baseline-arm64-clabbe 2025-09-26 17:03:29,982 - lava_callback - ERROR - Error processing callback for node 68d4244fa5c2c05277bc709e: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 17:03:30,599 - lava_callback - INFO - Received callback for node 68d42451a5c2c05277bc70c0 INFO: 172.21.0.1:53416 - "POST /node/68d42451a5c2c05277bc70c0 HTTP/1.0" 202 Accepted 2025-09-26 17:03:30,608 - lava_callback - INFO - Starting async job submit for node 68d42451a5c2c05277bc70c0 2025-09-26 17:03:30,609 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:03:30,625 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d42451a5c2c05277bc70c0 HTTP/1.1" 200 None 2025-09-26 17:03:30,625 - lava_callback - DEBUG - Processing node 68d42451a5c2c05277bc70c0: baseline-arm64-clabbe 2025-09-26 17:03:30,625 - lava_callback - ERROR - Error processing callback for node 68d42451a5c2c05277bc70c0: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 17:03:31,246 - lava_callback - INFO - Received callback for node 68d4244da5c2c05277bc7095 INFO: 172.21.0.1:53418 - "POST /node/68d4244da5c2c05277bc7095 HTTP/1.0" 202 Accepted 2025-09-26 17:03:31,251 - lava_callback - INFO - Starting async job submit for node 68d4244da5c2c05277bc7095 2025-09-26 17:03:31,252 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:03:31,265 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4244da5c2c05277bc7095 HTTP/1.1" 200 None 2025-09-26 17:03:31,265 - lava_callback - DEBUG - Processing node 68d4244da5c2c05277bc7095: baseline-arm64-clabbe 2025-09-26 17:03:31,265 - lava_callback - ERROR - Error processing callback for node 68d4244da5c2c05277bc7095: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 17:03:31,900 - lava_callback - INFO - Received callback for node 68d42450a5c2c05277bc70a4 INFO: 172.21.0.1:53432 - "POST /node/68d42450a5c2c05277bc70a4 HTTP/1.0" 202 Accepted 2025-09-26 17:03:31,906 - lava_callback - INFO - Starting async job submit for node 68d42450a5c2c05277bc70a4 2025-09-26 17:03:31,907 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:03:31,918 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d42450a5c2c05277bc70a4 HTTP/1.1" 200 None 2025-09-26 17:03:31,918 - lava_callback - DEBUG - Processing node 68d42450a5c2c05277bc70a4: baseline-arm64-clabbe 2025-09-26 17:03:31,919 - lava_callback - ERROR - Error processing callback for node 68d42450a5c2c05277bc70a4: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 17:03:32,574 - lava_callback - INFO - Received callback for node 68d42451a5c2c05277bc70b1 INFO: 172.21.0.1:53442 - "POST /node/68d42451a5c2c05277bc70b1 HTTP/1.0" 202 Accepted 2025-09-26 17:03:32,581 - lava_callback - INFO - Starting async job submit for node 68d42451a5c2c05277bc70b1 2025-09-26 17:03:32,581 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:03:32,611 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d42451a5c2c05277bc70b1 HTTP/1.1" 200 None 2025-09-26 17:03:32,612 - lava_callback - DEBUG - Processing node 68d42451a5c2c05277bc70b1: baseline-arm64-clabbe 2025-09-26 17:03:32,612 - lava_callback - ERROR - Error processing callback for node 68d42451a5c2c05277bc70b1: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:53452 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50326 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48518 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:03:58,843 - lava_callback - INFO - Received callback for node 68d6bd693c94e462c68814d2 INFO: 172.21.0.1:48532 - "POST /node/68d6bd693c94e462c68814d2 HTTP/1.0" 202 Accepted 2025-09-26 17:03:58,850 - lava_callback - INFO - Starting async job submit for node 68d6bd693c94e462c68814d2 2025-09-26 17:03:58,888 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:03:58,901 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6bd693c94e462c68814d2 HTTP/1.1" 200 None 2025-09-26 17:03:58,901 - lava_callback - DEBUG - Processing node 68d6bd693c94e462c68814d2: baseline-arm64 2025-09-26 17:04:00,008 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 17:04:00,149 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 17:04:00,150 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6bd693c94e462c68814d2/log.txt.gz 2025-09-26 17:04:00,171 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 17:04:00,361 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 17:04:00,362 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6bd693c94e462c68814d2/lava_callback.json.gz DEBUG: Setup failed for 68d6bd693c94e462c68814d2. Transitting job node result: incomplete -> fail 2025-09-26 17:04:00,437 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:04:00,451 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6bd693c94e462c68814d2 HTTP/1.1" 200 None 2025-09-26 17:04:00,452 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:04:00,465 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d651ddcca86fb17472aa24 HTTP/1.1" 200 None 2025-09-26 17:04:00,466 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:04:00,492 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6bd693c94e462c68814d2 HTTP/1.1" 200 None 2025-09-26 17:04:00,494 - lava_callback - INFO - Completed processing callback for node 68d6bd693c94e462c68814d2 INFO: 172.21.0.1:39036 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39042 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46902 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46904 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:04:21,245 - lava_callback - INFO - Received callback for node 68d4248ba5c2c05277bc7142 INFO: 172.21.0.1:45390 - "POST /node/68d4248ba5c2c05277bc7142 HTTP/1.0" 202 Accepted 2025-09-26 17:04:21,255 - lava_callback - INFO - Starting async job submit for node 68d4248ba5c2c05277bc7142 2025-09-26 17:04:21,256 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:04:21,280 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4248ba5c2c05277bc7142 HTTP/1.1" 200 None 2025-09-26 17:04:21,281 - lava_callback - DEBUG - Processing node 68d4248ba5c2c05277bc7142: baseline-arm64-baylibre 2025-09-26 17:04:21,281 - lava_callback - ERROR - Error processing callback for node 68d4248ba5c2c05277bc7142: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:45402 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45408 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33748 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:04:40,412 - lava_callback - INFO - Received callback for node 68d6a5262f83206171a9240c INFO: 172.21.0.1:36766 - "POST /node/68d6a5262f83206171a9240c HTTP/1.0" 202 Accepted 2025-09-26 17:04:40,419 - lava_callback - INFO - Starting async job submit for node 68d6a5262f83206171a9240c 2025-09-26 17:04:40,442 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:04:40,455 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6a5262f83206171a9240c HTTP/1.1" 200 None 2025-09-26 17:04:40,456 - lava_callback - DEBUG - Processing node 68d6a5262f83206171a9240c: baseline-arm64 2025-09-26 17:04:41,097 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 17:04:41,295 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 17:04:41,296 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6a5262f83206171a9240c/log.txt.gz 2025-09-26 17:04:41,308 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 17:04:41,434 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 17:04:41,434 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6a5262f83206171a9240c/lava_callback.json.gz 2025-09-26 17:04:41,456 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:04:41,468 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6a5262f83206171a9240c HTTP/1.1" 200 None 2025-09-26 17:04:41,469 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:04:41,480 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d5d90f2be2da466069ac22 HTTP/1.1" 200 None 2025-09-26 17:04:41,482 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:04:41,512 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6a5262f83206171a9240c HTTP/1.1" 200 None 2025-09-26 17:04:41,513 - lava_callback - INFO - Completed processing callback for node 68d6a5262f83206171a9240c INFO: 172.21.0.1:36774 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:04:48,921 - lava_callback - INFO - Received callback for node 68d4780ea5c2c05277bccbfe INFO: 172.21.0.1:36788 - "POST /node/68d4780ea5c2c05277bccbfe HTTP/1.0" 202 Accepted 2025-09-26 17:04:48,927 - lava_callback - INFO - Starting async job submit for node 68d4780ea5c2c05277bccbfe 2025-09-26 17:04:48,949 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:04:48,962 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4780ea5c2c05277bccbfe HTTP/1.1" 200 None 2025-09-26 17:04:48,962 - lava_callback - DEBUG - Processing node 68d4780ea5c2c05277bccbfe: baseline-arm 2025-09-26 17:04:49,317 - lava_callback - ERROR - Error processing callback for node 68d4780ea5c2c05277bccbfe: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:49202 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:04:59,333 - lava_callback - INFO - Received callback for node 68d6bd6a3c94e462c68814d5 INFO: 172.21.0.1:47040 - "POST /node/68d6bd6a3c94e462c68814d5 HTTP/1.0" 202 Accepted 2025-09-26 17:04:59,341 - lava_callback - INFO - Starting async job submit for node 68d6bd6a3c94e462c68814d5 2025-09-26 17:04:59,383 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:04:59,395 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6bd6a3c94e462c68814d5 HTTP/1.1" 200 None 2025-09-26 17:04:59,396 - lava_callback - DEBUG - Processing node 68d6bd6a3c94e462c68814d5: baseline-arm64 2025-09-26 17:04:59,737 - lava_callback - INFO - Received callback for node 68d6c35a3c94e462c68817cb INFO: 172.21.0.1:47050 - "POST /node/68d6c35a3c94e462c68817cb HTTP/1.0" 202 Accepted 2025-09-26 17:04:59,764 - lava_callback - INFO - Starting async job submit for node 68d6c35a3c94e462c68817cb 2025-09-26 17:04:59,828 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:04:59,895 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6c35a3c94e462c68817cb HTTP/1.1" 200 None 2025-09-26 17:04:59,906 - lava_callback - DEBUG - Processing node 68d6c35a3c94e462c68817cb: baseline-arm 2025-09-26 17:05:00,717 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 17:05:00,895 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 17:05:00,927 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 17:05:00,928 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-68d6c35a3c94e462c68817cb/log.txt.gz 2025-09-26 17:05:00,936 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 17:05:01,007 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 17:05:01,008 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6bd6a3c94e462c68814d5/log.txt.gz INFO: 172.21.0.1:47062 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:05:01,028 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 17:05:01,094 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 17:05:01,095 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-68d6c35a3c94e462c68817cb/lava_callback.json.gz DEBUG: Setup failed for 68d6c35a3c94e462c68817cb. Transitting job node result: incomplete -> fail 2025-09-26 17:05:01,160 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:05:01,173 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6c35a3c94e462c68817cb HTTP/1.1" 200 None 2025-09-26 17:05:01,175 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:05:01,183 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 17:05:01,184 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6bd6a3c94e462c68814d5/lava_callback.json.gz 2025-09-26 17:05:01,208 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6b9da2f83206171a92b45 HTTP/1.1" 200 None 2025-09-26 17:05:01,219 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 DEBUG: Setup failed for 68d6bd6a3c94e462c68814d5. Transitting job node result: incomplete -> fail 2025-09-26 17:05:01,275 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:05:01,294 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6bd6a3c94e462c68814d5 HTTP/1.1" 200 None 2025-09-26 17:05:01,295 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:05:01,296 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6c35a3c94e462c68817cb HTTP/1.1" 200 None 2025-09-26 17:05:01,298 - lava_callback - INFO - Completed processing callback for node 68d6c35a3c94e462c68817cb 2025-09-26 17:05:01,317 - lava_callback - INFO - Received callback for node 68d424b6a5c2c05277bc7179 2025-09-26 17:05:01,329 - lava_callback - INFO - Starting async job submit for node 68d424b6a5c2c05277bc7179 INFO: 172.21.0.1:47074 - "POST /node/68d424b6a5c2c05277bc7179 HTTP/1.0" 202 Accepted 2025-09-26 17:05:01,330 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:05:01,345 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d651ddcca86fb17472aa24 HTTP/1.1" 200 None 2025-09-26 17:05:01,346 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:05:01,366 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d424b6a5c2c05277bc7179 HTTP/1.1" 200 None 2025-09-26 17:05:01,367 - lava_callback - DEBUG - Processing node 68d424b6a5c2c05277bc7179: baseline-arm64-baylibre 2025-09-26 17:05:01,367 - lava_callback - ERROR - Error processing callback for node 68d424b6a5c2c05277bc7179: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 17:05:01,394 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6bd6a3c94e462c68814d5 HTTP/1.1" 200 None 2025-09-26 17:05:01,395 - lava_callback - INFO - Completed processing callback for node 68d6bd6a3c94e462c68814d5 2025-09-26 17:05:01,647 - lava_callback - INFO - Received callback for node 68d6a5262f83206171a9240f INFO: 172.21.0.1:47080 - "POST /node/68d6a5262f83206171a9240f HTTP/1.0" 202 Accepted 2025-09-26 17:05:01,654 - lava_callback - INFO - Starting async job submit for node 68d6a5262f83206171a9240f 2025-09-26 17:05:01,683 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:05:01,694 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6a5262f83206171a9240f HTTP/1.1" 200 None 2025-09-26 17:05:01,695 - lava_callback - DEBUG - Processing node 68d6a5262f83206171a9240f: baseline-arm64 2025-09-26 17:05:02,335 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 17:05:02,480 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 17:05:02,481 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6a5262f83206171a9240f/log.txt.gz 2025-09-26 17:05:02,492 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 17:05:02,616 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 17:05:02,617 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6a5262f83206171a9240f/lava_callback.json.gz 2025-09-26 17:05:02,642 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:05:02,661 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6a5262f83206171a9240f HTTP/1.1" 200 None 2025-09-26 17:05:02,662 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:05:02,689 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d5d90f2be2da466069ac22 HTTP/1.1" 200 None 2025-09-26 17:05:02,690 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:05:02,723 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6a5262f83206171a9240f HTTP/1.1" 200 None 2025-09-26 17:05:02,724 - lava_callback - INFO - Completed processing callback for node 68d6a5262f83206171a9240f INFO: 172.21.0.1:47086 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:05:12,368 - lava_callback - INFO - Received callback for node 68d6c7923c94e462c68818d1 INFO: 172.21.0.1:36154 - "POST /node/68d6c7923c94e462c68818d1 HTTP/1.0" 202 Accepted 2025-09-26 17:05:12,374 - lava_callback - INFO - Starting async job submit for node 68d6c7923c94e462c68818d1 2025-09-26 17:05:12,395 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:05:12,408 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6c7923c94e462c68818d1 HTTP/1.1" 200 None 2025-09-26 17:05:12,409 - lava_callback - DEBUG - Processing node 68d6c7923c94e462c68818d1: baseline-arm64-pengutronix 2025-09-26 17:05:12,593 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 17:05:12,742 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 17:05:12,743 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-68d6c7923c94e462c68818d1/log.txt.gz 2025-09-26 17:05:12,749 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 17:05:12,884 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 17:05:12,885 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-68d6c7923c94e462c68818d1/lava_callback.json.gz 2025-09-26 17:05:12,905 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:05:12,917 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6c7923c94e462c68818d1 HTTP/1.1" 200 None 2025-09-26 17:05:12,918 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:05:12,952 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6b9dd2f83206171a92b48 HTTP/1.1" 200 None 2025-09-26 17:05:12,953 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:05:12,983 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6c7923c94e462c68818d1 HTTP/1.1" 200 None 2025-09-26 17:05:12,984 - lava_callback - INFO - Completed processing callback for node 68d6c7923c94e462c68818d1 INFO: 172.21.0.1:36166 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36168 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:05:20,676 - lava_callback - INFO - Received callback for node 68d6c7943c94e462c68818d4 INFO: 172.21.0.1:48628 - "POST /node/68d6c7943c94e462c68818d4 HTTP/1.0" 202 Accepted 2025-09-26 17:05:20,682 - lava_callback - INFO - Starting async job submit for node 68d6c7943c94e462c68818d4 2025-09-26 17:05:20,701 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:05:20,734 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6c7943c94e462c68818d4 HTTP/1.1" 200 None 2025-09-26 17:05:20,735 - lava_callback - DEBUG - Processing node 68d6c7943c94e462c68818d4: baseline-arm64-pengutronix 2025-09-26 17:05:20,936 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 17:05:21,102 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 17:05:21,103 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-68d6c7943c94e462c68818d4/log.txt.gz 2025-09-26 17:05:21,109 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 17:05:21,239 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 17:05:21,240 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-68d6c7943c94e462c68818d4/lava_callback.json.gz 2025-09-26 17:05:21,260 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:05:21,272 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6c7943c94e462c68818d4 HTTP/1.1" 200 None 2025-09-26 17:05:21,273 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:05:21,286 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6b9dd2f83206171a92b48 HTTP/1.1" 200 None 2025-09-26 17:05:21,287 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:05:21,322 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6c7943c94e462c68818d4 HTTP/1.1" 200 None 2025-09-26 17:05:21,323 - lava_callback - INFO - Completed processing callback for node 68d6c7943c94e462c68818d4 2025-09-26 17:05:22,818 - lava_callback - INFO - Received callback for node 68d6c7953c94e462c68818da INFO: 172.21.0.1:48638 - "POST /node/68d6c7953c94e462c68818da HTTP/1.0" 202 Accepted 2025-09-26 17:05:22,826 - lava_callback - INFO - Starting async job submit for node 68d6c7953c94e462c68818da 2025-09-26 17:05:22,855 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:05:22,875 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6c7953c94e462c68818da HTTP/1.1" 200 None 2025-09-26 17:05:22,875 - lava_callback - DEBUG - Processing node 68d6c7953c94e462c68818da: baseline-arm64-foundriesio 2025-09-26 17:05:23,334 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 17:05:23,474 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 17:05:23,475 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-68d6c7953c94e462c68818da/log.txt.gz 2025-09-26 17:05:23,485 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 17:05:23,598 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 17:05:23,599 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-68d6c7953c94e462c68818da/lava_callback.json.gz 2025-09-26 17:05:23,630 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:05:23,642 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6c7953c94e462c68818da HTTP/1.1" 200 None 2025-09-26 17:05:23,644 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:05:23,655 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6b9dd2f83206171a92b48 HTTP/1.1" 200 None 2025-09-26 17:05:23,656 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:05:23,683 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6c7953c94e462c68818da HTTP/1.1" 200 None 2025-09-26 17:05:23,684 - lava_callback - INFO - Completed processing callback for node 68d6c7953c94e462c68818da INFO: 172.21.0.1:48642 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48656 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:05:30,747 - lava_callback - INFO - Received callback for node 68d6c7943c94e462c68818d7 INFO: 172.21.0.1:46238 - "POST /node/68d6c7943c94e462c68818d7 HTTP/1.0" 202 Accepted 2025-09-26 17:05:30,753 - lava_callback - INFO - Starting async job submit for node 68d6c7943c94e462c68818d7 2025-09-26 17:05:30,773 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:05:30,798 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6c7943c94e462c68818d7 HTTP/1.1" 200 None 2025-09-26 17:05:30,798 - lava_callback - DEBUG - Processing node 68d6c7943c94e462c68818d7: baseline-arm64-foundriesio 2025-09-26 17:05:30,997 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 17:05:31,112 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 17:05:31,113 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-68d6c7943c94e462c68818d7/log.txt.gz 2025-09-26 17:05:31,119 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 17:05:31,247 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 17:05:31,248 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-68d6c7943c94e462c68818d7/lava_callback.json.gz 2025-09-26 17:05:31,272 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:05:31,284 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6c7943c94e462c68818d7 HTTP/1.1" 200 None 2025-09-26 17:05:31,285 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:05:31,296 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6b9dd2f83206171a92b48 HTTP/1.1" 200 None 2025-09-26 17:05:31,298 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:05:31,335 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6c7943c94e462c68818d7 HTTP/1.1" 200 None 2025-09-26 17:05:31,336 - lava_callback - INFO - Completed processing callback for node 68d6c7943c94e462c68818d7 INFO: 172.21.0.1:46254 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:05:37,733 - lava_callback - INFO - Received callback for node 68d6c7913c94e462c68818c8 INFO: 172.21.0.1:46262 - "POST /node/68d6c7913c94e462c68818c8 HTTP/1.0" 202 Accepted 2025-09-26 17:05:37,739 - lava_callback - INFO - Starting async job submit for node 68d6c7913c94e462c68818c8 2025-09-26 17:05:37,774 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:05:37,789 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6c7913c94e462c68818c8 HTTP/1.1" 200 None 2025-09-26 17:05:37,790 - lava_callback - DEBUG - Processing node 68d6c7913c94e462c68818c8: baseline-arm64-kontron 2025-09-26 17:05:38,034 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 17:05:38,169 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 17:05:38,170 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-kontron-68d6c7913c94e462c68818c8/log.txt.gz 2025-09-26 17:05:38,176 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 17:05:38,283 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 17:05:38,284 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-kontron-68d6c7913c94e462c68818c8/lava_callback.json.gz 2025-09-26 17:05:38,322 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:05:38,334 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6c7913c94e462c68818c8 HTTP/1.1" 200 None 2025-09-26 17:05:38,334 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:05:38,346 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6b9dd2f83206171a92b48 HTTP/1.1" 200 None 2025-09-26 17:05:38,347 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:05:38,373 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6c7913c94e462c68818c8 HTTP/1.1" 200 None 2025-09-26 17:05:38,374 - lava_callback - INFO - Completed processing callback for node 68d6c7913c94e462c68818c8 2025-09-26 17:05:39,541 - lava_callback - INFO - Received callback for node 68d6c7913c94e462c68818cb INFO: 172.21.0.1:39230 - "POST /node/68d6c7913c94e462c68818cb HTTP/1.0" 202 Accepted 2025-09-26 17:05:39,554 - lava_callback - INFO - Starting async job submit for node 68d6c7913c94e462c68818cb 2025-09-26 17:05:39,594 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:05:39,622 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6c7913c94e462c68818cb HTTP/1.1" 200 None 2025-09-26 17:05:39,622 - lava_callback - DEBUG - Processing node 68d6c7913c94e462c68818cb: baseline-arm64-kontron 2025-09-26 17:05:40,034 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 17:05:40,157 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 17:05:40,158 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-kontron-68d6c7913c94e462c68818cb/log.txt.gz 2025-09-26 17:05:40,165 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 17:05:40,289 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 17:05:40,290 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-kontron-68d6c7913c94e462c68818cb/lava_callback.json.gz 2025-09-26 17:05:40,319 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:05:40,333 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6c7913c94e462c68818cb HTTP/1.1" 200 None 2025-09-26 17:05:40,334 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:05:40,349 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6b9dd2f83206171a92b48 HTTP/1.1" 200 None 2025-09-26 17:05:40,350 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:05:40,383 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6c7913c94e462c68818cb HTTP/1.1" 200 None 2025-09-26 17:05:40,383 - lava_callback - INFO - Completed processing callback for node 68d6c7913c94e462c68818cb INFO: 172.21.0.1:39242 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50266 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41388 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41398 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:06:10,105 - lava_callback - INFO - Received callback for node 68d424f8a5c2c05277bc71be INFO: 172.21.0.1:48816 - "POST /node/68d424f8a5c2c05277bc71be HTTP/1.0" 202 Accepted 2025-09-26 17:06:10,110 - lava_callback - INFO - Starting async job submit for node 68d424f8a5c2c05277bc71be 2025-09-26 17:06:10,111 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:06:10,124 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d424f8a5c2c05277bc71be HTTP/1.1" 200 None 2025-09-26 17:06:10,125 - lava_callback - DEBUG - Processing node 68d424f8a5c2c05277bc71be: baseline-arm64-clabbe 2025-09-26 17:06:10,125 - lava_callback - ERROR - Error processing callback for node 68d424f8a5c2c05277bc71be: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 17:06:12,210 - lava_callback - INFO - Received callback for node 68d6c0613c94e462c688160d INFO: 172.21.0.1:48828 - "POST /node/68d6c0613c94e462c688160d HTTP/1.0" 202 Accepted 2025-09-26 17:06:12,217 - lava_callback - INFO - Starting async job submit for node 68d6c0613c94e462c688160d 2025-09-26 17:06:12,255 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:06:12,267 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6c0613c94e462c688160d HTTP/1.1" 200 None 2025-09-26 17:06:12,267 - lava_callback - DEBUG - Processing node 68d6c0613c94e462c688160d: baseline-arm64 2025-09-26 17:06:13,259 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 17:06:13,401 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 17:06:13,402 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6c0613c94e462c688160d/log.txt.gz 2025-09-26 17:06:13,421 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 17:06:13,603 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 17:06:13,603 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6c0613c94e462c688160d/lava_callback.json.gz DEBUG: Setup failed for 68d6c0613c94e462c688160d. Transitting job node result: incomplete -> fail 2025-09-26 17:06:13,677 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:06:13,710 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6c0613c94e462c688160d HTTP/1.1" 200 None 2025-09-26 17:06:13,711 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:06:13,723 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d651ddcca86fb17472aa24 HTTP/1.1" 200 None 2025-09-26 17:06:13,724 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:06:13,762 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6c0613c94e462c688160d HTTP/1.1" 200 None 2025-09-26 17:06:13,763 - lava_callback - INFO - Completed processing callback for node 68d6c0613c94e462c688160d INFO: 172.21.0.1:48840 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48850 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:06:18,591 - lava_callback - INFO - Received callback for node 68d6c7883c94e462c688189b INFO: 172.21.0.1:48852 - "POST /node/68d6c7883c94e462c688189b HTTP/1.0" 202 Accepted 2025-09-26 17:06:18,597 - lava_callback - INFO - Starting async job submit for node 68d6c7883c94e462c688189b 2025-09-26 17:06:18,618 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:06:18,642 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6c7883c94e462c688189b HTTP/1.1" 200 None 2025-09-26 17:06:18,642 - lava_callback - DEBUG - Processing node 68d6c7883c94e462c688189b: baseline-arm64 2025-09-26 17:06:18,938 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 17:06:19,041 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 17:06:19,042 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6c7883c94e462c688189b/log.txt.gz 2025-09-26 17:06:19,049 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 17:06:19,158 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 17:06:19,159 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6c7883c94e462c688189b/lava_callback.json.gz 2025-09-26 17:06:19,180 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:06:19,202 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6c7883c94e462c688189b HTTP/1.1" 200 None 2025-09-26 17:06:19,203 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:06:19,213 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6b9dd2f83206171a92b48 HTTP/1.1" 200 None 2025-09-26 17:06:19,214 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:06:19,243 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6c7883c94e462c688189b HTTP/1.1" 200 None 2025-09-26 17:06:19,244 - lava_callback - INFO - Completed processing callback for node 68d6c7883c94e462c688189b INFO: 172.21.0.1:38252 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38260 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:06:30,104 - lava_callback - INFO - Received callback for node 68d42505a5c2c05277bc71c7 INFO: 172.21.0.1:58208 - "POST /node/68d42505a5c2c05277bc71c7 HTTP/1.0" 202 Accepted 2025-09-26 17:06:30,110 - lava_callback - INFO - Starting async job submit for node 68d42505a5c2c05277bc71c7 2025-09-26 17:06:30,110 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:06:30,135 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d42505a5c2c05277bc71c7 HTTP/1.1" 200 None 2025-09-26 17:06:30,135 - lava_callback - DEBUG - Processing node 68d42505a5c2c05277bc71c7: baseline-arm64-clabbe 2025-09-26 17:06:30,135 - lava_callback - ERROR - Error processing callback for node 68d42505a5c2c05277bc71c7: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 17:06:30,757 - lava_callback - INFO - Received callback for node 68d42509a5c2c05277bc71ca INFO: 172.21.0.1:58212 - "POST /node/68d42509a5c2c05277bc71ca HTTP/1.0" 202 Accepted 2025-09-26 17:06:30,763 - lava_callback - INFO - Starting async job submit for node 68d42509a5c2c05277bc71ca 2025-09-26 17:06:30,763 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:06:30,808 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d42509a5c2c05277bc71ca HTTP/1.1" 200 None 2025-09-26 17:06:30,809 - lava_callback - DEBUG - Processing node 68d42509a5c2c05277bc71ca: baseline-arm64-clabbe 2025-09-26 17:06:30,809 - lava_callback - ERROR - Error processing callback for node 68d42509a5c2c05277bc71ca: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 17:06:31,411 - lava_callback - INFO - Received callback for node 68d4250fa5c2c05277bc71e5 INFO: 172.21.0.1:58226 - "POST /node/68d4250fa5c2c05277bc71e5 HTTP/1.0" 202 Accepted 2025-09-26 17:06:31,417 - lava_callback - INFO - Starting async job submit for node 68d4250fa5c2c05277bc71e5 2025-09-26 17:06:31,417 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:06:31,435 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4250fa5c2c05277bc71e5 HTTP/1.1" 200 None 2025-09-26 17:06:31,435 - lava_callback - DEBUG - Processing node 68d4250fa5c2c05277bc71e5: baseline-arm64-clabbe 2025-09-26 17:06:31,435 - lava_callback - ERROR - Error processing callback for node 68d4250fa5c2c05277bc71e5: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:58230 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34274 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:06:50,013 - lava_callback - INFO - Received callback for node 68d6c7863c94e462c6881889 INFO: 172.21.0.1:53030 - "POST /node/68d6c7863c94e462c6881889 HTTP/1.0" 202 Accepted 2025-09-26 17:06:50,019 - lava_callback - INFO - Starting async job submit for node 68d6c7863c94e462c6881889 2025-09-26 17:06:50,042 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:06:50,054 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6c7863c94e462c6881889 HTTP/1.1" 200 None 2025-09-26 17:06:50,055 - lava_callback - DEBUG - Processing node 68d6c7863c94e462c6881889: baseline-arm64 2025-09-26 17:06:50,145 - lava_callback - INFO - Received callback for node 68d42514a5c2c05277bc71e8 INFO: 172.21.0.1:53036 - "POST /node/68d42514a5c2c05277bc71e8 HTTP/1.0" 202 Accepted 2025-09-26 17:06:50,167 - lava_callback - INFO - Starting async job submit for node 68d42514a5c2c05277bc71e8 2025-09-26 17:06:50,167 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:06:50,240 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d42514a5c2c05277bc71e8 HTTP/1.1" 200 None 2025-09-26 17:06:50,256 - lava_callback - DEBUG - Processing node 68d42514a5c2c05277bc71e8: baseline-arm64-clabbe 2025-09-26 17:06:50,256 - lava_callback - ERROR - Error processing callback for node 68d42514a5c2c05277bc71e8: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 17:06:50,341 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 17:06:50,459 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 17:06:50,460 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6c7863c94e462c6881889/log.txt.gz 2025-09-26 17:06:50,466 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 17:06:50,591 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 17:06:50,591 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6c7863c94e462c6881889/lava_callback.json.gz 2025-09-26 17:06:50,612 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:06:50,644 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6c7863c94e462c6881889 HTTP/1.1" 200 None 2025-09-26 17:06:50,645 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:06:50,657 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6b9dd2f83206171a92b48 HTTP/1.1" 200 None 2025-09-26 17:06:50,658 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:06:50,684 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6c7863c94e462c6881889 HTTP/1.1" 200 None 2025-09-26 17:06:50,685 - lava_callback - INFO - Completed processing callback for node 68d6c7863c94e462c6881889 2025-09-26 17:06:56,322 - lava_callback - INFO - Received callback for node 68d6c7823c94e462c688186e INFO: 172.21.0.1:53050 - "POST /node/68d6c7823c94e462c688186e HTTP/1.0" 202 Accepted 2025-09-26 17:06:56,328 - lava_callback - INFO - Starting async job submit for node 68d6c7823c94e462c688186e 2025-09-26 17:06:56,349 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:06:56,361 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6c7823c94e462c688186e HTTP/1.1" 200 None 2025-09-26 17:06:56,361 - lava_callback - DEBUG - Processing node 68d6c7823c94e462c688186e: baseline-arm64 INFO: 172.21.0.1:53054 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:06:56,602 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 17:06:56,711 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 17:06:56,712 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6c7823c94e462c688186e/log.txt.gz 2025-09-26 17:06:56,718 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 17:06:56,830 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 17:06:56,831 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6c7823c94e462c688186e/lava_callback.json.gz 2025-09-26 17:06:56,852 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:06:56,873 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6c7823c94e462c688186e HTTP/1.1" 200 None 2025-09-26 17:06:56,874 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:06:56,886 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6b9dd2f83206171a92b48 HTTP/1.1" 200 None 2025-09-26 17:06:56,887 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:06:56,914 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6c7823c94e462c688186e HTTP/1.1" 200 None 2025-09-26 17:06:56,915 - lava_callback - INFO - Completed processing callback for node 68d6c7823c94e462c688186e INFO: 172.21.0.1:43668 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:07:05,479 - lava_callback - INFO - Received callback for node 68d47810a5c2c05277bccc01 INFO: 172.21.0.1:43674 - "POST /node/68d47810a5c2c05277bccc01 HTTP/1.0" 202 Accepted 2025-09-26 17:07:05,489 - lava_callback - INFO - Starting async job submit for node 68d47810a5c2c05277bccc01 2025-09-26 17:07:05,508 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:07:05,528 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d47810a5c2c05277bccc01 HTTP/1.1" 200 None 2025-09-26 17:07:05,528 - lava_callback - DEBUG - Processing node 68d47810a5c2c05277bccc01: baseline-arm 2025-09-26 17:07:05,785 - lava_callback - ERROR - Error processing callback for node 68d47810a5c2c05277bccc01: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:43680 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:07:10,868 - lava_callback - INFO - Received callback for node 68d6c7843c94e462c688187a INFO: 172.21.0.1:55232 - "POST /node/68d6c7843c94e462c688187a HTTP/1.0" 202 Accepted 2025-09-26 17:07:10,874 - lava_callback - INFO - Starting async job submit for node 68d6c7843c94e462c688187a 2025-09-26 17:07:10,893 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:07:10,905 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6c7843c94e462c688187a HTTP/1.1" 200 None 2025-09-26 17:07:10,905 - lava_callback - DEBUG - Processing node 68d6c7843c94e462c688187a: baseline-arm64 2025-09-26 17:07:11,254 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 17:07:11,367 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 17:07:11,368 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6c7843c94e462c688187a/log.txt.gz 2025-09-26 17:07:11,377 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 17:07:11,509 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 17:07:11,510 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6c7843c94e462c688187a/lava_callback.json.gz 2025-09-26 17:07:11,529 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:07:11,542 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6c7843c94e462c688187a HTTP/1.1" 200 None 2025-09-26 17:07:11,543 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:07:11,554 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6b9dd2f83206171a92b48 HTTP/1.1" 200 None 2025-09-26 17:07:11,555 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:07:11,588 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6c7843c94e462c688187a HTTP/1.1" 200 None 2025-09-26 17:07:11,589 - lava_callback - INFO - Completed processing callback for node 68d6c7843c94e462c688187a INFO: 172.21.0.1:55236 - "GET /api/metrics HTTP/1.0" 200 OK 2025-09-26 17:07:15,502 - lava_callback - INFO - Received callback for node 68d6c7843c94e462c6881880 INFO: 172.21.0.1:55250 - "POST /node/68d6c7843c94e462c6881880 HTTP/1.0" 202 Accepted 2025-09-26 17:07:15,509 - lava_callback - INFO - Starting async job submit for node 68d6c7843c94e462c6881880 2025-09-26 17:07:15,529 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:07:15,544 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6c7843c94e462c6881880 HTTP/1.1" 200 None 2025-09-26 17:07:15,544 - lava_callback - DEBUG - Processing node 68d6c7843c94e462c6881880: baseline-arm64 2025-09-26 17:07:15,841 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 17:07:16,013 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 17:07:16,014 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6c7843c94e462c6881880/log.txt.gz 2025-09-26 17:07:16,022 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 17:07:16,149 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 17:07:16,149 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6c7843c94e462c6881880/lava_callback.json.gz 2025-09-26 17:07:16,170 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:07:16,205 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6c7843c94e462c6881880 HTTP/1.1" 200 None 2025-09-26 17:07:16,206 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:07:16,217 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6b9dd2f83206171a92b48 HTTP/1.1" 200 None 2025-09-26 17:07:16,218 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:07:16,248 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6c7843c94e462c6881880 HTTP/1.1" 200 None 2025-09-26 17:07:16,248 - lava_callback - INFO - Completed processing callback for node 68d6c7843c94e462c6881880 INFO: 172.21.0.1:55254 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:07:19,414 - lava_callback - INFO - Received callback for node 68d6c7873c94e462c6881895 INFO: 172.21.0.1:44128 - "POST /node/68d6c7873c94e462c6881895 HTTP/1.0" 202 Accepted 2025-09-26 17:07:19,425 - lava_callback - INFO - Starting async job submit for node 68d6c7873c94e462c6881895 2025-09-26 17:07:19,457 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:07:19,489 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6c7873c94e462c6881895 HTTP/1.1" 200 None 2025-09-26 17:07:19,489 - lava_callback - DEBUG - Processing node 68d6c7873c94e462c6881895: baseline-arm64 2025-09-26 17:07:19,909 - lava_callback - INFO - Received callback for node 68d6c7883c94e462c6881898 INFO: 172.21.0.1:44142 - "POST /node/68d6c7883c94e462c6881898 HTTP/1.0" 202 Accepted 2025-09-26 17:07:19,930 - lava_callback - INFO - Starting async job submit for node 68d6c7883c94e462c6881898 2025-09-26 17:07:19,998 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:07:20,054 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6c7883c94e462c6881898 HTTP/1.1" 200 None 2025-09-26 17:07:20,059 - lava_callback - DEBUG - Processing node 68d6c7883c94e462c6881898: baseline-arm64 2025-09-26 17:07:20,290 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 17:07:20,541 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 17:07:20,568 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6c7873c94e462c6881895/log.txt.gz 2025-09-26 17:07:20,607 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 17:07:20,608 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 17:07:20,781 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 17:07:20,782 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6c7873c94e462c6881895/lava_callback.json.gz 2025-09-26 17:07:20,812 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 17:07:20,819 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:07:20,819 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6c7883c94e462c6881898/log.txt.gz 2025-09-26 17:07:20,828 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 17:07:20,840 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6c7873c94e462c6881895 HTTP/1.1" 200 None 2025-09-26 17:07:20,842 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:07:20,860 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6b9dd2f83206171a92b48 HTTP/1.1" 200 None 2025-09-26 17:07:20,862 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:07:20,907 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6c7873c94e462c6881895 HTTP/1.1" 200 None 2025-09-26 17:07:20,908 - lava_callback - INFO - Completed processing callback for node 68d6c7873c94e462c6881895 2025-09-26 17:07:21,060 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 17:07:21,061 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6c7883c94e462c6881898/lava_callback.json.gz 2025-09-26 17:07:21,098 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:07:21,121 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6c7883c94e462c6881898 HTTP/1.1" 200 None 2025-09-26 17:07:21,124 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:07:21,147 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6b9dd2f83206171a92b48 HTTP/1.1" 200 None 2025-09-26 17:07:21,148 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:07:21,182 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6c7883c94e462c6881898 HTTP/1.1" 200 None 2025-09-26 17:07:21,183 - lava_callback - INFO - Completed processing callback for node 68d6c7883c94e462c6881898 2025-09-26 17:07:23,223 - lava_callback - INFO - Received callback for node 68d4238ca5c2c05277bc6f35 INFO: 172.21.0.1:44156 - "POST /node/68d4238ca5c2c05277bc6f35 HTTP/1.0" 202 Accepted 2025-09-26 17:07:23,236 - lava_callback - INFO - Starting async job submit for node 68d4238ca5c2c05277bc6f35 2025-09-26 17:07:23,612 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:07:23,630 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4238ca5c2c05277bc6f35 HTTP/1.1" 200 None 2025-09-26 17:07:23,630 - lava_callback - DEBUG - Processing node 68d4238ca5c2c05277bc6f35: ltp-controllers 2025-09-26 17:07:26,475 - lava_callback - ERROR - Error processing callback for node 68d4238ca5c2c05277bc6f35: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:44170 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:07:27,291 - lava_callback - INFO - Received callback for node 68d6c7843c94e462c688187d INFO: 172.21.0.1:44186 - "POST /node/68d6c7843c94e462c688187d HTTP/1.0" 202 Accepted 2025-09-26 17:07:27,297 - lava_callback - INFO - Starting async job submit for node 68d6c7843c94e462c688187d 2025-09-26 17:07:27,317 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:07:27,339 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6c7843c94e462c688187d HTTP/1.1" 200 None 2025-09-26 17:07:27,339 - lava_callback - DEBUG - Processing node 68d6c7843c94e462c688187d: baseline-arm64 2025-09-26 17:07:27,617 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 17:07:27,720 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 17:07:27,720 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6c7843c94e462c688187d/log.txt.gz 2025-09-26 17:07:27,727 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 17:07:27,837 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 17:07:27,837 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6c7843c94e462c688187d/lava_callback.json.gz 2025-09-26 17:07:27,859 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:07:27,870 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6c7843c94e462c688187d HTTP/1.1" 200 None 2025-09-26 17:07:27,871 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:07:27,894 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6b9dd2f83206171a92b48 HTTP/1.1" 200 None 2025-09-26 17:07:27,895 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:07:27,922 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6c7843c94e462c688187d HTTP/1.1" 200 None 2025-09-26 17:07:27,923 - lava_callback - INFO - Completed processing callback for node 68d6c7843c94e462c688187d INFO: 172.21.0.1:44202 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:07:34,447 - lava_callback - INFO - Received callback for node 68d42390a5c2c05277bc6f45 INFO: 172.21.0.1:60976 - "POST /node/68d42390a5c2c05277bc6f45 HTTP/1.0" 202 Accepted 2025-09-26 17:07:34,455 - lava_callback - INFO - Starting async job submit for node 68d42390a5c2c05277bc6f45 2025-09-26 17:07:34,563 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:07:34,576 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d42390a5c2c05277bc6f45 HTTP/1.1" 200 None 2025-09-26 17:07:34,576 - lava_callback - DEBUG - Processing node 68d42390a5c2c05277bc6f45: ltp-fs-bind 2025-09-26 17:07:35,099 - lava_callback - ERROR - Error processing callback for node 68d42390a5c2c05277bc6f45: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None 2025-09-26 17:07:35,104 - lava_callback - INFO - Received callback for node 68d6c7853c94e462c6881886 INFO: 172.21.0.1:60988 - "POST /node/68d6c7853c94e462c6881886 HTTP/1.0" 202 Accepted 2025-09-26 17:07:35,110 - lava_callback - INFO - Starting async job submit for node 68d6c7853c94e462c6881886 2025-09-26 17:07:35,130 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:07:35,143 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6c7853c94e462c6881886 HTTP/1.1" 200 None 2025-09-26 17:07:35,143 - lava_callback - DEBUG - Processing node 68d6c7853c94e462c6881886: baseline-arm64 2025-09-26 17:07:35,381 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 17:07:35,508 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 17:07:35,509 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6c7853c94e462c6881886/log.txt.gz 2025-09-26 17:07:35,515 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 17:07:35,707 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 17:07:35,708 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6c7853c94e462c6881886/lava_callback.json.gz 2025-09-26 17:07:35,728 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:07:35,740 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6c7853c94e462c6881886 HTTP/1.1" 200 None 2025-09-26 17:07:35,741 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:07:35,752 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6b9dd2f83206171a92b48 HTTP/1.1" 200 None 2025-09-26 17:07:35,753 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:07:35,781 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6c7853c94e462c6881886 HTTP/1.1" 200 None 2025-09-26 17:07:35,781 - lava_callback - INFO - Completed processing callback for node 68d6c7853c94e462c6881886 2025-09-26 17:07:36,295 - lava_callback - INFO - Received callback for node 68d6c7813c94e462c688186b INFO: 172.21.0.1:60998 - "POST /node/68d6c7813c94e462c688186b HTTP/1.0" 202 Accepted 2025-09-26 17:07:36,301 - lava_callback - INFO - Starting async job submit for node 68d6c7813c94e462c688186b 2025-09-26 17:07:36,322 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:07:36,339 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6c7813c94e462c688186b HTTP/1.1" 200 None 2025-09-26 17:07:36,339 - lava_callback - DEBUG - Processing node 68d6c7813c94e462c688186b: baseline-arm64 INFO: 172.21.0.1:32772 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:07:36,599 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 17:07:36,723 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 17:07:36,724 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6c7813c94e462c688186b/log.txt.gz 2025-09-26 17:07:36,730 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 17:07:36,839 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 17:07:36,840 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6c7813c94e462c688186b/lava_callback.json.gz 2025-09-26 17:07:36,861 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:07:36,872 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6c7813c94e462c688186b HTTP/1.1" 200 None 2025-09-26 17:07:36,873 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:07:36,884 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6b9dd2f83206171a92b48 HTTP/1.1" 200 None 2025-09-26 17:07:36,885 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:07:36,912 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6c7813c94e462c688186b HTTP/1.1" 200 None 2025-09-26 17:07:36,913 - lava_callback - INFO - Completed processing callback for node 68d6c7813c94e462c688186b INFO: 172.21.0.1:38366 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:07:54,413 - lava_callback - INFO - Received callback for node 68d6c7903c94e462c68818c5 INFO: 172.21.0.1:41206 - "POST /node/68d6c7903c94e462c68818c5 HTTP/1.0" 202 Accepted 2025-09-26 17:07:54,419 - lava_callback - INFO - Starting async job submit for node 68d6c7903c94e462c68818c5 2025-09-26 17:07:54,440 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:07:54,463 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6c7903c94e462c68818c5 HTTP/1.1" 200 None 2025-09-26 17:07:54,464 - lava_callback - DEBUG - Processing node 68d6c7903c94e462c68818c5: baseline-arm64-broonie 2025-09-26 17:07:54,720 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 17:07:54,856 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 17:07:54,857 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-68d6c7903c94e462c68818c5/log.txt.gz 2025-09-26 17:07:54,864 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 17:07:54,971 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 17:07:54,971 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-68d6c7903c94e462c68818c5/lava_callback.json.gz 2025-09-26 17:07:54,993 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:07:55,005 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6c7903c94e462c68818c5 HTTP/1.1" 200 None 2025-09-26 17:07:55,006 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:07:55,017 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6b9dd2f83206171a92b48 HTTP/1.1" 200 None 2025-09-26 17:07:55,018 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:07:55,045 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6c7903c94e462c68818c5 HTTP/1.1" 200 None 2025-09-26 17:07:55,046 - lava_callback - INFO - Completed processing callback for node 68d6c7903c94e462c68818c5 INFO: 172.21.0.1:41218 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57698 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57714 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:08:12,825 - lava_callback - INFO - Received callback for node 68d6c0623c94e462c6881610 INFO: 172.21.0.1:34996 - "POST /node/68d6c0623c94e462c6881610 HTTP/1.0" 202 Accepted 2025-09-26 17:08:12,833 - lava_callback - INFO - Starting async job submit for node 68d6c0623c94e462c6881610 2025-09-26 17:08:12,871 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:08:12,911 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6c0623c94e462c6881610 HTTP/1.1" 200 None 2025-09-26 17:08:12,912 - lava_callback - DEBUG - Processing node 68d6c0623c94e462c6881610: baseline-arm64 2025-09-26 17:08:13,922 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 17:08:14,040 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 17:08:14,041 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6c0623c94e462c6881610/log.txt.gz 2025-09-26 17:08:14,060 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 17:08:14,273 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 17:08:14,274 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6c0623c94e462c6881610/lava_callback.json.gz DEBUG: Setup failed for 68d6c0623c94e462c6881610. Transitting job node result: incomplete -> fail 2025-09-26 17:08:14,351 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:08:14,363 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6c0623c94e462c6881610 HTTP/1.1" 200 None 2025-09-26 17:08:14,364 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:08:14,376 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d651ddcca86fb17472aa24 HTTP/1.1" 200 None 2025-09-26 17:08:14,378 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:08:14,407 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6c0623c94e462c6881610 HTTP/1.1" 200 None 2025-09-26 17:08:14,408 - lava_callback - INFO - Completed processing callback for node 68d6c0623c94e462c6881610 INFO: 172.21.0.1:35004 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35006 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:08:18,281 - lava_callback - INFO - Received callback for node 68d6c7e33c94e462c6881904 INFO: 172.21.0.1:35016 - "POST /node/68d6c7e33c94e462c6881904 HTTP/1.0" 202 Accepted 2025-09-26 17:08:18,292 - lava_callback - INFO - Starting async job submit for node 68d6c7e33c94e462c6881904 2025-09-26 17:08:18,342 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:08:18,355 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6c7e33c94e462c6881904 HTTP/1.1" 200 None 2025-09-26 17:08:18,356 - lava_callback - DEBUG - Processing node 68d6c7e33c94e462c6881904: baseline-arm64-kontron 2025-09-26 17:08:18,720 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 17:08:18,854 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 17:08:18,855 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-kontron-68d6c7e33c94e462c6881904/log.txt.gz 2025-09-26 17:08:18,861 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 17:08:19,008 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 17:08:19,009 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-kontron-68d6c7e33c94e462c6881904/lava_callback.json.gz 2025-09-26 17:08:19,055 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:08:19,090 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6c7e33c94e462c6881904 HTTP/1.1" 200 None 2025-09-26 17:08:19,091 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:08:19,109 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6b9dd2f83206171a92b48 HTTP/1.1" 200 None 2025-09-26 17:08:19,110 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:08:19,139 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6c7e33c94e462c6881904 HTTP/1.1" 200 None 2025-09-26 17:08:19,140 - lava_callback - INFO - Completed processing callback for node 68d6c7e33c94e462c6881904 INFO: 172.21.0.1:59068 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59076 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42294 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39796 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:08:51,023 - lava_callback - INFO - Received callback for node 68d69aad2f83206171a92241 INFO: 172.21.0.1:42422 - "POST /node/68d69aad2f83206171a92241 HTTP/1.0" 202 Accepted 2025-09-26 17:08:51,032 - lava_callback - INFO - Starting async job submit for node 68d69aad2f83206171a92241 2025-09-26 17:08:51,071 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:08:51,084 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d69aad2f83206171a92241 HTTP/1.1" 200 None 2025-09-26 17:08:51,084 - lava_callback - DEBUG - Processing node 68d69aad2f83206171a92241: baseline-arm64 2025-09-26 17:08:52,119 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 17:08:52,289 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 17:08:52,290 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d69aad2f83206171a92241/log.txt.gz 2025-09-26 17:08:52,316 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 17:08:52,513 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 17:08:52,514 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d69aad2f83206171a92241/lava_callback.json.gz DEBUG: 68d69aad2f83206171a92241 Transitting job node result: pass -> incomplete 2025-09-26 17:08:52,553 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:08:52,566 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d69aad2f83206171a92241 HTTP/1.1" 200 None 2025-09-26 17:08:52,567 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:08:52,578 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d5dc8530c58dac7d9410cb HTTP/1.1" 200 None 2025-09-26 17:08:52,579 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:08:52,610 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d69aad2f83206171a92241 HTTP/1.1" 200 None 2025-09-26 17:08:52,610 - lava_callback - INFO - Completed processing callback for node 68d69aad2f83206171a92241 INFO: 172.21.0.1:42434 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35508 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35518 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45974 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45976 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:09:18,780 - lava_callback - INFO - Received callback for node 68d428b9a5c2c05277bc7638 INFO: 172.21.0.1:45978 - "POST /node/68d428b9a5c2c05277bc7638 HTTP/1.0" 202 Accepted 2025-09-26 17:09:18,789 - lava_callback - INFO - Starting async job submit for node 68d428b9a5c2c05277bc7638 2025-09-26 17:09:18,823 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:09:18,857 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d428b9a5c2c05277bc7638 HTTP/1.1" 200 None 2025-09-26 17:09:18,858 - lava_callback - DEBUG - Processing node 68d428b9a5c2c05277bc7638: baseline-arm-broonie 2025-09-26 17:09:19,051 - lava_callback - ERROR - Error processing callback for node 68d428b9a5c2c05277bc7638: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None 2025-09-26 17:09:22,553 - lava_callback - INFO - Received callback for node 68d428afa5c2c05277bc7623 INFO: 172.21.0.1:54928 - "POST /node/68d428afa5c2c05277bc7623 HTTP/1.0" 202 Accepted 2025-09-26 17:09:22,559 - lava_callback - INFO - Starting async job submit for node 68d428afa5c2c05277bc7623 2025-09-26 17:09:22,589 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:09:22,606 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d428afa5c2c05277bc7623 HTTP/1.1" 200 None 2025-09-26 17:09:22,607 - lava_callback - DEBUG - Processing node 68d428afa5c2c05277bc7623: baseline-arm-broonie 2025-09-26 17:09:22,839 - lava_callback - ERROR - Error processing callback for node 68d428afa5c2c05277bc7623: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:54942 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54948 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38846 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45692 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:09:56,025 - lava_callback - INFO - Received callback for node 68d6a7062f83206171a92452 INFO: 172.21.0.1:50764 - "POST /node/68d6a7062f83206171a92452 HTTP/1.0" 202 Accepted 2025-09-26 17:09:56,032 - lava_callback - INFO - Starting async job submit for node 68d6a7062f83206171a92452 2025-09-26 17:09:56,052 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:09:56,064 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6a7062f83206171a92452 HTTP/1.1" 200 None 2025-09-26 17:09:56,065 - lava_callback - DEBUG - Processing node 68d6a7062f83206171a92452: baseline-arm64 2025-09-26 17:09:56,487 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.21.0.1:50774 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:09:56,755 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 17:09:56,755 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6a7062f83206171a92452/log.txt.gz 2025-09-26 17:09:56,765 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 17:09:56,891 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 17:09:56,891 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6a7062f83206171a92452/lava_callback.json.gz 2025-09-26 17:09:56,912 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:09:56,933 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6a7062f83206171a92452 HTTP/1.1" 200 None 2025-09-26 17:09:56,935 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:09:56,990 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d5d90f2be2da466069ac22 HTTP/1.1" 200 None 2025-09-26 17:09:56,991 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:09:57,022 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6a7062f83206171a92452 HTTP/1.1" 200 None 2025-09-26 17:09:57,022 - lava_callback - INFO - Completed processing callback for node 68d6a7062f83206171a92452 2025-09-26 17:09:58,864 - lava_callback - INFO - Received callback for node 68d6a7052f83206171a9244f INFO: 172.21.0.1:50778 - "POST /node/68d6a7052f83206171a9244f HTTP/1.0" 202 Accepted 2025-09-26 17:09:58,870 - lava_callback - INFO - Starting async job submit for node 68d6a7052f83206171a9244f 2025-09-26 17:09:58,890 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:09:58,903 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6a7052f83206171a9244f HTTP/1.1" 200 None 2025-09-26 17:09:58,904 - lava_callback - DEBUG - Processing node 68d6a7052f83206171a9244f: baseline-arm64 2025-09-26 17:09:59,326 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 17:09:59,483 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 17:09:59,484 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6a7052f83206171a9244f/log.txt.gz 2025-09-26 17:09:59,493 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 17:09:59,614 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 17:09:59,615 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6a7052f83206171a9244f/lava_callback.json.gz 2025-09-26 17:09:59,636 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:09:59,648 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6a7052f83206171a9244f HTTP/1.1" 200 None 2025-09-26 17:09:59,649 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:09:59,660 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d5d90f2be2da466069ac22 HTTP/1.1" 200 None 2025-09-26 17:09:59,661 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:09:59,707 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6a7052f83206171a9244f HTTP/1.1" 200 None 2025-09-26 17:09:59,707 - lava_callback - INFO - Completed processing callback for node 68d6a7052f83206171a9244f INFO: 172.21.0.1:47226 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47234 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39566 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39582 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44058 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44060 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:10:33,271 - lava_callback - INFO - Received callback for node 68d6a71e2f83206171a9248f INFO: 172.21.0.1:50958 - "POST /node/68d6a71e2f83206171a9248f HTTP/1.0" 202 Accepted 2025-09-26 17:10:33,278 - lava_callback - INFO - Starting async job submit for node 68d6a71e2f83206171a9248f 2025-09-26 17:10:33,299 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:10:33,312 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6a71e2f83206171a9248f HTTP/1.1" 200 None 2025-09-26 17:10:33,313 - lava_callback - DEBUG - Processing node 68d6a71e2f83206171a9248f: baseline-arm64 2025-09-26 17:10:33,723 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 17:10:33,853 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 17:10:33,853 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6a71e2f83206171a9248f/log.txt.gz 2025-09-26 17:10:33,863 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 17:10:34,007 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 17:10:34,008 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6a71e2f83206171a9248f/lava_callback.json.gz 2025-09-26 17:10:34,029 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:10:34,063 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6a71e2f83206171a9248f HTTP/1.1" 200 None 2025-09-26 17:10:34,064 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:10:34,079 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d648ed5d233c363a762351 HTTP/1.1" 200 None 2025-09-26 17:10:34,080 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:10:34,110 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6a71e2f83206171a9248f HTTP/1.1" 200 None 2025-09-26 17:10:34,111 - lava_callback - INFO - Completed processing callback for node 68d6a71e2f83206171a9248f INFO: 172.21.0.1:50962 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:10:38,028 - lava_callback - INFO - Received callback for node 68d6c8833c94e462c6881946 INFO: 172.21.0.1:50964 - "POST /node/68d6c8833c94e462c6881946 HTTP/1.0" 202 Accepted 2025-09-26 17:10:38,034 - lava_callback - INFO - Starting async job submit for node 68d6c8833c94e462c6881946 2025-09-26 17:10:38,068 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:10:38,085 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6c8833c94e462c6881946 HTTP/1.1" 200 None 2025-09-26 17:10:38,086 - lava_callback - DEBUG - Processing node 68d6c8833c94e462c6881946: baseline-arm64-kontron 2025-09-26 17:10:38,326 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 17:10:38,438 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 17:10:38,439 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-kontron-68d6c8833c94e462c6881946/log.txt.gz 2025-09-26 17:10:38,445 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 17:10:38,573 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 17:10:38,574 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-kontron-68d6c8833c94e462c6881946/lava_callback.json.gz 2025-09-26 17:10:38,609 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:10:38,642 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6c8833c94e462c6881946 HTTP/1.1" 200 None 2025-09-26 17:10:38,643 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:10:38,657 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6b9dd2f83206171a92b48 HTTP/1.1" 200 None 2025-09-26 17:10:38,658 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:10:38,680 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6c8833c94e462c6881946 HTTP/1.1" 200 None 2025-09-26 17:10:38,681 - lava_callback - INFO - Completed processing callback for node 68d6c8833c94e462c6881946 INFO: 172.21.0.1:37648 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:10:54,652 - lava_callback - INFO - Received callback for node 68d6c12c3c94e462c6881668 INFO: 172.21.0.1:42560 - "POST /node/68d6c12c3c94e462c6881668 HTTP/1.0" 202 Accepted 2025-09-26 17:10:54,658 - lava_callback - INFO - Starting async job submit for node 68d6c12c3c94e462c6881668 2025-09-26 17:10:54,702 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:10:54,714 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6c12c3c94e462c6881668 HTTP/1.1" 200 None 2025-09-26 17:10:54,715 - lava_callback - DEBUG - Processing node 68d6c12c3c94e462c6881668: baseline-x86 2025-09-26 17:10:55,401 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 17:10:55,512 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 17:10:55,512 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-68d6c12c3c94e462c6881668/log.txt.gz 2025-09-26 17:10:55,529 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 17:10:55,700 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 17:10:55,701 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-68d6c12c3c94e462c6881668/lava_callback.json.gz DEBUG: Setup failed for 68d6c12c3c94e462c6881668. Transitting job node result: incomplete -> fail 2025-09-26 17:10:55,787 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:10:55,799 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6c12c3c94e462c6881668 HTTP/1.1" 200 None 2025-09-26 17:10:55,800 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:10:55,812 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6b9f22f83206171a92b66 HTTP/1.1" 200 None 2025-09-26 17:10:55,813 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:10:55,842 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6c12c3c94e462c6881668 HTTP/1.1" 200 None 2025-09-26 17:10:55,843 - lava_callback - INFO - Completed processing callback for node 68d6c12c3c94e462c6881668 INFO: 172.21.0.1:42570 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47566 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47570 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37234 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37250 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43304 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43312 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55840 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:11:42,697 - lava_callback - INFO - Received callback for node 68d428bca5c2c05277bc763b INFO: 172.21.0.1:42128 - "POST /node/68d428bca5c2c05277bc763b HTTP/1.0" 202 Accepted 2025-09-26 17:11:42,703 - lava_callback - INFO - Starting async job submit for node 68d428bca5c2c05277bc763b 2025-09-26 17:11:42,727 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:11:42,742 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d428bca5c2c05277bc763b HTTP/1.1" 200 None 2025-09-26 17:11:42,742 - lava_callback - DEBUG - Processing node 68d428bca5c2c05277bc763b: baseline-arm-broonie 2025-09-26 17:11:42,972 - lava_callback - ERROR - Error processing callback for node 68d428bca5c2c05277bc763b: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:42140 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33288 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:11:57,041 - lava_callback - INFO - Received callback for node 68d6c1253c94e462c688165f INFO: 172.21.0.1:33304 - "POST /node/68d6c1253c94e462c688165f HTTP/1.0" 202 Accepted 2025-09-26 17:11:57,048 - lava_callback - INFO - Starting async job submit for node 68d6c1253c94e462c688165f 2025-09-26 17:11:57,106 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:11:57,118 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6c1253c94e462c688165f HTTP/1.1" 200 None 2025-09-26 17:11:57,119 - lava_callback - DEBUG - Processing node 68d6c1253c94e462c688165f: ltp-smoketest 2025-09-26 17:11:57,605 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 17:11:57,746 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 17:11:57,747 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoketest-68d6c1253c94e462c688165f/log.txt.gz 2025-09-26 17:11:57,759 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 17:11:57,877 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 17:11:57,878 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoketest-68d6c1253c94e462c688165f/lava_callback.json.gz 2025-09-26 17:11:58,002 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:11:58,016 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6c1253c94e462c688165f HTTP/1.1" 200 None 2025-09-26 17:11:58,017 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:11:58,036 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d64d07a5c3ab168ecc7a34 HTTP/1.1" 200 None 2025-09-26 17:11:58,037 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:11:58,063 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6c1253c94e462c688165f HTTP/1.1" 200 None 2025-09-26 17:11:58,064 - lava_callback - INFO - Completed processing callback for node 68d6c1253c94e462c688165f INFO: 172.21.0.1:59432 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59442 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52590 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52596 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39026 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39038 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47378 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53326 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48530 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:12:58,437 - lava_callback - INFO - Received callback for node 68d6c90f3c94e462c6881964 INFO: 172.21.0.1:48542 - "POST /node/68d6c90f3c94e462c6881964 HTTP/1.0" 202 Accepted 2025-09-26 17:12:58,444 - lava_callback - INFO - Starting async job submit for node 68d6c90f3c94e462c6881964 2025-09-26 17:12:58,478 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:12:58,502 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6c90f3c94e462c6881964 HTTP/1.1" 200 None 2025-09-26 17:12:58,503 - lava_callback - DEBUG - Processing node 68d6c90f3c94e462c6881964: baseline-arm64-kontron 2025-09-26 17:12:58,758 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 17:12:58,904 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 17:12:58,904 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-kontron-68d6c90f3c94e462c6881964/log.txt.gz 2025-09-26 17:12:58,910 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 17:12:59,047 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 17:12:59,048 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-kontron-68d6c90f3c94e462c6881964/lava_callback.json.gz 2025-09-26 17:12:59,082 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:12:59,101 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6c90f3c94e462c6881964 HTTP/1.1" 200 None 2025-09-26 17:12:59,103 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:12:59,115 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6b9dd2f83206171a92b48 HTTP/1.1" 200 None 2025-09-26 17:12:59,116 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:12:59,145 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6c90f3c94e462c6881964 HTTP/1.1" 200 None 2025-09-26 17:12:59,146 - lava_callback - INFO - Completed processing callback for node 68d6c90f3c94e462c6881964 INFO: 172.21.0.1:53818 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53826 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44370 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44386 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35040 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35044 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46782 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:13:45,714 - lava_callback - INFO - Received callback for node 68d48646a5c2c05277bcd745 INFO: 172.21.0.1:42918 - "POST /node/68d48646a5c2c05277bcd745 HTTP/1.0" 202 Accepted 2025-09-26 17:13:45,720 - lava_callback - INFO - Starting async job submit for node 68d48646a5c2c05277bcd745 2025-09-26 17:13:45,741 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:13:45,753 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d48646a5c2c05277bcd745 HTTP/1.1" 200 None 2025-09-26 17:13:45,753 - lava_callback - DEBUG - Processing node 68d48646a5c2c05277bcd745: baseline-arm 2025-09-26 17:13:45,980 - lava_callback - ERROR - Error processing callback for node 68d48646a5c2c05277bcd745: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:42922 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42454 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60254 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60266 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33394 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33402 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44060 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44072 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48978 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56112 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:14:55,233 - lava_callback - INFO - Received callback for node 68d6c0b33c94e462c688162a INFO: 172.21.0.1:33766 - "POST /node/68d6c0b33c94e462c688162a HTTP/1.0" 202 Accepted 2025-09-26 17:14:55,241 - lava_callback - INFO - Starting async job submit for node 68d6c0b33c94e462c688162a 2025-09-26 17:14:55,281 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:14:55,315 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6c0b33c94e462c688162a HTTP/1.1" 200 None 2025-09-26 17:14:55,315 - lava_callback - DEBUG - Processing node 68d6c0b33c94e462c688162a: baseline-arm64 2025-09-26 17:14:56,349 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 17:14:56,499 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 17:14:56,500 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6c0b33c94e462c688162a/log.txt.gz 2025-09-26 17:14:56,520 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.21.0.1:33772 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:14:56,712 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 17:14:56,712 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6c0b33c94e462c688162a/lava_callback.json.gz DEBUG: Setup failed for 68d6c0b33c94e462c688162a. Transitting job node result: incomplete -> fail 2025-09-26 17:14:56,810 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:14:56,844 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6c0b33c94e462c688162a HTTP/1.1" 200 None 2025-09-26 17:14:56,845 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:14:56,856 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d651ddcca86fb17472aa24 HTTP/1.1" 200 None 2025-09-26 17:14:56,857 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:14:56,906 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6c0b33c94e462c688162a HTTP/1.1" 200 None 2025-09-26 17:14:56,906 - lava_callback - INFO - Completed processing callback for node 68d6c0b33c94e462c688162a INFO: 172.21.0.1:53488 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53496 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:15:08,588 - lava_callback - INFO - Received callback for node 68d6a72b2f83206171a924ab INFO: 172.21.0.1:53510 - "POST /node/68d6a72b2f83206171a924ab HTTP/1.0" 202 Accepted 2025-09-26 17:15:08,595 - lava_callback - INFO - Starting async job submit for node 68d6a72b2f83206171a924ab 2025-09-26 17:15:08,615 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:15:08,628 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6a72b2f83206171a924ab HTTP/1.1" 200 None 2025-09-26 17:15:08,628 - lava_callback - DEBUG - Processing node 68d6a72b2f83206171a924ab: baseline-arm64 2025-09-26 17:15:08,880 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 17:15:09,016 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 17:15:09,017 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6a72b2f83206171a924ab/log.txt.gz 2025-09-26 17:15:09,023 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 17:15:09,164 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 17:15:09,164 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6a72b2f83206171a924ab/lava_callback.json.gz 2025-09-26 17:15:09,185 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:15:09,197 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6a72b2f83206171a924ab HTTP/1.1" 200 None 2025-09-26 17:15:09,198 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:15:09,209 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d648ed5d233c363a762351 HTTP/1.1" 200 None 2025-09-26 17:15:09,211 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:15:09,237 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6a72b2f83206171a924ab HTTP/1.1" 200 None 2025-09-26 17:15:09,238 - lava_callback - INFO - Completed processing callback for node 68d6a72b2f83206171a924ab INFO: 172.21.0.1:55528 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55536 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47068 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47072 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:15:30,810 - lava_callback - INFO - Received callback for node 68d6a8112f83206171a925cd INFO: 172.21.0.1:48322 - "POST /node/68d6a8112f83206171a925cd HTTP/1.0" 202 Accepted 2025-09-26 17:15:30,819 - lava_callback - INFO - Starting async job submit for node 68d6a8112f83206171a925cd 2025-09-26 17:15:30,842 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:15:30,857 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6a8112f83206171a925cd HTTP/1.1" 200 None 2025-09-26 17:15:30,857 - lava_callback - DEBUG - Processing node 68d6a8112f83206171a925cd: baseline-arm64 2025-09-26 17:15:31,204 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 17:15:31,314 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 17:15:31,315 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6a8112f83206171a925cd/log.txt.gz 2025-09-26 17:15:31,323 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 17:15:31,430 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 17:15:31,431 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6a8112f83206171a925cd/lava_callback.json.gz 2025-09-26 17:15:31,453 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:15:31,465 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6a8112f83206171a925cd HTTP/1.1" 200 None 2025-09-26 17:15:31,466 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:15:31,479 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d64d12a5c3ab168ecc7a3d HTTP/1.1" 200 None 2025-09-26 17:15:31,480 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:15:31,508 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6a8112f83206171a925cd HTTP/1.1" 200 None 2025-09-26 17:15:31,508 - lava_callback - INFO - Completed processing callback for node 68d6a8112f83206171a925cd INFO: 172.21.0.1:48332 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:15:46,236 - lava_callback - INFO - Received callback for node 68d48647a5c2c05277bcd748 INFO: 172.21.0.1:56696 - "POST /node/68d48647a5c2c05277bcd748 HTTP/1.0" 202 Accepted 2025-09-26 17:15:46,242 - lava_callback - INFO - Starting async job submit for node 68d48647a5c2c05277bcd748 2025-09-26 17:15:46,263 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:15:46,275 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d48647a5c2c05277bcd748 HTTP/1.1" 200 None 2025-09-26 17:15:46,275 - lava_callback - DEBUG - Processing node 68d48647a5c2c05277bcd748: baseline-arm 2025-09-26 17:15:46,521 - lava_callback - ERROR - Error processing callback for node 68d48647a5c2c05277bcd748: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:56700 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39038 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57296 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57300 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:16:08,885 - lava_callback - INFO - Received callback for node 68d6a72c2f83206171a924ae INFO: 172.21.0.1:57310 - "POST /node/68d6a72c2f83206171a924ae HTTP/1.0" 202 Accepted 2025-09-26 17:16:08,891 - lava_callback - INFO - Starting async job submit for node 68d6a72c2f83206171a924ae 2025-09-26 17:16:08,912 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:16:08,926 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6a72c2f83206171a924ae HTTP/1.1" 200 None 2025-09-26 17:16:08,927 - lava_callback - DEBUG - Processing node 68d6a72c2f83206171a924ae: baseline-arm64 2025-09-26 17:16:09,170 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 17:16:09,303 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 17:16:09,304 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6a72c2f83206171a924ae/log.txt.gz 2025-09-26 17:16:09,310 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 17:16:09,444 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 17:16:09,445 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6a72c2f83206171a924ae/lava_callback.json.gz 2025-09-26 17:16:09,466 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:16:09,484 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6a72c2f83206171a924ae HTTP/1.1" 200 None 2025-09-26 17:16:09,485 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:16:09,501 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d648ed5d233c363a762351 HTTP/1.1" 200 None 2025-09-26 17:16:09,503 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:16:09,534 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6a72c2f83206171a924ae HTTP/1.1" 200 None 2025-09-26 17:16:09,535 - lava_callback - INFO - Completed processing callback for node 68d6a72c2f83206171a924ae INFO: 172.21.0.1:57730 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57732 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60642 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60650 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:16:30,340 - lava_callback - INFO - Received callback for node 68d4275da5c2c05277bc73c2 INFO: 172.21.0.1:60928 - "POST /node/68d4275da5c2c05277bc73c2 HTTP/1.0" 202 Accepted 2025-09-26 17:16:30,345 - lava_callback - INFO - Starting async job submit for node 68d4275da5c2c05277bc73c2 2025-09-26 17:16:30,346 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:16:30,373 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4275da5c2c05277bc73c2 HTTP/1.1" 200 None 2025-09-26 17:16:30,373 - lava_callback - DEBUG - Processing node 68d4275da5c2c05277bc73c2: baseline-arm 2025-09-26 17:16:30,373 - lava_callback - ERROR - Error processing callback for node 68d4275da5c2c05277bc73c2: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 17:16:31,045 - lava_callback - INFO - Received callback for node 68d42760a5c2c05277bc73c5 INFO: 172.21.0.1:60934 - "POST /node/68d42760a5c2c05277bc73c5 HTTP/1.0" 202 Accepted 2025-09-26 17:16:31,050 - lava_callback - INFO - Starting async job submit for node 68d42760a5c2c05277bc73c5 2025-09-26 17:16:31,051 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:16:31,063 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d42760a5c2c05277bc73c5 HTTP/1.1" 200 None 2025-09-26 17:16:31,064 - lava_callback - DEBUG - Processing node 68d42760a5c2c05277bc73c5: baseline-arm 2025-09-26 17:16:31,064 - lava_callback - ERROR - Error processing callback for node 68d42760a5c2c05277bc73c5: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 17:16:31,766 - lava_callback - INFO - Received callback for node 68d42761a5c2c05277bc73c8 INFO: 172.21.0.1:60942 - "POST /node/68d42761a5c2c05277bc73c8 HTTP/1.0" 202 Accepted 2025-09-26 17:16:31,772 - lava_callback - INFO - Starting async job submit for node 68d42761a5c2c05277bc73c8 2025-09-26 17:16:31,773 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:16:31,785 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d42761a5c2c05277bc73c8 HTTP/1.1" 200 None 2025-09-26 17:16:31,785 - lava_callback - DEBUG - Processing node 68d42761a5c2c05277bc73c8: baseline-arm 2025-09-26 17:16:31,785 - lava_callback - ERROR - Error processing callback for node 68d42761a5c2c05277bc73c8: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 17:16:32,500 - lava_callback - INFO - Received callback for node 68d42764a5c2c05277bc73cb INFO: 172.21.0.1:60948 - "POST /node/68d42764a5c2c05277bc73cb HTTP/1.0" 202 Accepted 2025-09-26 17:16:32,506 - lava_callback - INFO - Starting async job submit for node 68d42764a5c2c05277bc73cb 2025-09-26 17:16:32,506 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:16:32,519 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d42764a5c2c05277bc73cb HTTP/1.1" 200 None 2025-09-26 17:16:32,519 - lava_callback - DEBUG - Processing node 68d42764a5c2c05277bc73cb: baseline-arm 2025-09-26 17:16:32,519 - lava_callback - ERROR - Error processing callback for node 68d42764a5c2c05277bc73cb: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 17:16:33,216 - lava_callback - INFO - Received callback for node 68d42765a5c2c05277bc73ce INFO: 172.21.0.1:60956 - "POST /node/68d42765a5c2c05277bc73ce HTTP/1.0" 202 Accepted 2025-09-26 17:16:33,221 - lava_callback - INFO - Starting async job submit for node 68d42765a5c2c05277bc73ce 2025-09-26 17:16:33,222 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:16:33,234 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d42765a5c2c05277bc73ce HTTP/1.1" 200 None 2025-09-26 17:16:33,235 - lava_callback - DEBUG - Processing node 68d42765a5c2c05277bc73ce: baseline-arm 2025-09-26 17:16:33,235 - lava_callback - ERROR - Error processing callback for node 68d42765a5c2c05277bc73ce: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 17:16:33,942 - lava_callback - INFO - Received callback for node 68d42767a5c2c05277bc73d1 INFO: 172.21.0.1:60972 - "POST /node/68d42767a5c2c05277bc73d1 HTTP/1.0" 202 Accepted 2025-09-26 17:16:33,948 - lava_callback - INFO - Starting async job submit for node 68d42767a5c2c05277bc73d1 2025-09-26 17:16:33,948 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:16:33,968 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d42767a5c2c05277bc73d1 HTTP/1.1" 200 None 2025-09-26 17:16:33,968 - lava_callback - DEBUG - Processing node 68d42767a5c2c05277bc73d1: baseline-arm 2025-09-26 17:16:33,969 - lava_callback - ERROR - Error processing callback for node 68d42767a5c2c05277bc73d1: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 17:16:34,674 - lava_callback - INFO - Received callback for node 68d42768a5c2c05277bc73d4 INFO: 172.21.0.1:60984 - "POST /node/68d42768a5c2c05277bc73d4 HTTP/1.0" 202 Accepted 2025-09-26 17:16:34,680 - lava_callback - INFO - Starting async job submit for node 68d42768a5c2c05277bc73d4 2025-09-26 17:16:34,680 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:16:34,693 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d42768a5c2c05277bc73d4 HTTP/1.1" 200 None 2025-09-26 17:16:34,693 - lava_callback - DEBUG - Processing node 68d42768a5c2c05277bc73d4: baseline-arm 2025-09-26 17:16:34,693 - lava_callback - ERROR - Error processing callback for node 68d42768a5c2c05277bc73d4: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:60992 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50254 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:16:50,723 - lava_callback - INFO - Received callback for node 68d4276da5c2c05277bc73d7 INFO: 172.21.0.1:42598 - "POST /node/68d4276da5c2c05277bc73d7 HTTP/1.0" 202 Accepted 2025-09-26 17:16:50,728 - lava_callback - INFO - Starting async job submit for node 68d4276da5c2c05277bc73d7 2025-09-26 17:16:50,729 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:16:50,742 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4276da5c2c05277bc73d7 HTTP/1.1" 200 None 2025-09-26 17:16:50,743 - lava_callback - DEBUG - Processing node 68d4276da5c2c05277bc73d7: baseline-arm 2025-09-26 17:16:50,743 - lava_callback - ERROR - Error processing callback for node 68d4276da5c2c05277bc73d7: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 17:16:51,437 - lava_callback - INFO - Received callback for node 68d4276ea5c2c05277bc73da INFO: 172.21.0.1:42600 - "POST /node/68d4276ea5c2c05277bc73da HTTP/1.0" 202 Accepted 2025-09-26 17:16:51,442 - lava_callback - INFO - Starting async job submit for node 68d4276ea5c2c05277bc73da 2025-09-26 17:16:51,443 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:16:51,467 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4276ea5c2c05277bc73da HTTP/1.1" 200 None 2025-09-26 17:16:51,467 - lava_callback - DEBUG - Processing node 68d4276ea5c2c05277bc73da: baseline-arm 2025-09-26 17:16:51,467 - lava_callback - ERROR - Error processing callback for node 68d4276ea5c2c05277bc73da: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 17:16:52,131 - lava_callback - INFO - Received callback for node 68d42771a5c2c05277bc73dd INFO: 172.21.0.1:42608 - "POST /node/68d42771a5c2c05277bc73dd HTTP/1.0" 202 Accepted 2025-09-26 17:16:52,136 - lava_callback - INFO - Starting async job submit for node 68d42771a5c2c05277bc73dd 2025-09-26 17:16:52,137 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:16:52,156 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d42771a5c2c05277bc73dd HTTP/1.1" 200 None 2025-09-26 17:16:52,157 - lava_callback - DEBUG - Processing node 68d42771a5c2c05277bc73dd: baseline-arm 2025-09-26 17:16:52,157 - lava_callback - ERROR - Error processing callback for node 68d42771a5c2c05277bc73dd: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 17:16:52,846 - lava_callback - INFO - Received callback for node 68d42772a5c2c05277bc73e0 INFO: 172.21.0.1:42614 - "POST /node/68d42772a5c2c05277bc73e0 HTTP/1.0" 202 Accepted 2025-09-26 17:16:52,851 - lava_callback - INFO - Starting async job submit for node 68d42772a5c2c05277bc73e0 2025-09-26 17:16:52,852 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:16:52,864 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d42772a5c2c05277bc73e0 HTTP/1.1" 200 None 2025-09-26 17:16:52,864 - lava_callback - DEBUG - Processing node 68d42772a5c2c05277bc73e0: baseline-arm 2025-09-26 17:16:52,864 - lava_callback - ERROR - Error processing callback for node 68d42772a5c2c05277bc73e0: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 17:16:53,548 - lava_callback - INFO - Received callback for node 68d42774a5c2c05277bc73e3 INFO: 172.21.0.1:42628 - "POST /node/68d42774a5c2c05277bc73e3 HTTP/1.0" 202 Accepted 2025-09-26 17:16:53,554 - lava_callback - INFO - Starting async job submit for node 68d42774a5c2c05277bc73e3 2025-09-26 17:16:53,555 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:16:53,568 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d42774a5c2c05277bc73e3 HTTP/1.1" 200 None 2025-09-26 17:16:53,568 - lava_callback - DEBUG - Processing node 68d42774a5c2c05277bc73e3: baseline-arm 2025-09-26 17:16:53,568 - lava_callback - ERROR - Error processing callback for node 68d42774a5c2c05277bc73e3: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 17:16:54,255 - lava_callback - INFO - Received callback for node 68d42775a5c2c05277bc73e6 INFO: 172.21.0.1:42630 - "POST /node/68d42775a5c2c05277bc73e6 HTTP/1.0" 202 Accepted 2025-09-26 17:16:54,260 - lava_callback - INFO - Starting async job submit for node 68d42775a5c2c05277bc73e6 2025-09-26 17:16:54,261 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:16:54,273 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d42775a5c2c05277bc73e6 HTTP/1.1" 200 None 2025-09-26 17:16:54,274 - lava_callback - DEBUG - Processing node 68d42775a5c2c05277bc73e6: baseline-arm 2025-09-26 17:16:54,274 - lava_callback - ERROR - Error processing callback for node 68d42775a5c2c05277bc73e6: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 17:16:54,963 - lava_callback - INFO - Received callback for node 68d4277da5c2c05277bc7407 INFO: 172.21.0.1:42636 - "POST /node/68d4277da5c2c05277bc7407 HTTP/1.0" 202 Accepted 2025-09-26 17:16:54,968 - lava_callback - INFO - Starting async job submit for node 68d4277da5c2c05277bc7407 2025-09-26 17:16:54,970 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:16:54,982 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4277da5c2c05277bc7407 HTTP/1.1" 200 None 2025-09-26 17:16:54,982 - lava_callback - DEBUG - Processing node 68d4277da5c2c05277bc7407: baseline-arm 2025-09-26 17:16:54,982 - lava_callback - ERROR - Error processing callback for node 68d4277da5c2c05277bc7407: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 17:16:55,687 - lava_callback - INFO - Received callback for node 68d4277ea5c2c05277bc740a INFO: 172.21.0.1:42652 - "POST /node/68d4277ea5c2c05277bc740a HTTP/1.0" 202 Accepted 2025-09-26 17:16:55,692 - lava_callback - INFO - Starting async job submit for node 68d4277ea5c2c05277bc740a 2025-09-26 17:16:55,693 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:16:55,706 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4277ea5c2c05277bc740a HTTP/1.1" 200 None 2025-09-26 17:16:55,706 - lava_callback - DEBUG - Processing node 68d4277ea5c2c05277bc740a: baseline-arm 2025-09-26 17:16:55,706 - lava_callback - ERROR - Error processing callback for node 68d4277ea5c2c05277bc740a: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 17:16:56,431 - lava_callback - INFO - Received callback for node 68d42781a5c2c05277bc740d INFO: 172.21.0.1:42668 - "POST /node/68d42781a5c2c05277bc740d HTTP/1.0" 202 Accepted 2025-09-26 17:16:56,436 - lava_callback - INFO - Starting async job submit for node 68d42781a5c2c05277bc740d 2025-09-26 17:16:56,437 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:16:56,448 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d42781a5c2c05277bc740d HTTP/1.1" 200 None 2025-09-26 17:16:56,449 - lava_callback - DEBUG - Processing node 68d42781a5c2c05277bc740d: baseline-arm 2025-09-26 17:16:56,449 - lava_callback - ERROR - Error processing callback for node 68d42781a5c2c05277bc740d: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:42672 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41144 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41152 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:17:11,494 - lava_callback - INFO - Received callback for node 68d42782a5c2c05277bc7410 INFO: 172.21.0.1:53498 - "POST /node/68d42782a5c2c05277bc7410 HTTP/1.0" 202 Accepted 2025-09-26 17:17:11,501 - lava_callback - INFO - Starting async job submit for node 68d42782a5c2c05277bc7410 2025-09-26 17:17:11,502 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:17:11,518 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d42782a5c2c05277bc7410 HTTP/1.1" 200 None 2025-09-26 17:17:11,519 - lava_callback - DEBUG - Processing node 68d42782a5c2c05277bc7410: baseline-arm 2025-09-26 17:17:11,519 - lava_callback - ERROR - Error processing callback for node 68d42782a5c2c05277bc7410: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 17:17:12,013 - lava_callback - INFO - Received callback for node 68d6a7482f83206171a924e1 INFO: 172.21.0.1:53510 - "POST /node/68d6a7482f83206171a924e1 HTTP/1.0" 202 Accepted 2025-09-26 17:17:12,021 - lava_callback - INFO - Starting async job submit for node 68d6a7482f83206171a924e1 2025-09-26 17:17:12,047 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:17:12,069 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6a7482f83206171a924e1 HTTP/1.1" 200 None 2025-09-26 17:17:12,069 - lava_callback - DEBUG - Processing node 68d6a7482f83206171a924e1: baseline-arm64 2025-09-26 17:17:12,212 - lava_callback - INFO - Received callback for node 68d42785a5c2c05277bc7413 INFO: 172.21.0.1:53516 - "POST /node/68d42785a5c2c05277bc7413 HTTP/1.0" 202 Accepted 2025-09-26 17:17:12,223 - lava_callback - INFO - Starting async job submit for node 68d42785a5c2c05277bc7413 2025-09-26 17:17:12,229 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:17:12,295 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d42785a5c2c05277bc7413 HTTP/1.1" 200 None 2025-09-26 17:17:12,306 - lava_callback - DEBUG - Processing node 68d42785a5c2c05277bc7413: baseline-arm 2025-09-26 17:17:12,311 - lava_callback - ERROR - Error processing callback for node 68d42785a5c2c05277bc7413: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 17:17:12,389 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 17:17:12,454 - lava_callback - INFO - Received callback for node 68d6a8122f83206171a925d0 INFO: 172.21.0.1:53530 - "POST /node/68d6a8122f83206171a925d0 HTTP/1.0" 202 Accepted 2025-09-26 17:17:12,465 - lava_callback - INFO - Starting async job submit for node 68d6a8122f83206171a925d0 2025-09-26 17:17:12,493 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:17:12,509 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6a8122f83206171a925d0 HTTP/1.1" 200 None 2025-09-26 17:17:12,510 - lava_callback - DEBUG - Processing node 68d6a8122f83206171a925d0: baseline-arm64 2025-09-26 17:17:12,565 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 17:17:12,601 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6a7482f83206171a924e1/log.txt.gz 2025-09-26 17:17:12,704 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 17:17:12,950 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 17:17:12,967 - lava_callback - INFO - Received callback for node 68d42789a5c2c05277bc7416 INFO: 172.21.0.1:53532 - "POST /node/68d42789a5c2c05277bc7416 HTTP/1.0" 202 Accepted 2025-09-26 17:17:12,989 - lava_callback - INFO - Starting async job submit for node 68d42789a5c2c05277bc7416 2025-09-26 17:17:12,990 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:17:13,000 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6a7482f83206171a924e1/lava_callback.json.gz 2025-09-26 17:17:13,051 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:17:13,053 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 17:17:13,056 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d42789a5c2c05277bc7416 HTTP/1.1" 200 None 2025-09-26 17:17:13,056 - lava_callback - DEBUG - Processing node 68d42789a5c2c05277bc7416: baseline-arm 2025-09-26 17:17:13,056 - lava_callback - ERROR - Error processing callback for node 68d42789a5c2c05277bc7416: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 17:17:13,068 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6a7482f83206171a924e1 HTTP/1.1" 200 None 2025-09-26 17:17:13,069 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:17:13,082 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d648ed5d233c363a762351 HTTP/1.1" 200 None 2025-09-26 17:17:13,083 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:17:13,119 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6a7482f83206171a924e1 HTTP/1.1" 200 None 2025-09-26 17:17:13,120 - lava_callback - INFO - Completed processing callback for node 68d6a7482f83206171a924e1 2025-09-26 17:17:13,235 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 17:17:13,235 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6a8122f83206171a925d0/log.txt.gz 2025-09-26 17:17:13,244 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 17:17:13,396 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 17:17:13,397 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6a8122f83206171a925d0/lava_callback.json.gz 2025-09-26 17:17:13,422 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:17:13,434 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6a8122f83206171a925d0 HTTP/1.1" 200 None 2025-09-26 17:17:13,435 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:17:13,447 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d64d12a5c3ab168ecc7a3d HTTP/1.1" 200 None 2025-09-26 17:17:13,448 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:17:13,482 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6a8122f83206171a925d0 HTTP/1.1" 200 None 2025-09-26 17:17:13,482 - lava_callback - INFO - Completed processing callback for node 68d6a8122f83206171a925d0 2025-09-26 17:17:13,678 - lava_callback - INFO - Received callback for node 68d4278ba5c2c05277bc7419 INFO: 172.21.0.1:53538 - "POST /node/68d4278ba5c2c05277bc7419 HTTP/1.0" 202 Accepted 2025-09-26 17:17:13,686 - lava_callback - INFO - Starting async job submit for node 68d4278ba5c2c05277bc7419 2025-09-26 17:17:13,687 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:17:13,699 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4278ba5c2c05277bc7419 HTTP/1.1" 200 None 2025-09-26 17:17:13,700 - lava_callback - DEBUG - Processing node 68d4278ba5c2c05277bc7419: baseline-arm 2025-09-26 17:17:13,700 - lava_callback - ERROR - Error processing callback for node 68d4278ba5c2c05277bc7419: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 17:17:14,416 - lava_callback - INFO - Received callback for node 68d4278ca5c2c05277bc741c INFO: 172.21.0.1:53542 - "POST /node/68d4278ca5c2c05277bc741c HTTP/1.0" 202 Accepted 2025-09-26 17:17:14,422 - lava_callback - INFO - Starting async job submit for node 68d4278ca5c2c05277bc741c 2025-09-26 17:17:14,423 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:17:14,436 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4278ca5c2c05277bc741c HTTP/1.1" 200 None 2025-09-26 17:17:14,437 - lava_callback - DEBUG - Processing node 68d4278ca5c2c05277bc741c: baseline-arm 2025-09-26 17:17:14,437 - lava_callback - ERROR - Error processing callback for node 68d4278ca5c2c05277bc741c: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 17:17:14,983 - lava_callback - INFO - Received callback for node 68d6a71e2f83206171a92492 INFO: 172.21.0.1:53556 - "POST /node/68d6a71e2f83206171a92492 HTTP/1.0" 202 Accepted 2025-09-26 17:17:14,990 - lava_callback - INFO - Starting async job submit for node 68d6a71e2f83206171a92492 2025-09-26 17:17:15,020 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:17:15,034 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6a71e2f83206171a92492 HTTP/1.1" 200 None 2025-09-26 17:17:15,034 - lava_callback - DEBUG - Processing node 68d6a71e2f83206171a92492: baseline-arm64 2025-09-26 17:17:15,150 - lava_callback - INFO - Received callback for node 68d4278fa5c2c05277bc741f INFO: 172.21.0.1:53560 - "POST /node/68d4278fa5c2c05277bc741f HTTP/1.0" 202 Accepted 2025-09-26 17:17:15,160 - lava_callback - INFO - Starting async job submit for node 68d4278fa5c2c05277bc741f 2025-09-26 17:17:15,161 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.21.0.1:53574 - "GET /api/metrics HTTP/1.0" 200 OK 2025-09-26 17:17:15,232 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4278fa5c2c05277bc741f HTTP/1.1" 200 None 2025-09-26 17:17:15,243 - lava_callback - DEBUG - Processing node 68d4278fa5c2c05277bc741f: baseline-arm 2025-09-26 17:17:15,243 - lava_callback - ERROR - Error processing callback for node 68d4278fa5c2c05277bc741f: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 17:17:15,842 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 17:17:15,881 - lava_callback - INFO - Received callback for node 68d42790a5c2c05277bc7422 INFO: 172.21.0.1:53576 - "POST /node/68d42790a5c2c05277bc7422 HTTP/1.0" 202 Accepted 2025-09-26 17:17:15,890 - lava_callback - INFO - Starting async job submit for node 68d42790a5c2c05277bc7422 2025-09-26 17:17:15,890 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:17:15,905 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d42790a5c2c05277bc7422 HTTP/1.1" 200 None 2025-09-26 17:17:15,905 - lava_callback - DEBUG - Processing node 68d42790a5c2c05277bc7422: baseline-arm 2025-09-26 17:17:15,905 - lava_callback - ERROR - Error processing callback for node 68d42790a5c2c05277bc7422: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 17:17:15,961 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 17:17:15,961 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6a71e2f83206171a92492/log.txt.gz 2025-09-26 17:17:15,977 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 17:17:16,082 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 17:17:16,083 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6a71e2f83206171a92492/lava_callback.json.gz DEBUG: 68d6a71e2f83206171a92492 Transitting job node result: pass -> incomplete 2025-09-26 17:17:16,113 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:17:16,130 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6a71e2f83206171a92492 HTTP/1.1" 200 None 2025-09-26 17:17:16,131 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:17:16,142 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d648ed5d233c363a762351 HTTP/1.1" 200 None 2025-09-26 17:17:16,143 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:17:16,170 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6a71e2f83206171a92492 HTTP/1.1" 200 None 2025-09-26 17:17:16,171 - lava_callback - INFO - Completed processing callback for node 68d6a71e2f83206171a92492 INFO: 172.21.0.1:53582 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57676 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57688 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49170 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:17:38,377 - lava_callback - INFO - Received callback for node 68d6c6513c94e462c688184e INFO: 172.21.0.1:49180 - "POST /node/68d6c6513c94e462c688184e HTTP/1.0" 202 Accepted 2025-09-26 17:17:38,383 - lava_callback - INFO - Starting async job submit for node 68d6c6513c94e462c688184e 2025-09-26 17:17:38,413 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:17:38,427 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6c6513c94e462c688184e HTTP/1.1" 200 None 2025-09-26 17:17:38,427 - lava_callback - DEBUG - Processing node 68d6c6513c94e462c688184e: baseline-arm 2025-09-26 17:17:38,740 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 17:17:38,842 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 17:17:38,843 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-68d6c6513c94e462c688184e/log.txt.gz 2025-09-26 17:17:38,851 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 17:17:38,955 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 17:17:38,956 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-68d6c6513c94e462c688184e/lava_callback.json.gz DEBUG: Setup failed for 68d6c6513c94e462c688184e. Transitting job node result: incomplete -> fail 2025-09-26 17:17:39,018 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:17:39,030 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6c6513c94e462c688184e HTTP/1.1" 200 None 2025-09-26 17:17:39,031 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:17:39,042 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6bd793c94e462c68814e2 HTTP/1.1" 200 None 2025-09-26 17:17:39,043 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:17:39,069 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6c6513c94e462c688184e HTTP/1.1" 200 None 2025-09-26 17:17:39,069 - lava_callback - INFO - Completed processing callback for node 68d6c6513c94e462c688184e INFO: 172.21.0.1:42832 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:17:51,576 - lava_callback - INFO - Received callback for node 68d4864ca5c2c05277bcd74b INFO: 172.21.0.1:50216 - "POST /node/68d4864ca5c2c05277bcd74b HTTP/1.0" 202 Accepted 2025-09-26 17:17:51,582 - lava_callback - INFO - Starting async job submit for node 68d4864ca5c2c05277bcd74b 2025-09-26 17:17:51,602 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:17:51,615 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4864ca5c2c05277bcd74b HTTP/1.1" 200 None 2025-09-26 17:17:51,615 - lava_callback - DEBUG - Processing node 68d4864ca5c2c05277bcd74b: baseline-arm 2025-09-26 17:17:51,865 - lava_callback - ERROR - Error processing callback for node 68d4864ca5c2c05277bcd74b: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:50222 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:17:58,070 - lava_callback - INFO - Received callback for node 68d6a73b2f83206171a924c9 INFO: 172.21.0.1:50238 - "POST /node/68d6a73b2f83206171a924c9 HTTP/1.0" 202 Accepted 2025-09-26 17:17:58,077 - lava_callback - INFO - Starting async job submit for node 68d6a73b2f83206171a924c9 2025-09-26 17:17:58,106 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:17:58,123 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6a73b2f83206171a924c9 HTTP/1.1" 200 None 2025-09-26 17:17:58,123 - lava_callback - DEBUG - Processing node 68d6a73b2f83206171a924c9: baseline-arm64 2025-09-26 17:17:58,833 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 17:17:58,949 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 17:17:58,950 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6a73b2f83206171a924c9/log.txt.gz 2025-09-26 17:17:58,965 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 17:17:59,073 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 17:17:59,074 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6a73b2f83206171a924c9/lava_callback.json.gz DEBUG: 68d6a73b2f83206171a924c9 Transitting job node result: pass -> incomplete 2025-09-26 17:17:59,104 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:17:59,124 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6a73b2f83206171a924c9 HTTP/1.1" 200 None 2025-09-26 17:17:59,125 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:17:59,136 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d648ed5d233c363a762351 HTTP/1.1" 200 None 2025-09-26 17:17:59,137 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:17:59,168 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6a73b2f83206171a924c9 HTTP/1.1" 200 None 2025-09-26 17:17:59,169 - lava_callback - INFO - Completed processing callback for node 68d6a73b2f83206171a924c9 2025-09-26 17:18:02,006 - lava_callback - INFO - Received callback for node 68d6a73c2f83206171a924cc INFO: 172.21.0.1:41948 - "POST /node/68d6a73c2f83206171a924cc HTTP/1.0" 202 Accepted 2025-09-26 17:18:02,016 - lava_callback - INFO - Starting async job submit for node 68d6a73c2f83206171a924cc 2025-09-26 17:18:02,051 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:18:02,065 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6a73c2f83206171a924cc HTTP/1.1" 200 None 2025-09-26 17:18:02,065 - lava_callback - DEBUG - Processing node 68d6a73c2f83206171a924cc: baseline-arm64 2025-09-26 17:18:02,981 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 17:18:03,124 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 17:18:03,125 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6a73c2f83206171a924cc/log.txt.gz 2025-09-26 17:18:03,141 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 17:18:03,258 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 17:18:03,259 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6a73c2f83206171a924cc/lava_callback.json.gz DEBUG: 68d6a73c2f83206171a924cc Transitting job node result: pass -> incomplete 2025-09-26 17:18:03,301 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:18:03,316 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6a73c2f83206171a924cc HTTP/1.1" 200 None 2025-09-26 17:18:03,317 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:18:03,331 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d648ed5d233c363a762351 HTTP/1.1" 200 None 2025-09-26 17:18:03,332 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:18:03,374 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6a73c2f83206171a924cc HTTP/1.1" 200 None 2025-09-26 17:18:03,374 - lava_callback - INFO - Completed processing callback for node 68d6a73c2f83206171a924cc INFO: 172.21.0.1:41954 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41964 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:18:13,154 - lava_callback - INFO - Received callback for node 68d6a7492f83206171a924e4 INFO: 172.21.0.1:37746 - "POST /node/68d6a7492f83206171a924e4 HTTP/1.0" 202 Accepted 2025-09-26 17:18:13,165 - lava_callback - INFO - Starting async job submit for node 68d6a7492f83206171a924e4 2025-09-26 17:18:13,205 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:18:13,238 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6a7492f83206171a924e4 HTTP/1.1" 200 None 2025-09-26 17:18:13,239 - lava_callback - DEBUG - Processing node 68d6a7492f83206171a924e4: baseline-arm64 2025-09-26 17:18:13,520 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 17:18:13,651 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 17:18:13,651 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6a7492f83206171a924e4/log.txt.gz 2025-09-26 17:18:13,658 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 17:18:13,780 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 17:18:13,781 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6a7492f83206171a924e4/lava_callback.json.gz 2025-09-26 17:18:13,804 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:18:13,816 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6a7492f83206171a924e4 HTTP/1.1" 200 None 2025-09-26 17:18:13,818 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:18:13,833 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d648ed5d233c363a762351 HTTP/1.1" 200 None 2025-09-26 17:18:13,834 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:18:13,868 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6a7492f83206171a924e4 HTTP/1.1" 200 None 2025-09-26 17:18:13,869 - lava_callback - INFO - Completed processing callback for node 68d6a7492f83206171a924e4 INFO: 172.21.0.1:37754 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37770 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:18:19,518 - lava_callback - INFO - Received callback for node 68d6c0b43c94e462c688162d INFO: 172.21.0.1:35804 - "POST /node/68d6c0b43c94e462c688162d HTTP/1.0" 202 Accepted 2025-09-26 17:18:19,529 - lava_callback - INFO - Starting async job submit for node 68d6c0b43c94e462c688162d 2025-09-26 17:18:19,585 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:18:19,597 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6c0b43c94e462c688162d HTTP/1.1" 200 None 2025-09-26 17:18:19,598 - lava_callback - DEBUG - Processing node 68d6c0b43c94e462c688162d: baseline-arm64 2025-09-26 17:18:20,778 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 17:18:20,923 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 17:18:20,923 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6c0b43c94e462c688162d/log.txt.gz 2025-09-26 17:18:20,942 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 17:18:25,143 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 17:18:25,144 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6c0b43c94e462c688162d/lava_callback.json.gz DEBUG: Setup failed for 68d6c0b43c94e462c688162d. Transitting job node result: incomplete -> fail 2025-09-26 17:18:25,224 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:18:25,237 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6c0b43c94e462c688162d HTTP/1.1" 200 None 2025-09-26 17:18:25,238 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:18:25,249 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d651ddcca86fb17472aa24 HTTP/1.1" 200 None 2025-09-26 17:18:25,250 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:18:25,278 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6c0b43c94e462c688162d HTTP/1.1" 200 None 2025-09-26 17:18:25,279 - lava_callback - INFO - Completed processing callback for node 68d6c0b43c94e462c688162d INFO: 172.21.0.1:35806 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35808 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53004 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59986 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:18:52,866 - lava_callback - INFO - Received callback for node 68d6a82f2f83206171a92606 INFO: 172.21.0.1:51960 - "POST /node/68d6a82f2f83206171a92606 HTTP/1.0" 202 Accepted 2025-09-26 17:18:52,876 - lava_callback - INFO - Starting async job submit for node 68d6a82f2f83206171a92606 2025-09-26 17:18:52,904 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:18:52,932 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6a82f2f83206171a92606 HTTP/1.1" 200 None 2025-09-26 17:18:52,933 - lava_callback - DEBUG - Processing node 68d6a82f2f83206171a92606: baseline-arm64 2025-09-26 17:18:53,481 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 17:18:53,614 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 17:18:53,614 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6a82f2f83206171a92606/log.txt.gz 2025-09-26 17:18:53,623 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 17:18:53,838 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 17:18:53,839 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6a82f2f83206171a92606/lava_callback.json.gz 2025-09-26 17:18:53,876 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:18:53,895 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6a82f2f83206171a92606 HTTP/1.1" 200 None 2025-09-26 17:18:53,896 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:18:53,916 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d64d12a5c3ab168ecc7a3d HTTP/1.1" 200 None 2025-09-26 17:18:53,918 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:18:53,967 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6a82f2f83206171a92606 HTTP/1.1" 200 None 2025-09-26 17:18:53,969 - lava_callback - INFO - Completed processing callback for node 68d6a82f2f83206171a92606 2025-09-26 17:18:55,239 - lava_callback - INFO - Received callback for node 68d6a82f2f83206171a92603 INFO: 172.21.0.1:51972 - "POST /node/68d6a82f2f83206171a92603 HTTP/1.0" 202 Accepted 2025-09-26 17:18:55,246 - lava_callback - INFO - Starting async job submit for node 68d6a82f2f83206171a92603 2025-09-26 17:18:55,276 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:18:55,295 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6a82f2f83206171a92603 HTTP/1.1" 200 None 2025-09-26 17:18:55,295 - lava_callback - DEBUG - Processing node 68d6a82f2f83206171a92603: baseline-arm64 2025-09-26 17:18:55,826 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 17:18:55,934 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 17:18:55,935 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6a82f2f83206171a92603/log.txt.gz 2025-09-26 17:18:55,944 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 17:18:56,187 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 17:18:56,188 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6a82f2f83206171a92603/lava_callback.json.gz 2025-09-26 17:18:56,222 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:18:56,243 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6a82f2f83206171a92603 HTTP/1.1" 200 None 2025-09-26 17:18:56,244 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:18:56,258 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d64d12a5c3ab168ecc7a3d HTTP/1.1" 200 None 2025-09-26 17:18:56,259 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:18:56,289 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6a82f2f83206171a92603 HTTP/1.1" 200 None 2025-09-26 17:18:56,290 - lava_callback - INFO - Completed processing callback for node 68d6a82f2f83206171a92603 INFO: 172.21.0.1:51986 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55896 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55904 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40760 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40766 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:19:17,654 - lava_callback - INFO - Received callback for node 68d6ac7d2f83206171a9278f INFO: 172.21.0.1:40780 - "POST /node/68d6ac7d2f83206171a9278f HTTP/1.0" 202 Accepted 2025-09-26 17:19:17,664 - lava_callback - INFO - Starting async job submit for node 68d6ac7d2f83206171a9278f 2025-09-26 17:19:17,698 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:19:17,726 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6ac7d2f83206171a9278f HTTP/1.1" 200 None 2025-09-26 17:19:17,726 - lava_callback - DEBUG - Processing node 68d6ac7d2f83206171a9278f: baseline-arm64 2025-09-26 17:19:18,327 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 17:19:18,481 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 17:19:18,482 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6ac7d2f83206171a9278f/log.txt.gz 2025-09-26 17:19:18,492 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 17:19:18,685 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 17:19:18,686 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6ac7d2f83206171a9278f/lava_callback.json.gz 2025-09-26 17:19:18,725 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:19:18,747 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6ac7d2f83206171a9278f HTTP/1.1" 200 None 2025-09-26 17:19:18,752 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:19:18,776 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d637475d233c363a761c21 HTTP/1.1" 200 None 2025-09-26 17:19:18,778 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:19:18,834 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6ac7d2f83206171a9278f HTTP/1.1" 200 None 2025-09-26 17:19:18,835 - lava_callback - INFO - Completed processing callback for node 68d6ac7d2f83206171a9278f 2025-09-26 17:19:20,851 - lava_callback - INFO - Received callback for node 68d6ac7e2f83206171a92792 INFO: 172.21.0.1:55620 - "POST /node/68d6ac7e2f83206171a92792 HTTP/1.0" 202 Accepted 2025-09-26 17:19:20,861 - lava_callback - INFO - Starting async job submit for node 68d6ac7e2f83206171a92792 2025-09-26 17:19:20,895 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:19:20,915 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6ac7e2f83206171a92792 HTTP/1.1" 200 None 2025-09-26 17:19:20,916 - lava_callback - DEBUG - Processing node 68d6ac7e2f83206171a92792: baseline-arm64 2025-09-26 17:19:21,509 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 17:19:21,648 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 17:19:21,649 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6ac7e2f83206171a92792/log.txt.gz 2025-09-26 17:19:21,660 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 17:19:21,886 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 17:19:21,887 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6ac7e2f83206171a92792/lava_callback.json.gz 2025-09-26 17:19:21,922 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:19:21,938 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6ac7e2f83206171a92792 HTTP/1.1" 200 None 2025-09-26 17:19:21,939 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:19:21,960 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d637475d233c363a761c21 HTTP/1.1" 200 None 2025-09-26 17:19:21,961 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:19:22,017 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6ac7e2f83206171a92792 HTTP/1.1" 200 None 2025-09-26 17:19:22,018 - lava_callback - INFO - Completed processing callback for node 68d6ac7e2f83206171a92792 INFO: 172.21.0.1:55632 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55636 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55158 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40514 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:19:47,452 - lava_callback - INFO - Received callback for node 68d6a81f2f83206171a925e5 INFO: 172.21.0.1:40524 - "POST /node/68d6a81f2f83206171a925e5 HTTP/1.0" 202 Accepted 2025-09-26 17:19:47,460 - lava_callback - INFO - Starting async job submit for node 68d6a81f2f83206171a925e5 2025-09-26 17:19:47,492 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:19:47,511 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6a81f2f83206171a925e5 HTTP/1.1" 200 None 2025-09-26 17:19:47,512 - lava_callback - DEBUG - Processing node 68d6a81f2f83206171a925e5: baseline-arm64 2025-09-26 17:19:47,920 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 17:19:48,069 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 17:19:48,069 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6a81f2f83206171a925e5/log.txt.gz 2025-09-26 17:19:48,076 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 17:19:48,314 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 17:19:48,315 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6a81f2f83206171a925e5/lava_callback.json.gz 2025-09-26 17:19:48,350 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:19:48,364 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6a81f2f83206171a925e5 HTTP/1.1" 200 None 2025-09-26 17:19:48,366 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:19:48,380 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d64d12a5c3ab168ecc7a3d HTTP/1.1" 200 None 2025-09-26 17:19:48,382 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:19:48,432 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6a81f2f83206171a925e5 HTTP/1.1" 200 None 2025-09-26 17:19:48,433 - lava_callback - INFO - Completed processing callback for node 68d6a81f2f83206171a925e5 2025-09-26 17:19:50,504 - lava_callback - INFO - Received callback for node 68d42822a5c2c05277bc752f INFO: 172.21.0.1:49272 - "POST /node/68d42822a5c2c05277bc752f HTTP/1.0" 202 Accepted 2025-09-26 17:19:50,510 - lava_callback - INFO - Starting async job submit for node 68d42822a5c2c05277bc752f 2025-09-26 17:19:50,511 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:19:50,524 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d42822a5c2c05277bc752f HTTP/1.1" 200 None 2025-09-26 17:19:50,525 - lava_callback - DEBUG - Processing node 68d42822a5c2c05277bc752f: baseline-arm 2025-09-26 17:19:50,525 - lava_callback - ERROR - Error processing callback for node 68d42822a5c2c05277bc752f: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 17:19:51,260 - lava_callback - INFO - Received callback for node 68d42824a5c2c05277bc7532 INFO: 172.21.0.1:49286 - "POST /node/68d42824a5c2c05277bc7532 HTTP/1.0" 202 Accepted 2025-09-26 17:19:51,268 - lava_callback - INFO - Starting async job submit for node 68d42824a5c2c05277bc7532 2025-09-26 17:19:51,269 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:19:51,286 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d42824a5c2c05277bc7532 HTTP/1.1" 200 None 2025-09-26 17:19:51,287 - lava_callback - DEBUG - Processing node 68d42824a5c2c05277bc7532: baseline-arm 2025-09-26 17:19:51,287 - lava_callback - ERROR - Error processing callback for node 68d42824a5c2c05277bc7532: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 17:19:51,672 - lava_callback - INFO - Received callback for node 68d43315a5c2c05277bc8740 INFO: 172.21.0.1:49298 - "POST /node/68d43315a5c2c05277bc8740 HTTP/1.0" 202 Accepted 2025-09-26 17:19:51,682 - lava_callback - INFO - Starting async job submit for node 68d43315a5c2c05277bc8740 2025-09-26 17:19:51,720 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:19:51,740 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d43315a5c2c05277bc8740 HTTP/1.1" 200 None 2025-09-26 17:19:51,740 - lava_callback - DEBUG - Processing node 68d43315a5c2c05277bc8740: baseline-arm64 2025-09-26 17:19:51,964 - lava_callback - INFO - Received callback for node 68d42826a5c2c05277bc7535 INFO: 172.21.0.1:49310 - "POST /node/68d42826a5c2c05277bc7535 HTTP/1.0" 202 Accepted 2025-09-26 17:19:51,985 - lava_callback - INFO - Starting async job submit for node 68d42826a5c2c05277bc7535 2025-09-26 17:19:51,986 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:19:52,043 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d42826a5c2c05277bc7535 HTTP/1.1" 200 None 2025-09-26 17:19:52,048 - lava_callback - DEBUG - Processing node 68d42826a5c2c05277bc7535: baseline-arm 2025-09-26 17:19:52,048 - lava_callback - ERROR - Error processing callback for node 68d42826a5c2c05277bc7535: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 17:19:52,256 - lava_callback - ERROR - Error processing callback for node 68d43315a5c2c05277bc8740: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None 2025-09-26 17:19:52,704 - lava_callback - INFO - Received callback for node 68d42826a5c2c05277bc7538 INFO: 172.21.0.1:49314 - "POST /node/68d42826a5c2c05277bc7538 HTTP/1.0" 202 Accepted 2025-09-26 17:19:52,711 - lava_callback - INFO - Starting async job submit for node 68d42826a5c2c05277bc7538 2025-09-26 17:19:52,713 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:19:52,738 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d42826a5c2c05277bc7538 HTTP/1.1" 200 None 2025-09-26 17:19:52,739 - lava_callback - DEBUG - Processing node 68d42826a5c2c05277bc7538: baseline-arm 2025-09-26 17:19:52,739 - lava_callback - ERROR - Error processing callback for node 68d42826a5c2c05277bc7538: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 17:19:53,415 - lava_callback - INFO - Received callback for node 68d42828a5c2c05277bc753b INFO: 172.21.0.1:49322 - "POST /node/68d42828a5c2c05277bc753b HTTP/1.0" 202 Accepted 2025-09-26 17:19:53,422 - lava_callback - INFO - Starting async job submit for node 68d42828a5c2c05277bc753b 2025-09-26 17:19:53,423 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:19:53,438 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d42828a5c2c05277bc753b HTTP/1.1" 200 None 2025-09-26 17:19:53,438 - lava_callback - DEBUG - Processing node 68d42828a5c2c05277bc753b: baseline-arm 2025-09-26 17:19:53,438 - lava_callback - ERROR - Error processing callback for node 68d42828a5c2c05277bc753b: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 17:19:54,097 - lava_callback - INFO - Received callback for node 68d4282ba5c2c05277bc753f INFO: 172.21.0.1:49330 - "POST /node/68d4282ba5c2c05277bc753f HTTP/1.0" 202 Accepted 2025-09-26 17:19:54,105 - lava_callback - INFO - Starting async job submit for node 68d4282ba5c2c05277bc753f 2025-09-26 17:19:54,107 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:19:54,122 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4282ba5c2c05277bc753f HTTP/1.1" 200 None 2025-09-26 17:19:54,123 - lava_callback - DEBUG - Processing node 68d4282ba5c2c05277bc753f: baseline-arm 2025-09-26 17:19:54,123 - lava_callback - ERROR - Error processing callback for node 68d4282ba5c2c05277bc753f: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 17:19:54,803 - lava_callback - INFO - Received callback for node 68d4282da5c2c05277bc7542 INFO: 172.21.0.1:49332 - "POST /node/68d4282da5c2c05277bc7542 HTTP/1.0" 202 Accepted 2025-09-26 17:19:54,810 - lava_callback - INFO - Starting async job submit for node 68d4282da5c2c05277bc7542 2025-09-26 17:19:54,810 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:19:54,823 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4282da5c2c05277bc7542 HTTP/1.1" 200 None 2025-09-26 17:19:54,823 - lava_callback - DEBUG - Processing node 68d4282da5c2c05277bc7542: baseline-arm 2025-09-26 17:19:54,823 - lava_callback - ERROR - Error processing callback for node 68d4282da5c2c05277bc7542: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 17:19:55,529 - lava_callback - INFO - Received callback for node 68d4282da5c2c05277bc7545 INFO: 172.21.0.1:49344 - "POST /node/68d4282da5c2c05277bc7545 HTTP/1.0" 202 Accepted 2025-09-26 17:19:55,537 - lava_callback - INFO - Starting async job submit for node 68d4282da5c2c05277bc7545 2025-09-26 17:19:55,539 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:19:55,572 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4282da5c2c05277bc7545 HTTP/1.1" 200 None 2025-09-26 17:19:55,573 - lava_callback - DEBUG - Processing node 68d4282da5c2c05277bc7545: baseline-arm 2025-09-26 17:19:55,573 - lava_callback - ERROR - Error processing callback for node 68d4282da5c2c05277bc7545: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 17:19:56,245 - lava_callback - INFO - Received callback for node 68d4282fa5c2c05277bc7548 INFO: 172.21.0.1:49346 - "POST /node/68d4282fa5c2c05277bc7548 HTTP/1.0" 202 Accepted 2025-09-26 17:19:56,252 - lava_callback - INFO - Starting async job submit for node 68d4282fa5c2c05277bc7548 2025-09-26 17:19:56,253 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:19:56,272 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4282fa5c2c05277bc7548 HTTP/1.1" 200 None 2025-09-26 17:19:56,273 - lava_callback - DEBUG - Processing node 68d4282fa5c2c05277bc7548: baseline-arm 2025-09-26 17:19:56,273 - lava_callback - ERROR - Error processing callback for node 68d4282fa5c2c05277bc7548: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:49348 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:19:56,937 - lava_callback - INFO - Received callback for node 68d42831a5c2c05277bc754b INFO: 172.21.0.1:49360 - "POST /node/68d42831a5c2c05277bc754b HTTP/1.0" 202 Accepted 2025-09-26 17:19:56,945 - lava_callback - INFO - Starting async job submit for node 68d42831a5c2c05277bc754b 2025-09-26 17:19:56,950 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:19:56,978 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d42831a5c2c05277bc754b HTTP/1.1" 200 None 2025-09-26 17:19:56,978 - lava_callback - DEBUG - Processing node 68d42831a5c2c05277bc754b: baseline-arm 2025-09-26 17:19:56,978 - lava_callback - ERROR - Error processing callback for node 68d42831a5c2c05277bc754b: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:50656 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50660 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:20:10,111 - lava_callback - INFO - Received callback for node 68d4864ea5c2c05277bcd74e INFO: 172.21.0.1:38044 - "POST /node/68d4864ea5c2c05277bcd74e HTTP/1.0" 202 Accepted 2025-09-26 17:20:10,117 - lava_callback - INFO - Starting async job submit for node 68d4864ea5c2c05277bcd74e 2025-09-26 17:20:10,136 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:20:10,148 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4864ea5c2c05277bcd74e HTTP/1.1" 200 None 2025-09-26 17:20:10,148 - lava_callback - DEBUG - Processing node 68d4864ea5c2c05277bcd74e: baseline-arm 2025-09-26 17:20:10,374 - lava_callback - ERROR - Error processing callback for node 68d4864ea5c2c05277bcd74e: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None 2025-09-26 17:20:10,399 - lava_callback - INFO - Received callback for node 68d42835a5c2c05277bc7561 INFO: 172.21.0.1:38056 - "POST /node/68d42835a5c2c05277bc7561 HTTP/1.0" 202 Accepted 2025-09-26 17:20:10,405 - lava_callback - INFO - Starting async job submit for node 68d42835a5c2c05277bc7561 2025-09-26 17:20:10,406 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:20:10,428 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d42835a5c2c05277bc7561 HTTP/1.1" 200 None 2025-09-26 17:20:10,429 - lava_callback - DEBUG - Processing node 68d42835a5c2c05277bc7561: baseline-arm 2025-09-26 17:20:10,429 - lava_callback - ERROR - Error processing callback for node 68d42835a5c2c05277bc7561: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 17:20:11,092 - lava_callback - INFO - Received callback for node 68d42836a5c2c05277bc7564 INFO: 172.21.0.1:38060 - "POST /node/68d42836a5c2c05277bc7564 HTTP/1.0" 202 Accepted 2025-09-26 17:20:11,098 - lava_callback - INFO - Starting async job submit for node 68d42836a5c2c05277bc7564 2025-09-26 17:20:11,099 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:20:11,122 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d42836a5c2c05277bc7564 HTTP/1.1" 200 None 2025-09-26 17:20:11,122 - lava_callback - DEBUG - Processing node 68d42836a5c2c05277bc7564: baseline-arm 2025-09-26 17:20:11,122 - lava_callback - ERROR - Error processing callback for node 68d42836a5c2c05277bc7564: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 17:20:11,837 - lava_callback - INFO - Received callback for node 68d42838a5c2c05277bc7567 INFO: 172.21.0.1:38076 - "POST /node/68d42838a5c2c05277bc7567 HTTP/1.0" 202 Accepted 2025-09-26 17:20:11,842 - lava_callback - INFO - Starting async job submit for node 68d42838a5c2c05277bc7567 2025-09-26 17:20:11,843 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:20:11,856 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d42838a5c2c05277bc7567 HTTP/1.1" 200 None 2025-09-26 17:20:11,856 - lava_callback - DEBUG - Processing node 68d42838a5c2c05277bc7567: baseline-arm 2025-09-26 17:20:11,857 - lava_callback - ERROR - Error processing callback for node 68d42838a5c2c05277bc7567: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 17:20:12,556 - lava_callback - INFO - Received callback for node 68d42839a5c2c05277bc756a INFO: 172.21.0.1:38080 - "POST /node/68d42839a5c2c05277bc756a HTTP/1.0" 202 Accepted 2025-09-26 17:20:12,562 - lava_callback - INFO - Starting async job submit for node 68d42839a5c2c05277bc756a 2025-09-26 17:20:12,562 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:20:12,574 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d42839a5c2c05277bc756a HTTP/1.1" 200 None 2025-09-26 17:20:12,575 - lava_callback - DEBUG - Processing node 68d42839a5c2c05277bc756a: baseline-arm 2025-09-26 17:20:12,575 - lava_callback - ERROR - Error processing callback for node 68d42839a5c2c05277bc756a: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 17:20:13,313 - lava_callback - INFO - Received callback for node 68d4283ba5c2c05277bc756d INFO: 172.21.0.1:38088 - "POST /node/68d4283ba5c2c05277bc756d HTTP/1.0" 202 Accepted 2025-09-26 17:20:13,318 - lava_callback - INFO - Starting async job submit for node 68d4283ba5c2c05277bc756d 2025-09-26 17:20:13,319 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:20:13,333 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4283ba5c2c05277bc756d HTTP/1.1" 200 None 2025-09-26 17:20:13,334 - lava_callback - DEBUG - Processing node 68d4283ba5c2c05277bc756d: baseline-arm 2025-09-26 17:20:13,334 - lava_callback - ERROR - Error processing callback for node 68d4283ba5c2c05277bc756d: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 17:20:14,032 - lava_callback - INFO - Received callback for node 68d4283ea5c2c05277bc7570 INFO: 172.21.0.1:38090 - "POST /node/68d4283ea5c2c05277bc7570 HTTP/1.0" 202 Accepted 2025-09-26 17:20:14,038 - lava_callback - INFO - Starting async job submit for node 68d4283ea5c2c05277bc7570 2025-09-26 17:20:14,038 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:20:14,050 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4283ea5c2c05277bc7570 HTTP/1.1" 200 None 2025-09-26 17:20:14,051 - lava_callback - DEBUG - Processing node 68d4283ea5c2c05277bc7570: baseline-arm 2025-09-26 17:20:14,051 - lava_callback - ERROR - Error processing callback for node 68d4283ea5c2c05277bc7570: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 17:20:14,745 - lava_callback - INFO - Received callback for node 68d42840a5c2c05277bc7573 INFO: 172.21.0.1:38106 - "POST /node/68d42840a5c2c05277bc7573 HTTP/1.0" 202 Accepted 2025-09-26 17:20:14,750 - lava_callback - INFO - Starting async job submit for node 68d42840a5c2c05277bc7573 2025-09-26 17:20:14,751 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:20:14,768 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d42840a5c2c05277bc7573 HTTP/1.1" 200 None 2025-09-26 17:20:14,768 - lava_callback - DEBUG - Processing node 68d42840a5c2c05277bc7573: baseline-arm 2025-09-26 17:20:14,768 - lava_callback - ERROR - Error processing callback for node 68d42840a5c2c05277bc7573: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:38112 - "GET /api/metrics HTTP/1.0" 200 OK 2025-09-26 17:20:15,451 - lava_callback - INFO - Received callback for node 68d42840a5c2c05277bc7576 INFO: 172.21.0.1:38128 - "POST /node/68d42840a5c2c05277bc7576 HTTP/1.0" 202 Accepted 2025-09-26 17:20:15,456 - lava_callback - INFO - Starting async job submit for node 68d42840a5c2c05277bc7576 2025-09-26 17:20:15,457 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:20:15,468 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d42840a5c2c05277bc7576 HTTP/1.1" 200 None 2025-09-26 17:20:15,469 - lava_callback - DEBUG - Processing node 68d42840a5c2c05277bc7576: baseline-arm 2025-09-26 17:20:15,469 - lava_callback - ERROR - Error processing callback for node 68d42840a5c2c05277bc7576: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 17:20:16,138 - lava_callback - INFO - Received callback for node 68d42842a5c2c05277bc7579 INFO: 172.21.0.1:38140 - "POST /node/68d42842a5c2c05277bc7579 HTTP/1.0" 202 Accepted 2025-09-26 17:20:16,144 - lava_callback - INFO - Starting async job submit for node 68d42842a5c2c05277bc7579 2025-09-26 17:20:16,145 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:20:16,164 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d42842a5c2c05277bc7579 HTTP/1.1" 200 None 2025-09-26 17:20:16,165 - lava_callback - DEBUG - Processing node 68d42842a5c2c05277bc7579: baseline-arm 2025-09-26 17:20:16,165 - lava_callback - ERROR - Error processing callback for node 68d42842a5c2c05277bc7579: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:38150 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:20:16,837 - lava_callback - INFO - Received callback for node 68d42844a5c2c05277bc757c INFO: 172.21.0.1:38166 - "POST /node/68d42844a5c2c05277bc757c HTTP/1.0" 202 Accepted 2025-09-26 17:20:16,842 - lava_callback - INFO - Starting async job submit for node 68d42844a5c2c05277bc757c 2025-09-26 17:20:16,843 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:20:16,858 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d42844a5c2c05277bc757c HTTP/1.1" 200 None 2025-09-26 17:20:16,859 - lava_callback - DEBUG - Processing node 68d42844a5c2c05277bc757c: baseline-arm 2025-09-26 17:20:16,859 - lava_callback - ERROR - Error processing callback for node 68d42844a5c2c05277bc757c: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 17:20:17,535 - lava_callback - INFO - Received callback for node 68d42846a5c2c05277bc757f INFO: 172.21.0.1:38168 - "POST /node/68d42846a5c2c05277bc757f HTTP/1.0" 202 Accepted 2025-09-26 17:20:17,542 - lava_callback - INFO - Starting async job submit for node 68d42846a5c2c05277bc757f 2025-09-26 17:20:17,543 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:20:17,555 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d42846a5c2c05277bc757f HTTP/1.1" 200 None 2025-09-26 17:20:17,555 - lava_callback - DEBUG - Processing node 68d42846a5c2c05277bc757f: baseline-arm 2025-09-26 17:20:17,555 - lava_callback - ERROR - Error processing callback for node 68d42846a5c2c05277bc757f: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 17:20:18,246 - lava_callback - INFO - Received callback for node 68d42847a5c2c05277bc7582 INFO: 172.21.0.1:38182 - "POST /node/68d42847a5c2c05277bc7582 HTTP/1.0" 202 Accepted 2025-09-26 17:20:18,252 - lava_callback - INFO - Starting async job submit for node 68d42847a5c2c05277bc7582 2025-09-26 17:20:18,253 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:20:18,265 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d42847a5c2c05277bc7582 HTTP/1.1" 200 None 2025-09-26 17:20:18,265 - lava_callback - DEBUG - Processing node 68d42847a5c2c05277bc7582: baseline-arm 2025-09-26 17:20:18,265 - lava_callback - ERROR - Error processing callback for node 68d42847a5c2c05277bc7582: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:37256 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37268 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:20:30,245 - lava_callback - INFO - Received callback for node 68d42849a5c2c05277bc7585 INFO: 172.21.0.1:37230 - "POST /node/68d42849a5c2c05277bc7585 HTTP/1.0" 202 Accepted 2025-09-26 17:20:30,251 - lava_callback - INFO - Starting async job submit for node 68d42849a5c2c05277bc7585 2025-09-26 17:20:30,251 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:20:30,290 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d42849a5c2c05277bc7585 HTTP/1.1" 200 None 2025-09-26 17:20:30,290 - lava_callback - DEBUG - Processing node 68d42849a5c2c05277bc7585: baseline-arm 2025-09-26 17:20:30,290 - lava_callback - ERROR - Error processing callback for node 68d42849a5c2c05277bc7585: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 17:20:30,950 - lava_callback - INFO - Received callback for node 68d4284ba5c2c05277bc7588 INFO: 172.21.0.1:37236 - "POST /node/68d4284ba5c2c05277bc7588 HTTP/1.0" 202 Accepted 2025-09-26 17:20:30,956 - lava_callback - INFO - Starting async job submit for node 68d4284ba5c2c05277bc7588 2025-09-26 17:20:30,956 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:20:30,969 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4284ba5c2c05277bc7588 HTTP/1.1" 200 None 2025-09-26 17:20:30,970 - lava_callback - DEBUG - Processing node 68d4284ba5c2c05277bc7588: baseline-arm 2025-09-26 17:20:30,970 - lava_callback - ERROR - Error processing callback for node 68d4284ba5c2c05277bc7588: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:37248 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:20:38,699 - lava_callback - INFO - Received callback for node 68d6c0c73c94e462c6881636 INFO: 172.21.0.1:37252 - "POST /node/68d6c0c73c94e462c6881636 HTTP/1.0" 202 Accepted 2025-09-26 17:20:38,706 - lava_callback - INFO - Starting async job submit for node 68d6c0c73c94e462c6881636 2025-09-26 17:20:38,746 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:20:38,760 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6c0c73c94e462c6881636 HTTP/1.1" 200 None 2025-09-26 17:20:38,760 - lava_callback - DEBUG - Processing node 68d6c0c73c94e462c6881636: baseline-arm64 2025-09-26 17:20:39,874 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 17:20:40,008 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 17:20:40,009 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6c0c73c94e462c6881636/log.txt.gz 2025-09-26 17:20:40,028 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 17:20:40,190 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 17:20:40,191 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6c0c73c94e462c6881636/lava_callback.json.gz DEBUG: Setup failed for 68d6c0c73c94e462c6881636. Transitting job node result: incomplete -> fail 2025-09-26 17:20:40,265 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:20:40,279 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6c0c73c94e462c6881636 HTTP/1.1" 200 None 2025-09-26 17:20:40,280 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:20:40,291 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d651ddcca86fb17472aa24 HTTP/1.1" 200 None 2025-09-26 17:20:40,292 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:20:40,319 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6c0c73c94e462c6881636 HTTP/1.1" 200 None 2025-09-26 17:20:40,320 - lava_callback - INFO - Completed processing callback for node 68d6c0c73c94e462c6881636 INFO: 172.21.0.1:44914 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:20:49,690 - lava_callback - INFO - Received callback for node 68d6a81f2f83206171a925e8 INFO: 172.21.0.1:49254 - "POST /node/68d6a81f2f83206171a925e8 HTTP/1.0" 202 Accepted 2025-09-26 17:20:49,696 - lava_callback - INFO - Starting async job submit for node 68d6a81f2f83206171a925e8 2025-09-26 17:20:49,716 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:20:49,729 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6a81f2f83206171a925e8 HTTP/1.1" 200 None 2025-09-26 17:20:49,729 - lava_callback - DEBUG - Processing node 68d6a81f2f83206171a925e8: baseline-arm64 2025-09-26 17:20:49,980 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 17:20:50,115 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 17:20:50,115 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6a81f2f83206171a925e8/log.txt.gz 2025-09-26 17:20:50,121 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 17:20:50,228 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 17:20:50,228 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6a81f2f83206171a925e8/lava_callback.json.gz 2025-09-26 17:20:50,250 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:20:50,262 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6a81f2f83206171a925e8 HTTP/1.1" 200 None 2025-09-26 17:20:50,263 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:20:50,275 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d64d12a5c3ab168ecc7a3d HTTP/1.1" 200 None 2025-09-26 17:20:50,276 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:20:50,304 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6a81f2f83206171a925e8 HTTP/1.1" 200 None 2025-09-26 17:20:50,305 - lava_callback - INFO - Completed processing callback for node 68d6a81f2f83206171a925e8 2025-09-26 17:20:50,314 - lava_callback - INFO - Received callback for node 68d6c0c63c94e462c6881633 INFO: 172.21.0.1:49260 - "POST /node/68d6c0c63c94e462c6881633 HTTP/1.0" 202 Accepted 2025-09-26 17:20:50,330 - lava_callback - INFO - Starting async job submit for node 68d6c0c63c94e462c6881633 2025-09-26 17:20:50,383 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:20:50,401 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6c0c63c94e462c6881633 HTTP/1.1" 200 None 2025-09-26 17:20:50,402 - lava_callback - DEBUG - Processing node 68d6c0c63c94e462c6881633: baseline-arm64 2025-09-26 17:20:51,578 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 17:20:51,711 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 17:20:51,712 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6c0c63c94e462c6881633/log.txt.gz 2025-09-26 17:20:51,732 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 17:20:51,898 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 17:20:51,899 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6c0c63c94e462c6881633/lava_callback.json.gz 2025-09-26 17:20:51,931 - lava_callback - INFO - Received callback for node 68d6a83c2f83206171a9261b INFO: 172.21.0.1:49266 - "POST /node/68d6a83c2f83206171a9261b HTTP/1.0" 202 Accepted 2025-09-26 17:20:51,971 - lava_callback - INFO - Starting async job submit for node 68d6a83c2f83206171a9261b DEBUG: Setup failed for 68d6c0c63c94e462c6881633. Transitting job node result: incomplete -> fail 2025-09-26 17:20:52,023 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:20:52,024 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:20:52,037 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6c0c63c94e462c6881633 HTTP/1.1" 200 None 2025-09-26 17:20:52,038 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:20:52,038 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6a83c2f83206171a9261b HTTP/1.1" 200 None 2025-09-26 17:20:52,039 - lava_callback - DEBUG - Processing node 68d6a83c2f83206171a9261b: baseline-arm64 2025-09-26 17:20:52,087 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d651ddcca86fb17472aa24 HTTP/1.1" 200 None 2025-09-26 17:20:52,109 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:20:52,203 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6c0c63c94e462c6881633 HTTP/1.1" 200 None 2025-09-26 17:20:52,216 - lava_callback - INFO - Completed processing callback for node 68d6c0c63c94e462c6881633 2025-09-26 17:20:52,337 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 17:20:52,498 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 17:20:52,499 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6a83c2f83206171a9261b/log.txt.gz 2025-09-26 17:20:52,505 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 17:20:52,634 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 17:20:52,635 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6a83c2f83206171a9261b/lava_callback.json.gz 2025-09-26 17:20:52,656 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:20:52,668 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6a83c2f83206171a9261b HTTP/1.1" 200 None 2025-09-26 17:20:52,669 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:20:52,681 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d64d12a5c3ab168ecc7a3d HTTP/1.1" 200 None 2025-09-26 17:20:52,682 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:20:52,710 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6a83c2f83206171a9261b HTTP/1.1" 200 None 2025-09-26 17:20:52,710 - lava_callback - INFO - Completed processing callback for node 68d6a83c2f83206171a9261b 2025-09-26 17:20:56,436 - lava_callback - INFO - Received callback for node 68d43318a5c2c05277bc8743 INFO: 172.21.0.1:49274 - "POST /node/68d43318a5c2c05277bc8743 HTTP/1.0" 202 Accepted 2025-09-26 17:20:56,442 - lava_callback - INFO - Starting async job submit for node 68d43318a5c2c05277bc8743 2025-09-26 17:20:56,485 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:20:56,502 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d43318a5c2c05277bc8743 HTTP/1.1" 200 None 2025-09-26 17:20:56,502 - lava_callback - DEBUG - Processing node 68d43318a5c2c05277bc8743: baseline-arm64 INFO: 172.21.0.1:49290 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:20:56,915 - lava_callback - ERROR - Error processing callback for node 68d43318a5c2c05277bc8743: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:55550 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55566 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41894 - "GET /api/metrics HTTP/1.0" 200 OK 2025-09-26 17:21:16,366 - lava_callback - INFO - Received callback for node 68d4331da5c2c05277bc8746 INFO: 172.21.0.1:41904 - "POST /node/68d4331da5c2c05277bc8746 HTTP/1.0" 202 Accepted 2025-09-26 17:21:16,378 - lava_callback - INFO - Starting async job submit for node 68d4331da5c2c05277bc8746 2025-09-26 17:21:16,396 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:21:16,409 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4331da5c2c05277bc8746 HTTP/1.1" 200 None 2025-09-26 17:21:16,409 - lava_callback - DEBUG - Processing node 68d4331da5c2c05277bc8746: baseline-arm64 INFO: 172.21.0.1:41910 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:21:19,187 - lava_callback - ERROR - Error processing callback for node 68d4331da5c2c05277bc8746: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:35714 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35728 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40898 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54830 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34834 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58464 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58480 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:22:07,241 - lava_callback - INFO - Received callback for node 68d6a83c2f83206171a9261e INFO: 172.21.0.1:58494 - "POST /node/68d6a83c2f83206171a9261e HTTP/1.0" 202 Accepted 2025-09-26 17:22:07,247 - lava_callback - INFO - Starting async job submit for node 68d6a83c2f83206171a9261e 2025-09-26 17:22:07,268 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:22:07,280 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6a83c2f83206171a9261e HTTP/1.1" 200 None 2025-09-26 17:22:07,280 - lava_callback - DEBUG - Processing node 68d6a83c2f83206171a9261e: baseline-arm64 2025-09-26 17:22:07,522 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 17:22:07,691 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 17:22:07,691 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6a83c2f83206171a9261e/log.txt.gz 2025-09-26 17:22:07,697 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 17:22:07,838 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 17:22:07,839 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6a83c2f83206171a9261e/lava_callback.json.gz 2025-09-26 17:22:07,860 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:22:07,873 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6a83c2f83206171a9261e HTTP/1.1" 200 None 2025-09-26 17:22:07,874 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:22:07,885 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d64d12a5c3ab168ecc7a3d HTTP/1.1" 200 None 2025-09-26 17:22:07,886 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:22:07,913 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6a83c2f83206171a9261e HTTP/1.1" 200 None 2025-09-26 17:22:07,913 - lava_callback - INFO - Completed processing callback for node 68d6a83c2f83206171a9261e 2025-09-26 17:22:14,091 - lava_callback - INFO - Received callback for node 68d6adff2f83206171a9282c INFO: 172.21.0.1:50198 - "POST /node/68d6adff2f83206171a9282c HTTP/1.0" 202 Accepted 2025-09-26 17:22:14,097 - lava_callback - INFO - Starting async job submit for node 68d6adff2f83206171a9282c 2025-09-26 17:22:14,118 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:22:14,131 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6adff2f83206171a9282c HTTP/1.1" 200 None 2025-09-26 17:22:14,131 - lava_callback - DEBUG - Processing node 68d6adff2f83206171a9282c: baseline-arm64 2025-09-26 17:22:14,386 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 17:22:14,539 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 17:22:14,540 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6adff2f83206171a9282c/log.txt.gz 2025-09-26 17:22:14,546 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 17:22:14,676 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 17:22:14,676 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6adff2f83206171a9282c/lava_callback.json.gz 2025-09-26 17:22:14,697 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:22:14,709 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6adff2f83206171a9282c HTTP/1.1" 200 None 2025-09-26 17:22:14,710 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:22:14,726 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d651ddcca86fb17472aa24 HTTP/1.1" 200 None 2025-09-26 17:22:14,727 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:22:14,754 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6adff2f83206171a9282c HTTP/1.1" 200 None 2025-09-26 17:22:14,756 - lava_callback - INFO - Completed processing callback for node 68d6adff2f83206171a9282c INFO: 172.21.0.1:50206 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50220 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:22:23,967 - lava_callback - INFO - Received callback for node 68d4331da5c2c05277bc8749 INFO: 172.21.0.1:39738 - "POST /node/68d4331da5c2c05277bc8749 HTTP/1.0" 202 Accepted 2025-09-26 17:22:23,973 - lava_callback - INFO - Starting async job submit for node 68d4331da5c2c05277bc8749 2025-09-26 17:22:23,993 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:22:24,007 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4331da5c2c05277bc8749 HTTP/1.1" 200 None 2025-09-26 17:22:24,007 - lava_callback - DEBUG - Processing node 68d4331da5c2c05277bc8749: baseline-arm64 2025-09-26 17:22:24,296 - lava_callback - ERROR - Error processing callback for node 68d4331da5c2c05277bc8749: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:39742 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39754 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:22:30,179 - lava_callback - INFO - Received callback for node 68d4869ca5c2c05277bcdcf2 INFO: 172.21.0.1:57946 - "POST /node/68d4869ca5c2c05277bcdcf2 HTTP/1.0" 202 Accepted 2025-09-26 17:22:30,185 - lava_callback - INFO - Starting async job submit for node 68d4869ca5c2c05277bcdcf2 2025-09-26 17:22:30,206 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:22:30,219 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4869ca5c2c05277bcdcf2 HTTP/1.1" 200 None 2025-09-26 17:22:30,220 - lava_callback - DEBUG - Processing node 68d4869ca5c2c05277bcdcf2: baseline-arm 2025-09-26 17:22:30,468 - lava_callback - ERROR - Error processing callback for node 68d4869ca5c2c05277bcdcf2: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None 2025-09-26 17:22:33,909 - lava_callback - INFO - Received callback for node 68d43320a5c2c05277bc874c INFO: 172.21.0.1:57958 - "POST /node/68d43320a5c2c05277bc874c HTTP/1.0" 202 Accepted 2025-09-26 17:22:33,916 - lava_callback - INFO - Starting async job submit for node 68d43320a5c2c05277bc874c 2025-09-26 17:22:33,936 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:22:33,949 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d43320a5c2c05277bc874c HTTP/1.1" 200 None 2025-09-26 17:22:33,949 - lava_callback - DEBUG - Processing node 68d43320a5c2c05277bc874c: baseline-arm64 2025-09-26 17:22:34,246 - lava_callback - ERROR - Error processing callback for node 68d43320a5c2c05277bc874c: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:57970 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41274 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:22:48,305 - lava_callback - INFO - Received callback for node 68d428d3a5c2c05277bc7650 INFO: 172.21.0.1:41284 - "POST /node/68d428d3a5c2c05277bc7650 HTTP/1.0" 202 Accepted 2025-09-26 17:22:48,311 - lava_callback - INFO - Starting async job submit for node 68d428d3a5c2c05277bc7650 2025-09-26 17:22:48,312 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:22:48,325 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d428d3a5c2c05277bc7650 HTTP/1.1" 200 None 2025-09-26 17:22:48,326 - lava_callback - DEBUG - Processing node 68d428d3a5c2c05277bc7650: baseline-arm-broonie 2025-09-26 17:22:48,326 - lava_callback - ERROR - Error processing callback for node 68d428d3a5c2c05277bc7650: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 17:22:49,060 - lava_callback - INFO - Received callback for node 68d428d6a5c2c05277bc7653 INFO: 172.21.0.1:41294 - "POST /node/68d428d6a5c2c05277bc7653 HTTP/1.0" 202 Accepted 2025-09-26 17:22:49,065 - lava_callback - INFO - Starting async job submit for node 68d428d6a5c2c05277bc7653 2025-09-26 17:22:49,066 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:22:49,078 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d428d6a5c2c05277bc7653 HTTP/1.1" 200 None 2025-09-26 17:22:49,078 - lava_callback - DEBUG - Processing node 68d428d6a5c2c05277bc7653: baseline-arm-broonie 2025-09-26 17:22:49,078 - lava_callback - ERROR - Error processing callback for node 68d428d6a5c2c05277bc7653: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 17:22:49,850 - lava_callback - INFO - Received callback for node 68d428dca5c2c05277bc7659 INFO: 172.21.0.1:54488 - "POST /node/68d428dca5c2c05277bc7659 HTTP/1.0" 202 Accepted 2025-09-26 17:22:49,856 - lava_callback - INFO - Starting async job submit for node 68d428dca5c2c05277bc7659 2025-09-26 17:22:49,856 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:22:49,870 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d428dca5c2c05277bc7659 HTTP/1.1" 200 None 2025-09-26 17:22:49,870 - lava_callback - DEBUG - Processing node 68d428dca5c2c05277bc7659: baseline-arm-broonie 2025-09-26 17:22:49,870 - lava_callback - ERROR - Error processing callback for node 68d428dca5c2c05277bc7659: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 17:22:51,401 - lava_callback - INFO - Received callback for node 68d6c1263c94e462c6881662 INFO: 172.21.0.1:54502 - "POST /node/68d6c1263c94e462c6881662 HTTP/1.0" 202 Accepted 2025-09-26 17:22:51,410 - lava_callback - INFO - Starting async job submit for node 68d6c1263c94e462c6881662 2025-09-26 17:22:51,447 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:22:51,460 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6c1263c94e462c6881662 HTTP/1.1" 200 None 2025-09-26 17:22:51,461 - lava_callback - DEBUG - Processing node 68d6c1263c94e462c6881662: baseline-arm64 2025-09-26 17:22:52,421 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 17:22:52,557 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 17:22:52,558 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6c1263c94e462c6881662/log.txt.gz 2025-09-26 17:22:52,576 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 17:22:52,768 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 17:22:52,769 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6c1263c94e462c6881662/lava_callback.json.gz DEBUG: Setup failed for 68d6c1263c94e462c6881662. Transitting job node result: incomplete -> fail 2025-09-26 17:22:52,843 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:22:52,863 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6c1263c94e462c6881662 HTTP/1.1" 200 None 2025-09-26 17:22:52,865 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:22:52,876 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d651ddcca86fb17472aa24 HTTP/1.1" 200 None 2025-09-26 17:22:52,877 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:22:52,903 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6c1263c94e462c6881662 HTTP/1.1" 200 None 2025-09-26 17:22:52,904 - lava_callback - INFO - Completed processing callback for node 68d6c1263c94e462c6881662 2025-09-26 17:22:55,219 - lava_callback - INFO - Received callback for node 68d4275aa5c2c05277bc73bf INFO: 172.21.0.1:54504 - "POST /node/68d4275aa5c2c05277bc73bf HTTP/1.0" 202 Accepted 2025-09-26 17:22:55,228 - lava_callback - INFO - Starting async job submit for node 68d4275aa5c2c05277bc73bf 2025-09-26 17:22:55,266 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:22:55,281 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4275aa5c2c05277bc73bf HTTP/1.1" 200 None 2025-09-26 17:22:55,281 - lava_callback - DEBUG - Processing node 68d4275aa5c2c05277bc73bf: baseline-arm 2025-09-26 17:22:55,636 - lava_callback - ERROR - Error processing callback for node 68d4275aa5c2c05277bc73bf: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:54512 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34716 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34730 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:23:08,613 - lava_callback - INFO - Received callback for node 68d6ae002f83206171a9282f INFO: 172.21.0.1:34746 - "POST /node/68d6ae002f83206171a9282f HTTP/1.0" 202 Accepted 2025-09-26 17:23:08,619 - lava_callback - INFO - Starting async job submit for node 68d6ae002f83206171a9282f 2025-09-26 17:23:08,640 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:23:08,654 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6ae002f83206171a9282f HTTP/1.1" 200 None 2025-09-26 17:23:08,655 - lava_callback - DEBUG - Processing node 68d6ae002f83206171a9282f: baseline-arm64 2025-09-26 17:23:08,915 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 17:23:09,097 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 17:23:09,098 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6ae002f83206171a9282f/log.txt.gz 2025-09-26 17:23:09,106 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 17:23:09,229 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 17:23:09,230 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6ae002f83206171a9282f/lava_callback.json.gz 2025-09-26 17:23:09,257 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:23:09,269 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6ae002f83206171a9282f HTTP/1.1" 200 None 2025-09-26 17:23:09,270 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:23:09,289 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d651ddcca86fb17472aa24 HTTP/1.1" 200 None 2025-09-26 17:23:09,290 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:23:09,318 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6ae002f83206171a9282f HTTP/1.1" 200 None 2025-09-26 17:23:09,319 - lava_callback - INFO - Completed processing callback for node 68d6ae002f83206171a9282f INFO: 172.21.0.1:56982 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56986 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36770 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36778 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:23:28,862 - lava_callback - INFO - Received callback for node 68d6ae1c2f83206171a9286a INFO: 172.21.0.1:36780 - "POST /node/68d6ae1c2f83206171a9286a HTTP/1.0" 202 Accepted 2025-09-26 17:23:28,868 - lava_callback - INFO - Starting async job submit for node 68d6ae1c2f83206171a9286a 2025-09-26 17:23:28,888 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:23:28,903 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6ae1c2f83206171a9286a HTTP/1.1" 200 None 2025-09-26 17:23:28,903 - lava_callback - DEBUG - Processing node 68d6ae1c2f83206171a9286a: baseline-arm64 2025-09-26 17:23:29,160 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 17:23:29,303 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 17:23:29,303 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6ae1c2f83206171a9286a/log.txt.gz 2025-09-26 17:23:29,309 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 17:23:29,473 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 17:23:29,474 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6ae1c2f83206171a9286a/lava_callback.json.gz 2025-09-26 17:23:29,495 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:23:29,510 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6ae1c2f83206171a9286a HTTP/1.1" 200 None 2025-09-26 17:23:29,511 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:23:29,523 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d651ddcca86fb17472aa24 HTTP/1.1" 200 None 2025-09-26 17:23:29,524 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:23:29,552 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6ae1c2f83206171a9286a HTTP/1.1" 200 None 2025-09-26 17:23:29,554 - lava_callback - INFO - Completed processing callback for node 68d6ae1c2f83206171a9286a 2025-09-26 17:23:34,384 - lava_callback - INFO - Received callback for node 68d6c7bd3c94e462c68818ec INFO: 172.21.0.1:55292 - "POST /node/68d6c7bd3c94e462c68818ec HTTP/1.0" 202 Accepted 2025-09-26 17:23:34,391 - lava_callback - INFO - Starting async job submit for node 68d6c7bd3c94e462c68818ec 2025-09-26 17:23:34,421 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:23:34,434 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6c7bd3c94e462c68818ec HTTP/1.1" 200 None 2025-09-26 17:23:34,434 - lava_callback - DEBUG - Processing node 68d6c7bd3c94e462c68818ec: baseline-arm 2025-09-26 17:23:34,779 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 17:23:34,971 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 17:23:34,972 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-68d6c7bd3c94e462c68818ec/log.txt.gz 2025-09-26 17:23:34,981 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 17:23:35,125 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 17:23:35,126 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-68d6c7bd3c94e462c68818ec/lava_callback.json.gz DEBUG: Setup failed for 68d6c7bd3c94e462c68818ec. Transitting job node result: incomplete -> fail 2025-09-26 17:23:35,186 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:23:35,198 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6c7bd3c94e462c68818ec HTTP/1.1" 200 None 2025-09-26 17:23:35,199 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:23:35,211 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6b9da2f83206171a92b45 HTTP/1.1" 200 None 2025-09-26 17:23:35,212 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:23:35,237 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6c7bd3c94e462c68818ec HTTP/1.1" 200 None 2025-09-26 17:23:35,237 - lava_callback - INFO - Completed processing callback for node 68d6c7bd3c94e462c68818ec INFO: 172.21.0.1:55294 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:23:46,395 - lava_callback - INFO - Received callback for node 68d43322a5c2c05277bc874f INFO: 172.21.0.1:41162 - "POST /node/68d43322a5c2c05277bc874f HTTP/1.0" 202 Accepted 2025-09-26 17:23:46,401 - lava_callback - INFO - Starting async job submit for node 68d43322a5c2c05277bc874f 2025-09-26 17:23:46,422 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:23:46,437 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d43322a5c2c05277bc874f HTTP/1.1" 200 None 2025-09-26 17:23:46,437 - lava_callback - DEBUG - Processing node 68d43322a5c2c05277bc874f: baseline-arm64 INFO: 172.21.0.1:41172 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:23:46,741 - lava_callback - ERROR - Error processing callback for node 68d43322a5c2c05277bc874f: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None 2025-09-26 17:23:54,817 - lava_callback - INFO - Received callback for node 68d43327a5c2c05277bc8752 INFO: 172.21.0.1:41874 - "POST /node/68d43327a5c2c05277bc8752 HTTP/1.0" 202 Accepted 2025-09-26 17:23:54,825 - lava_callback - INFO - Starting async job submit for node 68d43327a5c2c05277bc8752 2025-09-26 17:23:54,843 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:23:54,876 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d43327a5c2c05277bc8752 HTTP/1.1" 200 None 2025-09-26 17:23:54,876 - lava_callback - DEBUG - Processing node 68d43327a5c2c05277bc8752: baseline-arm64 2025-09-26 17:23:56,087 - lava_callback - ERROR - Error processing callback for node 68d43327a5c2c05277bc8752: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:41890 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57794 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57796 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:24:09,253 - lava_callback - INFO - Received callback for node 68d6ae1d2f83206171a9286d INFO: 172.21.0.1:57802 - "POST /node/68d6ae1d2f83206171a9286d HTTP/1.0" 202 Accepted 2025-09-26 17:24:09,259 - lava_callback - INFO - Starting async job submit for node 68d6ae1d2f83206171a9286d 2025-09-26 17:24:09,281 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:24:09,316 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6ae1d2f83206171a9286d HTTP/1.1" 200 None 2025-09-26 17:24:09,317 - lava_callback - DEBUG - Processing node 68d6ae1d2f83206171a9286d: baseline-arm64 2025-09-26 17:24:09,567 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 17:24:09,711 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 17:24:09,711 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6ae1d2f83206171a9286d/log.txt.gz 2025-09-26 17:24:09,717 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 17:24:09,829 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 17:24:09,829 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6ae1d2f83206171a9286d/lava_callback.json.gz 2025-09-26 17:24:09,850 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:24:09,884 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6ae1d2f83206171a9286d HTTP/1.1" 200 None 2025-09-26 17:24:09,885 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:24:09,897 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d651ddcca86fb17472aa24 HTTP/1.1" 200 None 2025-09-26 17:24:09,898 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:24:09,935 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6ae1d2f83206171a9286d HTTP/1.1" 200 None 2025-09-26 17:24:09,936 - lava_callback - INFO - Completed processing callback for node 68d6ae1d2f83206171a9286d INFO: 172.21.0.1:55220 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55236 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48998 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49006 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32962 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:24:46,049 - lava_callback - INFO - Received callback for node 68d4869da5c2c05277bcdcf5 INFO: 172.21.0.1:55322 - "POST /node/68d4869da5c2c05277bcdcf5 HTTP/1.0" 202 Accepted 2025-09-26 17:24:46,055 - lava_callback - INFO - Starting async job submit for node 68d4869da5c2c05277bcdcf5 2025-09-26 17:24:46,076 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:24:46,116 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4869da5c2c05277bcdcf5 HTTP/1.1" 200 None 2025-09-26 17:24:46,116 - lava_callback - DEBUG - Processing node 68d4869da5c2c05277bcdcf5: baseline-arm 2025-09-26 17:24:46,337 - lava_callback - ERROR - Error processing callback for node 68d4869da5c2c05277bcdcf5: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:55336 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42616 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:25:01,158 - lava_callback - INFO - Received callback for node 68d43328a5c2c05277bc8755 INFO: 172.21.0.1:59944 - "POST /node/68d43328a5c2c05277bc8755 HTTP/1.0" 202 Accepted 2025-09-26 17:25:01,164 - lava_callback - INFO - Starting async job submit for node 68d43328a5c2c05277bc8755 2025-09-26 17:25:01,185 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:25:01,197 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d43328a5c2c05277bc8755 HTTP/1.1" 200 None 2025-09-26 17:25:01,197 - lava_callback - DEBUG - Processing node 68d43328a5c2c05277bc8755: baseline-arm64 2025-09-26 17:25:01,539 - lava_callback - ERROR - Error processing callback for node 68d43328a5c2c05277bc8755: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:59956 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59968 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:25:11,409 - lava_callback - INFO - Received callback for node 68d4332ba5c2c05277bc8758 INFO: 172.21.0.1:39172 - "POST /node/68d4332ba5c2c05277bc8758 HTTP/1.0" 202 Accepted 2025-09-26 17:25:11,416 - lava_callback - INFO - Starting async job submit for node 68d4332ba5c2c05277bc8758 2025-09-26 17:25:11,437 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:25:11,463 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4332ba5c2c05277bc8758 HTTP/1.1" 200 None 2025-09-26 17:25:11,463 - lava_callback - DEBUG - Processing node 68d4332ba5c2c05277bc8758: baseline-arm64 2025-09-26 17:25:11,764 - lava_callback - ERROR - Error processing callback for node 68d4332ba5c2c05277bc8758: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:39174 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39180 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58238 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58246 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:25:31,063 - lava_callback - INFO - Received callback for node 68d42977a5c2c05277bc7770 INFO: 172.21.0.1:36812 - "POST /node/68d42977a5c2c05277bc7770 HTTP/1.0" 202 Accepted 2025-09-26 17:25:31,069 - lava_callback - INFO - Starting async job submit for node 68d42977a5c2c05277bc7770 2025-09-26 17:25:31,070 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:25:31,338 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d42977a5c2c05277bc7770 HTTP/1.1" 200 None 2025-09-26 17:25:31,338 - lava_callback - DEBUG - Processing node 68d42977a5c2c05277bc7770: baseline-arm 2025-09-26 17:25:31,338 - lava_callback - ERROR - Error processing callback for node 68d42977a5c2c05277bc7770: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 17:25:31,765 - lava_callback - INFO - Received callback for node 68d42978a5c2c05277bc7775 INFO: 172.21.0.1:36816 - "POST /node/68d42978a5c2c05277bc7775 HTTP/1.0" 202 Accepted 2025-09-26 17:25:31,771 - lava_callback - INFO - Starting async job submit for node 68d42978a5c2c05277bc7775 2025-09-26 17:25:31,772 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:25:31,806 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d42978a5c2c05277bc7775 HTTP/1.1" 200 None 2025-09-26 17:25:31,807 - lava_callback - DEBUG - Processing node 68d42978a5c2c05277bc7775: baseline-arm 2025-09-26 17:25:31,807 - lava_callback - ERROR - Error processing callback for node 68d42978a5c2c05277bc7775: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 17:25:32,462 - lava_callback - INFO - Received callback for node 68d4297ba5c2c05277bc7778 INFO: 172.21.0.1:36820 - "POST /node/68d4297ba5c2c05277bc7778 HTTP/1.0" 202 Accepted 2025-09-26 17:25:32,468 - lava_callback - INFO - Starting async job submit for node 68d4297ba5c2c05277bc7778 2025-09-26 17:25:32,469 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:25:32,484 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4297ba5c2c05277bc7778 HTTP/1.1" 200 None 2025-09-26 17:25:32,485 - lava_callback - DEBUG - Processing node 68d4297ba5c2c05277bc7778: baseline-arm 2025-09-26 17:25:32,485 - lava_callback - ERROR - Error processing callback for node 68d4297ba5c2c05277bc7778: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 17:25:33,177 - lava_callback - INFO - Received callback for node 68d4297da5c2c05277bc777b INFO: 172.21.0.1:36832 - "POST /node/68d4297da5c2c05277bc777b HTTP/1.0" 202 Accepted 2025-09-26 17:25:33,182 - lava_callback - INFO - Starting async job submit for node 68d4297da5c2c05277bc777b 2025-09-26 17:25:33,183 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:25:33,196 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4297da5c2c05277bc777b HTTP/1.1" 200 None 2025-09-26 17:25:33,196 - lava_callback - DEBUG - Processing node 68d4297da5c2c05277bc777b: baseline-arm 2025-09-26 17:25:33,196 - lava_callback - ERROR - Error processing callback for node 68d4297da5c2c05277bc777b: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 17:25:33,895 - lava_callback - INFO - Received callback for node 68d4297da5c2c05277bc777e INFO: 172.21.0.1:36838 - "POST /node/68d4297da5c2c05277bc777e HTTP/1.0" 202 Accepted 2025-09-26 17:25:33,900 - lava_callback - INFO - Starting async job submit for node 68d4297da5c2c05277bc777e 2025-09-26 17:25:33,901 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:25:33,918 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4297da5c2c05277bc777e HTTP/1.1" 200 None 2025-09-26 17:25:33,918 - lava_callback - DEBUG - Processing node 68d4297da5c2c05277bc777e: baseline-arm 2025-09-26 17:25:33,918 - lava_callback - ERROR - Error processing callback for node 68d4297da5c2c05277bc777e: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 17:25:34,642 - lava_callback - INFO - Received callback for node 68d4297fa5c2c05277bc7781 INFO: 172.21.0.1:36840 - "POST /node/68d4297fa5c2c05277bc7781 HTTP/1.0" 202 Accepted 2025-09-26 17:25:34,647 - lava_callback - INFO - Starting async job submit for node 68d4297fa5c2c05277bc7781 2025-09-26 17:25:34,648 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:25:34,663 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4297fa5c2c05277bc7781 HTTP/1.1" 200 None 2025-09-26 17:25:34,663 - lava_callback - DEBUG - Processing node 68d4297fa5c2c05277bc7781: baseline-arm 2025-09-26 17:25:34,663 - lava_callback - ERROR - Error processing callback for node 68d4297fa5c2c05277bc7781: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 17:25:35,334 - lava_callback - INFO - Received callback for node 68d42980a5c2c05277bc7784 INFO: 172.21.0.1:36844 - "POST /node/68d42980a5c2c05277bc7784 HTTP/1.0" 202 Accepted 2025-09-26 17:25:35,343 - lava_callback - INFO - Starting async job submit for node 68d42980a5c2c05277bc7784 2025-09-26 17:25:35,344 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:25:35,358 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d42980a5c2c05277bc7784 HTTP/1.1" 200 None 2025-09-26 17:25:35,358 - lava_callback - DEBUG - Processing node 68d42980a5c2c05277bc7784: baseline-arm 2025-09-26 17:25:35,358 - lava_callback - ERROR - Error processing callback for node 68d42980a5c2c05277bc7784: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 17:25:36,060 - lava_callback - INFO - Received callback for node 68d42984a5c2c05277bc778a INFO: 172.21.0.1:36848 - "POST /node/68d42984a5c2c05277bc778a HTTP/1.0" 202 Accepted 2025-09-26 17:25:36,066 - lava_callback - INFO - Starting async job submit for node 68d42984a5c2c05277bc778a 2025-09-26 17:25:36,067 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:25:36,079 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d42984a5c2c05277bc778a HTTP/1.1" 200 None 2025-09-26 17:25:36,079 - lava_callback - DEBUG - Processing node 68d42984a5c2c05277bc778a: baseline-arm 2025-09-26 17:25:36,079 - lava_callback - ERROR - Error processing callback for node 68d42984a5c2c05277bc778a: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:36862 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:25:36,779 - lava_callback - INFO - Received callback for node 68d42986a5c2c05277bc778d INFO: 172.21.0.1:36864 - "POST /node/68d42986a5c2c05277bc778d HTTP/1.0" 202 Accepted 2025-09-26 17:25:36,784 - lava_callback - INFO - Starting async job submit for node 68d42986a5c2c05277bc778d 2025-09-26 17:25:36,785 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:25:36,797 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d42986a5c2c05277bc778d HTTP/1.1" 200 None 2025-09-26 17:25:36,797 - lava_callback - DEBUG - Processing node 68d42986a5c2c05277bc778d: baseline-arm 2025-09-26 17:25:36,797 - lava_callback - ERROR - Error processing callback for node 68d42986a5c2c05277bc778d: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 17:25:37,467 - lava_callback - INFO - Received callback for node 68d4298aa5c2c05277bc7790 INFO: 172.21.0.1:36870 - "POST /node/68d4298aa5c2c05277bc7790 HTTP/1.0" 202 Accepted 2025-09-26 17:25:37,472 - lava_callback - INFO - Starting async job submit for node 68d4298aa5c2c05277bc7790 2025-09-26 17:25:37,473 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:25:37,486 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4298aa5c2c05277bc7790 HTTP/1.1" 200 None 2025-09-26 17:25:37,487 - lava_callback - DEBUG - Processing node 68d4298aa5c2c05277bc7790: baseline-arm 2025-09-26 17:25:37,487 - lava_callback - ERROR - Error processing callback for node 68d4298aa5c2c05277bc7790: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:37686 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:25:49,330 - lava_callback - INFO - Received callback for node 68d6c7853c94e462c6881883 INFO: 172.21.0.1:48578 - "POST /node/68d6c7853c94e462c6881883 HTTP/1.0" 202 Accepted 2025-09-26 17:25:49,336 - lava_callback - INFO - Starting async job submit for node 68d6c7853c94e462c6881883 2025-09-26 17:25:49,357 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:25:49,369 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6c7853c94e462c6881883 HTTP/1.1" 200 None 2025-09-26 17:25:49,369 - lava_callback - DEBUG - Processing node 68d6c7853c94e462c6881883: baseline-arm64 2025-09-26 17:25:49,628 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 17:25:49,766 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 17:25:49,767 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6c7853c94e462c6881883/log.txt.gz 2025-09-26 17:25:49,773 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 17:25:49,921 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 17:25:49,922 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6c7853c94e462c6881883/lava_callback.json.gz 2025-09-26 17:25:49,960 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:25:49,972 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6c7853c94e462c6881883 HTTP/1.1" 200 None 2025-09-26 17:25:49,973 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:25:49,985 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6b9dd2f83206171a92b48 HTTP/1.1" 200 None 2025-09-26 17:25:49,986 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:25:50,013 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6c7853c94e462c6881883 HTTP/1.1" 200 None 2025-09-26 17:25:50,013 - lava_callback - INFO - Completed processing callback for node 68d6c7853c94e462c6881883 2025-09-26 17:25:50,580 - lava_callback - INFO - Received callback for node 68d4298aa5c2c05277bc7793 INFO: 172.21.0.1:48590 - "POST /node/68d4298aa5c2c05277bc7793 HTTP/1.0" 202 Accepted 2025-09-26 17:25:50,585 - lava_callback - INFO - Starting async job submit for node 68d4298aa5c2c05277bc7793 2025-09-26 17:25:50,586 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:25:50,599 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4298aa5c2c05277bc7793 HTTP/1.1" 200 None 2025-09-26 17:25:50,599 - lava_callback - DEBUG - Processing node 68d4298aa5c2c05277bc7793: baseline-arm 2025-09-26 17:25:50,599 - lava_callback - ERROR - Error processing callback for node 68d4298aa5c2c05277bc7793: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 17:25:51,288 - lava_callback - INFO - Received callback for node 68d4298fa5c2c05277bc7799 INFO: 172.21.0.1:48596 - "POST /node/68d4298fa5c2c05277bc7799 HTTP/1.0" 202 Accepted 2025-09-26 17:25:51,293 - lava_callback - INFO - Starting async job submit for node 68d4298fa5c2c05277bc7799 2025-09-26 17:25:51,294 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:25:51,306 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4298fa5c2c05277bc7799 HTTP/1.1" 200 None 2025-09-26 17:25:51,306 - lava_callback - DEBUG - Processing node 68d4298fa5c2c05277bc7799: baseline-arm 2025-09-26 17:25:51,306 - lava_callback - ERROR - Error processing callback for node 68d4298fa5c2c05277bc7799: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 17:25:52,000 - lava_callback - INFO - Received callback for node 68d4298fa5c2c05277bc779c INFO: 172.21.0.1:48602 - "POST /node/68d4298fa5c2c05277bc779c HTTP/1.0" 202 Accepted 2025-09-26 17:25:52,005 - lava_callback - INFO - Starting async job submit for node 68d4298fa5c2c05277bc779c 2025-09-26 17:25:52,006 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:25:52,024 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4298fa5c2c05277bc779c HTTP/1.1" 200 None 2025-09-26 17:25:52,025 - lava_callback - DEBUG - Processing node 68d4298fa5c2c05277bc779c: baseline-arm 2025-09-26 17:25:52,025 - lava_callback - ERROR - Error processing callback for node 68d4298fa5c2c05277bc779c: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 17:25:52,702 - lava_callback - INFO - Received callback for node 68d42991a5c2c05277bc779f INFO: 172.21.0.1:48618 - "POST /node/68d42991a5c2c05277bc779f HTTP/1.0" 202 Accepted 2025-09-26 17:25:52,708 - lava_callback - INFO - Starting async job submit for node 68d42991a5c2c05277bc779f 2025-09-26 17:25:52,709 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:25:52,721 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d42991a5c2c05277bc779f HTTP/1.1" 200 None 2025-09-26 17:25:52,721 - lava_callback - DEBUG - Processing node 68d42991a5c2c05277bc779f: baseline-arm 2025-09-26 17:25:52,721 - lava_callback - ERROR - Error processing callback for node 68d42991a5c2c05277bc779f: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 17:25:53,411 - lava_callback - INFO - Received callback for node 68d42993a5c2c05277bc77a2 INFO: 172.21.0.1:48620 - "POST /node/68d42993a5c2c05277bc77a2 HTTP/1.0" 202 Accepted 2025-09-26 17:25:53,416 - lava_callback - INFO - Starting async job submit for node 68d42993a5c2c05277bc77a2 2025-09-26 17:25:53,417 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:25:53,429 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d42993a5c2c05277bc77a2 HTTP/1.1" 200 None 2025-09-26 17:25:53,429 - lava_callback - DEBUG - Processing node 68d42993a5c2c05277bc77a2: baseline-arm 2025-09-26 17:25:53,429 - lava_callback - ERROR - Error processing callback for node 68d42993a5c2c05277bc77a2: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 17:25:54,118 - lava_callback - INFO - Received callback for node 68d42999a5c2c05277bc77a8 INFO: 172.21.0.1:48628 - "POST /node/68d42999a5c2c05277bc77a8 HTTP/1.0" 202 Accepted 2025-09-26 17:25:54,123 - lava_callback - INFO - Starting async job submit for node 68d42999a5c2c05277bc77a8 2025-09-26 17:25:54,124 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:25:54,141 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d42999a5c2c05277bc77a8 HTTP/1.1" 200 None 2025-09-26 17:25:54,141 - lava_callback - DEBUG - Processing node 68d42999a5c2c05277bc77a8: baseline-arm 2025-09-26 17:25:54,141 - lava_callback - ERROR - Error processing callback for node 68d42999a5c2c05277bc77a8: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:48630 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35300 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35316 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:26:10,943 - lava_callback - INFO - Received callback for node 68d4299ba5c2c05277bc77ab INFO: 172.21.0.1:58724 - "POST /node/68d4299ba5c2c05277bc77ab HTTP/1.0" 202 Accepted 2025-09-26 17:26:10,948 - lava_callback - INFO - Starting async job submit for node 68d4299ba5c2c05277bc77ab 2025-09-26 17:26:10,949 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:26:10,967 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4299ba5c2c05277bc77ab HTTP/1.1" 200 None 2025-09-26 17:26:10,968 - lava_callback - DEBUG - Processing node 68d4299ba5c2c05277bc77ab: baseline-arm 2025-09-26 17:26:10,968 - lava_callback - ERROR - Error processing callback for node 68d4299ba5c2c05277bc77ab: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 17:26:11,677 - lava_callback - INFO - Received callback for node 68d4299fa5c2c05277bc77ae INFO: 172.21.0.1:58730 - "POST /node/68d4299fa5c2c05277bc77ae HTTP/1.0" 202 Accepted 2025-09-26 17:26:11,682 - lava_callback - INFO - Starting async job submit for node 68d4299fa5c2c05277bc77ae 2025-09-26 17:26:11,683 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:26:11,703 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4299fa5c2c05277bc77ae HTTP/1.1" 200 None 2025-09-26 17:26:11,703 - lava_callback - DEBUG - Processing node 68d4299fa5c2c05277bc77ae: baseline-arm 2025-09-26 17:26:11,703 - lava_callback - ERROR - Error processing callback for node 68d4299fa5c2c05277bc77ae: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 17:26:12,384 - lava_callback - INFO - Received callback for node 68d429a0a5c2c05277bc77b1 INFO: 172.21.0.1:58734 - "POST /node/68d429a0a5c2c05277bc77b1 HTTP/1.0" 202 Accepted 2025-09-26 17:26:12,389 - lava_callback - INFO - Starting async job submit for node 68d429a0a5c2c05277bc77b1 2025-09-26 17:26:12,390 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:26:12,402 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d429a0a5c2c05277bc77b1 HTTP/1.1" 200 None 2025-09-26 17:26:12,402 - lava_callback - DEBUG - Processing node 68d429a0a5c2c05277bc77b1: baseline-arm 2025-09-26 17:26:12,402 - lava_callback - ERROR - Error processing callback for node 68d429a0a5c2c05277bc77b1: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 17:26:13,102 - lava_callback - INFO - Received callback for node 68d429a2a5c2c05277bc77b4 INFO: 172.21.0.1:58744 - "POST /node/68d429a2a5c2c05277bc77b4 HTTP/1.0" 202 Accepted 2025-09-26 17:26:13,108 - lava_callback - INFO - Starting async job submit for node 68d429a2a5c2c05277bc77b4 2025-09-26 17:26:13,109 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:26:13,121 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d429a2a5c2c05277bc77b4 HTTP/1.1" 200 None 2025-09-26 17:26:13,121 - lava_callback - DEBUG - Processing node 68d429a2a5c2c05277bc77b4: baseline-arm 2025-09-26 17:26:13,122 - lava_callback - ERROR - Error processing callback for node 68d429a2a5c2c05277bc77b4: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 17:26:13,825 - lava_callback - INFO - Received callback for node 68d429a3a5c2c05277bc77b7 INFO: 172.21.0.1:58756 - "POST /node/68d429a3a5c2c05277bc77b7 HTTP/1.0" 202 Accepted 2025-09-26 17:26:13,831 - lava_callback - INFO - Starting async job submit for node 68d429a3a5c2c05277bc77b7 2025-09-26 17:26:13,831 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:26:13,843 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d429a3a5c2c05277bc77b7 HTTP/1.1" 200 None 2025-09-26 17:26:13,844 - lava_callback - DEBUG - Processing node 68d429a3a5c2c05277bc77b7: baseline-arm 2025-09-26 17:26:13,844 - lava_callback - ERROR - Error processing callback for node 68d429a3a5c2c05277bc77b7: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 17:26:14,539 - lava_callback - INFO - Received callback for node 68d429a4a5c2c05277bc77ba INFO: 172.21.0.1:58772 - "POST /node/68d429a4a5c2c05277bc77ba HTTP/1.0" 202 Accepted 2025-09-26 17:26:14,544 - lava_callback - INFO - Starting async job submit for node 68d429a4a5c2c05277bc77ba 2025-09-26 17:26:14,545 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:26:14,561 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d429a4a5c2c05277bc77ba HTTP/1.1" 200 None 2025-09-26 17:26:14,562 - lava_callback - DEBUG - Processing node 68d429a4a5c2c05277bc77ba: baseline-arm 2025-09-26 17:26:14,562 - lava_callback - ERROR - Error processing callback for node 68d429a4a5c2c05277bc77ba: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:58784 - "GET /api/metrics HTTP/1.0" 200 OK 2025-09-26 17:26:15,270 - lava_callback - INFO - Received callback for node 68d429a6a5c2c05277bc77bd INFO: 172.21.0.1:58790 - "POST /node/68d429a6a5c2c05277bc77bd HTTP/1.0" 202 Accepted 2025-09-26 17:26:15,276 - lava_callback - INFO - Starting async job submit for node 68d429a6a5c2c05277bc77bd 2025-09-26 17:26:15,276 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:26:15,289 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d429a6a5c2c05277bc77bd HTTP/1.1" 200 None 2025-09-26 17:26:15,289 - lava_callback - DEBUG - Processing node 68d429a6a5c2c05277bc77bd: baseline-arm 2025-09-26 17:26:15,289 - lava_callback - ERROR - Error processing callback for node 68d429a6a5c2c05277bc77bd: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:58798 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:26:20,998 - lava_callback - INFO - Received callback for node 68d4332da5c2c05277bc875b INFO: 172.21.0.1:48120 - "POST /node/68d4332da5c2c05277bc875b HTTP/1.0" 202 Accepted 2025-09-26 17:26:21,004 - lava_callback - INFO - Starting async job submit for node 68d4332da5c2c05277bc875b 2025-09-26 17:26:21,025 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:26:21,048 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4332da5c2c05277bc875b HTTP/1.1" 200 None 2025-09-26 17:26:21,048 - lava_callback - DEBUG - Processing node 68d4332da5c2c05277bc875b: baseline-arm64 2025-09-26 17:26:21,350 - lava_callback - ERROR - Error processing callback for node 68d4332da5c2c05277bc875b: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:48128 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:26:27,045 - lava_callback - INFO - Received callback for node 68d6c7823c94e462c6881871 INFO: 172.21.0.1:48134 - "POST /node/68d6c7823c94e462c6881871 HTTP/1.0" 202 Accepted 2025-09-26 17:26:27,051 - lava_callback - INFO - Starting async job submit for node 68d6c7823c94e462c6881871 2025-09-26 17:26:27,072 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:26:27,084 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6c7823c94e462c6881871 HTTP/1.1" 200 None 2025-09-26 17:26:27,084 - lava_callback - DEBUG - Processing node 68d6c7823c94e462c6881871: baseline-arm64 2025-09-26 17:26:27,431 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 17:26:27,582 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 17:26:27,583 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6c7823c94e462c6881871/log.txt.gz 2025-09-26 17:26:27,591 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 17:26:27,744 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 17:26:27,745 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6c7823c94e462c6881871/lava_callback.json.gz 2025-09-26 17:26:27,766 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:26:27,778 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6c7823c94e462c6881871 HTTP/1.1" 200 None 2025-09-26 17:26:27,779 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:26:27,794 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6b9dd2f83206171a92b48 HTTP/1.1" 200 None 2025-09-26 17:26:27,795 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:26:27,821 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6c7823c94e462c6881871 HTTP/1.1" 200 None 2025-09-26 17:26:27,822 - lava_callback - INFO - Completed processing callback for node 68d6c7823c94e462c6881871 2025-09-26 17:26:28,466 - lava_callback - INFO - Received callback for node 68d429c2a5c2c05277bc77f7 INFO: 172.21.0.1:48138 - "POST /node/68d429c2a5c2c05277bc77f7 HTTP/1.0" 202 Accepted 2025-09-26 17:26:28,472 - lava_callback - INFO - Starting async job submit for node 68d429c2a5c2c05277bc77f7 2025-09-26 17:26:28,473 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:26:28,485 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d429c2a5c2c05277bc77f7 HTTP/1.1" 200 None 2025-09-26 17:26:28,485 - lava_callback - DEBUG - Processing node 68d429c2a5c2c05277bc77f7: ltp-containers 2025-09-26 17:26:28,485 - lava_callback - ERROR - Error processing callback for node 68d429c2a5c2c05277bc77f7: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:48152 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:26:34,247 - lava_callback - INFO - Received callback for node 68d43332a5c2c05277bc875e INFO: 172.21.0.1:38908 - "POST /node/68d43332a5c2c05277bc875e HTTP/1.0" 202 Accepted 2025-09-26 17:26:34,254 - lava_callback - INFO - Starting async job submit for node 68d43332a5c2c05277bc875e 2025-09-26 17:26:34,278 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:26:34,290 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d43332a5c2c05277bc875e HTTP/1.1" 200 None 2025-09-26 17:26:34,290 - lava_callback - DEBUG - Processing node 68d43332a5c2c05277bc875e: baseline-arm64 2025-09-26 17:26:34,615 - lava_callback - ERROR - Error processing callback for node 68d43332a5c2c05277bc875e: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:38920 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:26:43,835 - lava_callback - INFO - Received callback for node 68d43333a5c2c05277bc8761 INFO: 172.21.0.1:33424 - "POST /node/68d43333a5c2c05277bc8761 HTTP/1.0" 202 Accepted 2025-09-26 17:26:43,841 - lava_callback - INFO - Starting async job submit for node 68d43333a5c2c05277bc8761 2025-09-26 17:26:43,862 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:26:43,874 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d43333a5c2c05277bc8761 HTTP/1.1" 200 None 2025-09-26 17:26:43,874 - lava_callback - DEBUG - Processing node 68d43333a5c2c05277bc8761: baseline-arm64 2025-09-26 17:26:44,179 - lava_callback - ERROR - Error processing callback for node 68d43333a5c2c05277bc8761: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:33436 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:26:48,900 - lava_callback - INFO - Received callback for node 68d6c7893c94e462c688189e INFO: 172.21.0.1:33448 - "POST /node/68d6c7893c94e462c688189e HTTP/1.0" 202 Accepted 2025-09-26 17:26:48,906 - lava_callback - INFO - Starting async job submit for node 68d6c7893c94e462c688189e 2025-09-26 17:26:48,927 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:26:48,958 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6c7893c94e462c688189e HTTP/1.1" 200 None 2025-09-26 17:26:48,958 - lava_callback - DEBUG - Processing node 68d6c7893c94e462c688189e: baseline-arm64 2025-09-26 17:26:49,087 - lava_callback - INFO - Received callback for node 68d486a0a5c2c05277bcdcf8 INFO: 172.21.0.1:33454 - "POST /node/68d486a0a5c2c05277bcdcf8 HTTP/1.0" 202 Accepted 2025-09-26 17:26:49,114 - lava_callback - INFO - Starting async job submit for node 68d486a0a5c2c05277bcdcf8 2025-09-26 17:26:49,161 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:26:49,230 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d486a0a5c2c05277bcdcf8 HTTP/1.1" 200 None 2025-09-26 17:26:49,235 - lava_callback - DEBUG - Processing node 68d486a0a5c2c05277bcdcf8: baseline-arm 2025-09-26 17:26:49,318 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 17:26:49,516 - lava_callback - ERROR - Error processing callback for node 68d486a0a5c2c05277bcdcf8: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None 2025-09-26 17:26:49,523 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 17:26:49,524 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6c7893c94e462c688189e/log.txt.gz 2025-09-26 17:26:49,533 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 17:26:49,660 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 17:26:49,660 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6c7893c94e462c688189e/lava_callback.json.gz 2025-09-26 17:26:49,681 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:26:49,699 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6c7893c94e462c688189e HTTP/1.1" 200 None 2025-09-26 17:26:49,700 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:26:49,712 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6b9dd2f83206171a92b48 HTTP/1.1" 200 None 2025-09-26 17:26:49,713 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:26:49,743 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6c7893c94e462c688189e HTTP/1.1" 200 None 2025-09-26 17:26:49,743 - lava_callback - INFO - Completed processing callback for node 68d6c7893c94e462c688189e INFO: 172.21.0.1:54512 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54654 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54662 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:27:08,462 - lava_callback - INFO - Received callback for node 68d429daa5c2c05277bc7821 INFO: 172.21.0.1:54676 - "POST /node/68d429daa5c2c05277bc7821 HTTP/1.0" 202 Accepted 2025-09-26 17:27:08,468 - lava_callback - INFO - Starting async job submit for node 68d429daa5c2c05277bc7821 2025-09-26 17:27:08,469 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:27:08,483 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d429daa5c2c05277bc7821 HTTP/1.1" 200 None 2025-09-26 17:27:08,483 - lava_callback - DEBUG - Processing node 68d429daa5c2c05277bc7821: ltp-containers 2025-09-26 17:27:08,483 - lava_callback - ERROR - Error processing callback for node 68d429daa5c2c05277bc7821: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 17:27:09,314 - lava_callback - INFO - Received callback for node 68d429e3a5c2c05277bc7830 INFO: 172.21.0.1:46364 - "POST /node/68d429e3a5c2c05277bc7830 HTTP/1.0" 202 Accepted 2025-09-26 17:27:09,320 - lava_callback - INFO - Starting async job submit for node 68d429e3a5c2c05277bc7830 2025-09-26 17:27:09,321 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:27:09,333 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d429e3a5c2c05277bc7830 HTTP/1.1" 200 None 2025-09-26 17:27:09,333 - lava_callback - DEBUG - Processing node 68d429e3a5c2c05277bc7830: ltp-controllers 2025-09-26 17:27:09,333 - lava_callback - ERROR - Error processing callback for node 68d429e3a5c2c05277bc7830: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:46370 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46376 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39320 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:27:28,433 - lava_callback - INFO - Received callback for node 68d429efa5c2c05277bc7854 INFO: 172.21.0.1:39336 - "POST /node/68d429efa5c2c05277bc7854 HTTP/1.0" 202 Accepted 2025-09-26 17:27:28,439 - lava_callback - INFO - Starting async job submit for node 68d429efa5c2c05277bc7854 2025-09-26 17:27:28,440 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:27:28,452 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d429efa5c2c05277bc7854 HTTP/1.1" 200 None 2025-09-26 17:27:28,452 - lava_callback - DEBUG - Processing node 68d429efa5c2c05277bc7854: ltp-controllers 2025-09-26 17:27:28,452 - lava_callback - ERROR - Error processing callback for node 68d429efa5c2c05277bc7854: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:39348 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:27:29,205 - lava_callback - INFO - Received callback for node 68d429fca5c2c05277bc7863 INFO: 172.21.0.1:39364 - "POST /node/68d429fca5c2c05277bc7863 HTTP/1.0" 202 Accepted 2025-09-26 17:27:29,211 - lava_callback - INFO - Starting async job submit for node 68d429fca5c2c05277bc7863 2025-09-26 17:27:29,212 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:27:29,223 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d429fca5c2c05277bc7863 HTTP/1.1" 200 None 2025-09-26 17:27:29,224 - lava_callback - DEBUG - Processing node 68d429fca5c2c05277bc7863: ltp-cve 2025-09-26 17:27:29,224 - lava_callback - ERROR - Error processing callback for node 68d429fca5c2c05277bc7863: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:34438 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:27:46,365 - lava_callback - INFO - Received callback for node 68d43336a5c2c05277bc8764 INFO: 172.21.0.1:39478 - "POST /node/68d43336a5c2c05277bc8764 HTTP/1.0" 202 Accepted 2025-09-26 17:27:46,372 - lava_callback - INFO - Starting async job submit for node 68d43336a5c2c05277bc8764 2025-09-26 17:27:46,408 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:27:46,420 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d43336a5c2c05277bc8764 HTTP/1.1" 200 None 2025-09-26 17:27:46,421 - lava_callback - DEBUG - Processing node 68d43336a5c2c05277bc8764: baseline-arm64 INFO: 172.21.0.1:39488 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:27:46,916 - lava_callback - ERROR - Error processing callback for node 68d43336a5c2c05277bc8764: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None 2025-09-26 17:27:48,482 - lava_callback - INFO - Received callback for node 68d42a0fa5c2c05277bc787e INFO: 172.21.0.1:39502 - "POST /node/68d42a0fa5c2c05277bc787e HTTP/1.0" 202 Accepted 2025-09-26 17:27:48,488 - lava_callback - INFO - Starting async job submit for node 68d42a0fa5c2c05277bc787e 2025-09-26 17:27:48,489 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:27:48,523 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d42a0fa5c2c05277bc787e HTTP/1.1" 200 None 2025-09-26 17:27:48,523 - lava_callback - DEBUG - Processing node 68d42a0fa5c2c05277bc787e: ltp-cve 2025-09-26 17:27:48,523 - lava_callback - ERROR - Error processing callback for node 68d42a0fa5c2c05277bc787e: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 17:27:50,062 - lava_callback - INFO - Received callback for node 68d6c7863c94e462c688188c INFO: 172.21.0.1:60806 - "POST /node/68d6c7863c94e462c688188c HTTP/1.0" 202 Accepted 2025-09-26 17:27:50,068 - lava_callback - INFO - Starting async job submit for node 68d6c7863c94e462c688188c 2025-09-26 17:27:50,089 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:27:50,131 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6c7863c94e462c688188c HTTP/1.1" 200 None 2025-09-26 17:27:50,132 - lava_callback - DEBUG - Processing node 68d6c7863c94e462c688188c: baseline-arm64 2025-09-26 17:27:50,498 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 17:27:50,646 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 17:27:50,647 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6c7863c94e462c688188c/log.txt.gz 2025-09-26 17:27:50,654 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 17:27:50,763 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 17:27:50,764 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6c7863c94e462c688188c/lava_callback.json.gz 2025-09-26 17:27:50,784 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:27:50,813 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6c7863c94e462c688188c HTTP/1.1" 200 None 2025-09-26 17:27:50,814 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:27:50,826 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6b9dd2f83206171a92b48 HTTP/1.1" 200 None 2025-09-26 17:27:50,827 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:27:50,855 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6c7863c94e462c688188c HTTP/1.1" 200 None 2025-09-26 17:27:50,855 - lava_callback - INFO - Completed processing callback for node 68d6c7863c94e462c688188c 2025-09-26 17:27:54,076 - lava_callback - INFO - Received callback for node 68d43338a5c2c05277bc8767 INFO: 172.21.0.1:60822 - "POST /node/68d43338a5c2c05277bc8767 HTTP/1.0" 202 Accepted 2025-09-26 17:27:54,082 - lava_callback - INFO - Starting async job submit for node 68d43338a5c2c05277bc8767 2025-09-26 17:27:54,102 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:27:54,114 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d43338a5c2c05277bc8767 HTTP/1.1" 200 None 2025-09-26 17:27:54,115 - lava_callback - DEBUG - Processing node 68d43338a5c2c05277bc8767: baseline-arm64 2025-09-26 17:27:54,410 - lava_callback - ERROR - Error processing callback for node 68d43338a5c2c05277bc8767: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:60824 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:28:04,446 - lava_callback - INFO - Received callback for node 68d4333ba5c2c05277bc876a INFO: 172.21.0.1:38744 - "POST /node/68d4333ba5c2c05277bc876a HTTP/1.0" 202 Accepted 2025-09-26 17:28:04,452 - lava_callback - INFO - Starting async job submit for node 68d4333ba5c2c05277bc876a 2025-09-26 17:28:04,472 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:28:04,485 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4333ba5c2c05277bc876a HTTP/1.1" 200 None 2025-09-26 17:28:04,485 - lava_callback - DEBUG - Processing node 68d4333ba5c2c05277bc876a: baseline-arm64 2025-09-26 17:28:04,807 - lava_callback - ERROR - Error processing callback for node 68d4333ba5c2c05277bc876a: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:38752 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38756 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:28:08,536 - lava_callback - INFO - Received callback for node 68d42a22a5c2c05277bc7899 INFO: 172.21.0.1:38764 - "POST /node/68d42a22a5c2c05277bc7899 HTTP/1.0" 202 Accepted 2025-09-26 17:28:08,542 - lava_callback - INFO - Starting async job submit for node 68d42a22a5c2c05277bc7899 2025-09-26 17:28:08,542 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:28:08,555 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d42a22a5c2c05277bc7899 HTTP/1.1" 200 None 2025-09-26 17:28:08,555 - lava_callback - DEBUG - Processing node 68d42a22a5c2c05277bc7899: ltp-cap-bounds 2025-09-26 17:28:08,555 - lava_callback - ERROR - Error processing callback for node 68d42a22a5c2c05277bc7899: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:45560 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45576 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47796 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47804 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45930 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60722 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:28:49,059 - lava_callback - INFO - Received callback for node 68d486a1a5c2c05277bcdcfb INFO: 172.21.0.1:60734 - "POST /node/68d486a1a5c2c05277bcdcfb HTTP/1.0" 202 Accepted 2025-09-26 17:28:49,065 - lava_callback - INFO - Starting async job submit for node 68d486a1a5c2c05277bcdcfb 2025-09-26 17:28:49,086 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:28:49,102 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d486a1a5c2c05277bcdcfb HTTP/1.1" 200 None 2025-09-26 17:28:49,103 - lava_callback - DEBUG - Processing node 68d486a1a5c2c05277bcdcfb: baseline-arm 2025-09-26 17:28:49,321 - lava_callback - ERROR - Error processing callback for node 68d486a1a5c2c05277bcdcfb: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None 2025-09-26 17:28:52,611 - lava_callback - INFO - Received callback for node 68d4238ca5c2c05277bc6f39 INFO: 172.21.0.1:36528 - "POST /node/68d4238ca5c2c05277bc6f39 HTTP/1.0" 202 Accepted 2025-09-26 17:28:52,619 - lava_callback - INFO - Starting async job submit for node 68d4238ca5c2c05277bc6f39 2025-09-26 17:28:52,722 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:28:52,802 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4238ca5c2c05277bc6f39 HTTP/1.1" 200 None 2025-09-26 17:28:52,803 - lava_callback - DEBUG - Processing node 68d4238ca5c2c05277bc6f39: ltp-cve 2025-09-26 17:28:53,529 - lava_callback - ERROR - Error processing callback for node 68d4238ca5c2c05277bc6f39: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None 2025-09-26 17:28:54,063 - lava_callback - INFO - Received callback for node 68d429ada5c2c05277bc77dc INFO: 172.21.0.1:36530 - "POST /node/68d429ada5c2c05277bc77dc HTTP/1.0" 202 Accepted 2025-09-26 17:28:54,069 - lava_callback - INFO - Starting async job submit for node 68d429ada5c2c05277bc77dc 2025-09-26 17:28:54,108 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:28:54,142 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d429ada5c2c05277bc77dc HTTP/1.1" 200 None 2025-09-26 17:28:54,143 - lava_callback - DEBUG - Processing node 68d429ada5c2c05277bc77dc: ltp-cap-bounds 2025-09-26 17:28:54,270 - lava_callback - ERROR - Error processing callback for node 68d429ada5c2c05277bc77dc: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:36546 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:29:02,460 - lava_callback - INFO - Received callback for node 68d4333ea5c2c05277bc876d INFO: 172.21.0.1:53522 - "POST /node/68d4333ea5c2c05277bc876d HTTP/1.0" 202 Accepted 2025-09-26 17:29:02,466 - lava_callback - INFO - Starting async job submit for node 68d4333ea5c2c05277bc876d 2025-09-26 17:29:02,486 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:29:02,499 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4333ea5c2c05277bc876d HTTP/1.1" 200 None 2025-09-26 17:29:02,499 - lava_callback - DEBUG - Processing node 68d4333ea5c2c05277bc876d: baseline-arm64 2025-09-26 17:29:02,815 - lava_callback - ERROR - Error processing callback for node 68d4333ea5c2c05277bc876d: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:53534 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53536 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:29:11,058 - lava_callback - INFO - Received callback for node 68d42a61a5c2c05277bc7903 INFO: 172.21.0.1:33506 - "POST /node/68d42a61a5c2c05277bc7903 HTTP/1.0" 202 Accepted 2025-09-26 17:29:11,063 - lava_callback - INFO - Starting async job submit for node 68d42a61a5c2c05277bc7903 2025-09-26 17:29:11,064 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:29:11,076 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d42a61a5c2c05277bc7903 HTTP/1.1" 200 None 2025-09-26 17:29:11,076 - lava_callback - DEBUG - Processing node 68d42a61a5c2c05277bc7903: baseline-arm 2025-09-26 17:29:11,076 - lava_callback - ERROR - Error processing callback for node 68d42a61a5c2c05277bc7903: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 17:29:14,462 - lava_callback - INFO - Received callback for node 68d43346a5c2c05277bc8787 INFO: 172.21.0.1:33520 - "POST /node/68d43346a5c2c05277bc8787 HTTP/1.0" 202 Accepted 2025-09-26 17:29:14,468 - lava_callback - INFO - Starting async job submit for node 68d43346a5c2c05277bc8787 2025-09-26 17:29:14,489 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:29:14,512 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d43346a5c2c05277bc8787 HTTP/1.1" 200 None 2025-09-26 17:29:14,513 - lava_callback - DEBUG - Processing node 68d43346a5c2c05277bc8787: baseline-arm64 2025-09-26 17:29:14,804 - lava_callback - ERROR - Error processing callback for node 68d43346a5c2c05277bc8787: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:33522 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33530 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:29:24,377 - lava_callback - INFO - Received callback for node 68d43343a5c2c05277bc8770 INFO: 172.21.0.1:55530 - "POST /node/68d43343a5c2c05277bc8770 HTTP/1.0" 202 Accepted 2025-09-26 17:29:24,384 - lava_callback - INFO - Starting async job submit for node 68d43343a5c2c05277bc8770 2025-09-26 17:29:24,404 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:29:24,420 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d43343a5c2c05277bc8770 HTTP/1.1" 200 None 2025-09-26 17:29:24,421 - lava_callback - DEBUG - Processing node 68d43343a5c2c05277bc8770: baseline-arm64 2025-09-26 17:29:24,717 - lava_callback - ERROR - Error processing callback for node 68d43343a5c2c05277bc8770: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:55532 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:29:28,575 - lava_callback - INFO - Received callback for node 68d42a6ea5c2c05277bc7927 INFO: 172.21.0.1:55542 - "POST /node/68d42a6ea5c2c05277bc7927 HTTP/1.0" 202 Accepted 2025-09-26 17:29:28,581 - lava_callback - INFO - Starting async job submit for node 68d42a6ea5c2c05277bc7927 2025-09-26 17:29:28,582 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:29:28,594 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d42a6ea5c2c05277bc7927 HTTP/1.1" 200 None 2025-09-26 17:29:28,595 - lava_callback - DEBUG - Processing node 68d42a6ea5c2c05277bc7927: ltp-fs-bind 2025-09-26 17:29:28,595 - lava_callback - ERROR - Error processing callback for node 68d42a6ea5c2c05277bc7927: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:55550 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:29:29,429 - lava_callback - INFO - Received callback for node 68d42a75a5c2c05277bc7936 INFO: 172.21.0.1:36172 - "POST /node/68d42a75a5c2c05277bc7936 HTTP/1.0" 202 Accepted 2025-09-26 17:29:29,442 - lava_callback - INFO - Starting async job submit for node 68d42a75a5c2c05277bc7936 2025-09-26 17:29:29,443 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:29:29,505 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d42a75a5c2c05277bc7936 HTTP/1.1" 200 None 2025-09-26 17:29:29,506 - lava_callback - DEBUG - Processing node 68d42a75a5c2c05277bc7936: ltp-fs-bind 2025-09-26 17:29:29,506 - lava_callback - ERROR - Error processing callback for node 68d42a75a5c2c05277bc7936: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 17:29:31,318 - lava_callback - INFO - Received callback for node 68d42a64a5c2c05277bc791e INFO: 172.21.0.1:36180 - "POST /node/68d42a64a5c2c05277bc791e HTTP/1.0" 202 Accepted 2025-09-26 17:29:31,324 - lava_callback - INFO - Starting async job submit for node 68d42a64a5c2c05277bc791e 2025-09-26 17:29:31,325 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:29:31,338 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d42a64a5c2c05277bc791e HTTP/1.1" 200 None 2025-09-26 17:29:31,338 - lava_callback - DEBUG - Processing node 68d42a64a5c2c05277bc791e: baseline-arm 2025-09-26 17:29:31,338 - lava_callback - ERROR - Error processing callback for node 68d42a64a5c2c05277bc791e: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 17:29:32,037 - lava_callback - INFO - Received callback for node 68d42a68a5c2c05277bc7921 INFO: 172.21.0.1:36194 - "POST /node/68d42a68a5c2c05277bc7921 HTTP/1.0" 202 Accepted 2025-09-26 17:29:32,042 - lava_callback - INFO - Starting async job submit for node 68d42a68a5c2c05277bc7921 2025-09-26 17:29:32,043 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:29:32,055 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d42a68a5c2c05277bc7921 HTTP/1.1" 200 None 2025-09-26 17:29:32,055 - lava_callback - DEBUG - Processing node 68d42a68a5c2c05277bc7921: baseline-arm 2025-09-26 17:29:32,055 - lava_callback - ERROR - Error processing callback for node 68d42a68a5c2c05277bc7921: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 17:29:32,749 - lava_callback - INFO - Received callback for node 68d42a69a5c2c05277bc7924 INFO: 172.21.0.1:36204 - "POST /node/68d42a69a5c2c05277bc7924 HTTP/1.0" 202 Accepted 2025-09-26 17:29:32,755 - lava_callback - INFO - Starting async job submit for node 68d42a69a5c2c05277bc7924 2025-09-26 17:29:32,756 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:29:32,773 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d42a69a5c2c05277bc7924 HTTP/1.1" 200 None 2025-09-26 17:29:32,774 - lava_callback - DEBUG - Processing node 68d42a69a5c2c05277bc7924: baseline-arm 2025-09-26 17:29:32,774 - lava_callback - ERROR - Error processing callback for node 68d42a69a5c2c05277bc7924: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 17:29:33,456 - lava_callback - INFO - Received callback for node 68d42a6fa5c2c05277bc792a INFO: 172.21.0.1:36218 - "POST /node/68d42a6fa5c2c05277bc792a HTTP/1.0" 202 Accepted 2025-09-26 17:29:33,462 - lava_callback - INFO - Starting async job submit for node 68d42a6fa5c2c05277bc792a 2025-09-26 17:29:33,463 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:29:33,476 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d42a6fa5c2c05277bc792a HTTP/1.1" 200 None 2025-09-26 17:29:33,476 - lava_callback - DEBUG - Processing node 68d42a6fa5c2c05277bc792a: baseline-arm 2025-09-26 17:29:33,476 - lava_callback - ERROR - Error processing callback for node 68d42a6fa5c2c05277bc792a: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 17:29:34,121 - lava_callback - INFO - Received callback for node 68d42a6fa5c2c05277bc792d INFO: 172.21.0.1:36230 - "POST /node/68d42a6fa5c2c05277bc792d HTTP/1.0" 202 Accepted 2025-09-26 17:29:34,129 - lava_callback - INFO - Starting async job submit for node 68d42a6fa5c2c05277bc792d 2025-09-26 17:29:34,130 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:29:34,158 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d42a6fa5c2c05277bc792d HTTP/1.1" 200 None 2025-09-26 17:29:34,159 - lava_callback - DEBUG - Processing node 68d42a6fa5c2c05277bc792d: baseline-arm 2025-09-26 17:29:34,159 - lava_callback - ERROR - Error processing callback for node 68d42a6fa5c2c05277bc792d: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 17:29:34,817 - lava_callback - INFO - Received callback for node 68d42a71a5c2c05277bc7930 INFO: 172.21.0.1:36234 - "POST /node/68d42a71a5c2c05277bc7930 HTTP/1.0" 202 Accepted 2025-09-26 17:29:34,824 - lava_callback - INFO - Starting async job submit for node 68d42a71a5c2c05277bc7930 2025-09-26 17:29:34,825 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:29:34,838 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d42a71a5c2c05277bc7930 HTTP/1.1" 200 None 2025-09-26 17:29:34,838 - lava_callback - DEBUG - Processing node 68d42a71a5c2c05277bc7930: baseline-arm 2025-09-26 17:29:34,838 - lava_callback - ERROR - Error processing callback for node 68d42a71a5c2c05277bc7930: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 17:29:35,500 - lava_callback - INFO - Received callback for node 68d42a72a5c2c05277bc7933 INFO: 172.21.0.1:36246 - "POST /node/68d42a72a5c2c05277bc7933 HTTP/1.0" 202 Accepted 2025-09-26 17:29:35,506 - lava_callback - INFO - Starting async job submit for node 68d42a72a5c2c05277bc7933 2025-09-26 17:29:35,506 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:29:35,519 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d42a72a5c2c05277bc7933 HTTP/1.1" 200 None 2025-09-26 17:29:35,520 - lava_callback - DEBUG - Processing node 68d42a72a5c2c05277bc7933: baseline-arm 2025-09-26 17:29:35,520 - lava_callback - ERROR - Error processing callback for node 68d42a72a5c2c05277bc7933: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 17:29:36,174 - lava_callback - INFO - Received callback for node 68d42a77a5c2c05277bc7939 INFO: 172.21.0.1:36250 - "POST /node/68d42a77a5c2c05277bc7939 HTTP/1.0" 202 Accepted 2025-09-26 17:29:36,182 - lava_callback - INFO - Starting async job submit for node 68d42a77a5c2c05277bc7939 2025-09-26 17:29:36,183 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:29:36,200 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d42a77a5c2c05277bc7939 HTTP/1.1" 200 None 2025-09-26 17:29:36,200 - lava_callback - DEBUG - Processing node 68d42a77a5c2c05277bc7939: baseline-arm 2025-09-26 17:29:36,200 - lava_callback - ERROR - Error processing callback for node 68d42a77a5c2c05277bc7939: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:36256 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:29:36,866 - lava_callback - INFO - Received callback for node 68d42a78a5c2c05277bc793e INFO: 172.21.0.1:36264 - "POST /node/68d42a78a5c2c05277bc793e HTTP/1.0" 202 Accepted 2025-09-26 17:29:36,871 - lava_callback - INFO - Starting async job submit for node 68d42a78a5c2c05277bc793e 2025-09-26 17:29:36,872 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:29:36,885 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d42a78a5c2c05277bc793e HTTP/1.1" 200 None 2025-09-26 17:29:36,886 - lava_callback - DEBUG - Processing node 68d42a78a5c2c05277bc793e: baseline-arm 2025-09-26 17:29:36,886 - lava_callback - ERROR - Error processing callback for node 68d42a78a5c2c05277bc793e: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:37810 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:29:48,598 - lava_callback - INFO - Received callback for node 68d42a87a5c2c05277bc7954 INFO: 172.21.0.1:37814 - "POST /node/68d42a87a5c2c05277bc7954 HTTP/1.0" 202 Accepted 2025-09-26 17:29:48,604 - lava_callback - INFO - Starting async job submit for node 68d42a87a5c2c05277bc7954 2025-09-26 17:29:48,605 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:29:48,634 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d42a87a5c2c05277bc7954 HTTP/1.1" 200 None 2025-09-26 17:29:48,635 - lava_callback - DEBUG - Processing node 68d42a87a5c2c05277bc7954: ltp-controllers 2025-09-26 17:29:48,635 - lava_callback - ERROR - Error processing callback for node 68d42a87a5c2c05277bc7954: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 17:29:51,245 - lava_callback - INFO - Received callback for node 68d42a7aa5c2c05277bc7941 INFO: 172.21.0.1:46708 - "POST /node/68d42a7aa5c2c05277bc7941 HTTP/1.0" 202 Accepted 2025-09-26 17:29:51,250 - lava_callback - INFO - Starting async job submit for node 68d42a7aa5c2c05277bc7941 2025-09-26 17:29:51,251 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:29:51,263 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d42a7aa5c2c05277bc7941 HTTP/1.1" 200 None 2025-09-26 17:29:51,264 - lava_callback - DEBUG - Processing node 68d42a7aa5c2c05277bc7941: baseline-arm 2025-09-26 17:29:51,264 - lava_callback - ERROR - Error processing callback for node 68d42a7aa5c2c05277bc7941: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 17:29:51,941 - lava_callback - INFO - Received callback for node 68d42a7ea5c2c05277bc7944 INFO: 172.21.0.1:46712 - "POST /node/68d42a7ea5c2c05277bc7944 HTTP/1.0" 202 Accepted 2025-09-26 17:29:51,946 - lava_callback - INFO - Starting async job submit for node 68d42a7ea5c2c05277bc7944 2025-09-26 17:29:51,947 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:29:51,963 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d42a7ea5c2c05277bc7944 HTTP/1.1" 200 None 2025-09-26 17:29:51,963 - lava_callback - DEBUG - Processing node 68d42a7ea5c2c05277bc7944: baseline-arm 2025-09-26 17:29:51,963 - lava_callback - ERROR - Error processing callback for node 68d42a7ea5c2c05277bc7944: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 17:29:52,635 - lava_callback - INFO - Received callback for node 68d42a80a5c2c05277bc7947 INFO: 172.21.0.1:46726 - "POST /node/68d42a80a5c2c05277bc7947 HTTP/1.0" 202 Accepted 2025-09-26 17:29:52,640 - lava_callback - INFO - Starting async job submit for node 68d42a80a5c2c05277bc7947 2025-09-26 17:29:52,641 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:29:52,654 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d42a80a5c2c05277bc7947 HTTP/1.1" 200 None 2025-09-26 17:29:52,654 - lava_callback - DEBUG - Processing node 68d42a80a5c2c05277bc7947: baseline-arm 2025-09-26 17:29:52,654 - lava_callback - ERROR - Error processing callback for node 68d42a80a5c2c05277bc7947: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 17:29:53,341 - lava_callback - INFO - Received callback for node 68d42a82a5c2c05277bc794a INFO: 172.21.0.1:46740 - "POST /node/68d42a82a5c2c05277bc794a HTTP/1.0" 202 Accepted 2025-09-26 17:29:53,347 - lava_callback - INFO - Starting async job submit for node 68d42a82a5c2c05277bc794a 2025-09-26 17:29:53,348 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:29:53,370 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d42a82a5c2c05277bc794a HTTP/1.1" 200 None 2025-09-26 17:29:53,371 - lava_callback - DEBUG - Processing node 68d42a82a5c2c05277bc794a: baseline-arm 2025-09-26 17:29:53,371 - lava_callback - ERROR - Error processing callback for node 68d42a82a5c2c05277bc794a: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 17:29:54,049 - lava_callback - INFO - Received callback for node 68d42a83a5c2c05277bc794e INFO: 172.21.0.1:46744 - "POST /node/68d42a83a5c2c05277bc794e HTTP/1.0" 202 Accepted 2025-09-26 17:29:54,055 - lava_callback - INFO - Starting async job submit for node 68d42a83a5c2c05277bc794e 2025-09-26 17:29:54,056 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:29:54,090 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d42a83a5c2c05277bc794e HTTP/1.1" 200 None 2025-09-26 17:29:54,091 - lava_callback - DEBUG - Processing node 68d42a83a5c2c05277bc794e: baseline-arm 2025-09-26 17:29:54,091 - lava_callback - ERROR - Error processing callback for node 68d42a83a5c2c05277bc794e: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 17:29:54,768 - lava_callback - INFO - Received callback for node 68d42a84a5c2c05277bc7951 INFO: 172.21.0.1:46754 - "POST /node/68d42a84a5c2c05277bc7951 HTTP/1.0" 202 Accepted 2025-09-26 17:29:54,773 - lava_callback - INFO - Starting async job submit for node 68d42a84a5c2c05277bc7951 2025-09-26 17:29:54,774 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:29:54,810 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d42a84a5c2c05277bc7951 HTTP/1.1" 200 None 2025-09-26 17:29:54,811 - lava_callback - DEBUG - Processing node 68d42a84a5c2c05277bc7951: baseline-arm 2025-09-26 17:29:54,811 - lava_callback - ERROR - Error processing callback for node 68d42a84a5c2c05277bc7951: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 17:29:55,479 - lava_callback - INFO - Received callback for node 68d42a88a5c2c05277bc7957 INFO: 172.21.0.1:46758 - "POST /node/68d42a88a5c2c05277bc7957 HTTP/1.0" 202 Accepted 2025-09-26 17:29:55,485 - lava_callback - INFO - Starting async job submit for node 68d42a88a5c2c05277bc7957 2025-09-26 17:29:55,485 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:29:55,515 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d42a88a5c2c05277bc7957 HTTP/1.1" 200 None 2025-09-26 17:29:55,515 - lava_callback - DEBUG - Processing node 68d42a88a5c2c05277bc7957: baseline-arm 2025-09-26 17:29:55,515 - lava_callback - ERROR - Error processing callback for node 68d42a88a5c2c05277bc7957: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 17:29:56,151 - lava_callback - INFO - Received callback for node 68d42a8aa5c2c05277bc795a INFO: 172.21.0.1:46772 - "POST /node/68d42a8aa5c2c05277bc795a HTTP/1.0" 202 Accepted 2025-09-26 17:29:56,156 - lava_callback - INFO - Starting async job submit for node 68d42a8aa5c2c05277bc795a 2025-09-26 17:29:56,157 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:29:56,169 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d42a8aa5c2c05277bc795a HTTP/1.1" 200 None 2025-09-26 17:29:56,170 - lava_callback - DEBUG - Processing node 68d42a8aa5c2c05277bc795a: baseline-arm 2025-09-26 17:29:56,170 - lava_callback - ERROR - Error processing callback for node 68d42a8aa5c2c05277bc795a: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:46786 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37776 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37784 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:30:08,534 - lava_callback - INFO - Received callback for node 68d42a94a5c2c05277bc7963 INFO: 172.21.0.1:37790 - "POST /node/68d42a94a5c2c05277bc7963 HTTP/1.0" 202 Accepted 2025-09-26 17:30:08,540 - lava_callback - INFO - Starting async job submit for node 68d42a94a5c2c05277bc7963 2025-09-26 17:30:08,541 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:30:08,599 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d42a94a5c2c05277bc7963 HTTP/1.1" 200 None 2025-09-26 17:30:08,600 - lava_callback - DEBUG - Processing node 68d42a94a5c2c05277bc7963: ltp-fsx 2025-09-26 17:30:08,600 - lava_callback - ERROR - Error processing callback for node 68d42a94a5c2c05277bc7963: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 17:30:11,205 - lava_callback - INFO - Received callback for node 68d42a8da5c2c05277bc795d INFO: 172.21.0.1:60198 - "POST /node/68d42a8da5c2c05277bc795d HTTP/1.0" 202 Accepted 2025-09-26 17:30:11,211 - lava_callback - INFO - Starting async job submit for node 68d42a8da5c2c05277bc795d 2025-09-26 17:30:11,211 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:30:11,223 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d42a8da5c2c05277bc795d HTTP/1.1" 200 None 2025-09-26 17:30:11,224 - lava_callback - DEBUG - Processing node 68d42a8da5c2c05277bc795d: baseline-arm 2025-09-26 17:30:11,224 - lava_callback - ERROR - Error processing callback for node 68d42a8da5c2c05277bc795d: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 17:30:11,925 - lava_callback - INFO - Received callback for node 68d42a8fa5c2c05277bc7960 INFO: 172.21.0.1:60214 - "POST /node/68d42a8fa5c2c05277bc7960 HTTP/1.0" 202 Accepted 2025-09-26 17:30:11,931 - lava_callback - INFO - Starting async job submit for node 68d42a8fa5c2c05277bc7960 2025-09-26 17:30:11,931 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:30:11,943 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d42a8fa5c2c05277bc7960 HTTP/1.1" 200 None 2025-09-26 17:30:11,943 - lava_callback - DEBUG - Processing node 68d42a8fa5c2c05277bc7960: baseline-arm 2025-09-26 17:30:11,943 - lava_callback - ERROR - Error processing callback for node 68d42a8fa5c2c05277bc7960: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 17:30:12,666 - lava_callback - INFO - Received callback for node 68d42a97a5c2c05277bc796a INFO: 172.21.0.1:60220 - "POST /node/68d42a97a5c2c05277bc796a HTTP/1.0" 202 Accepted 2025-09-26 17:30:12,671 - lava_callback - INFO - Starting async job submit for node 68d42a97a5c2c05277bc796a 2025-09-26 17:30:12,672 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:30:12,687 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d42a97a5c2c05277bc796a HTTP/1.1" 200 None 2025-09-26 17:30:12,688 - lava_callback - DEBUG - Processing node 68d42a97a5c2c05277bc796a: baseline-arm 2025-09-26 17:30:12,688 - lava_callback - ERROR - Error processing callback for node 68d42a97a5c2c05277bc796a: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 17:30:13,364 - lava_callback - INFO - Received callback for node 68d42a9aa5c2c05277bc796d INFO: 172.21.0.1:60234 - "POST /node/68d42a9aa5c2c05277bc796d HTTP/1.0" 202 Accepted 2025-09-26 17:30:13,370 - lava_callback - INFO - Starting async job submit for node 68d42a9aa5c2c05277bc796d 2025-09-26 17:30:13,371 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:30:13,383 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d42a9aa5c2c05277bc796d HTTP/1.1" 200 None 2025-09-26 17:30:13,384 - lava_callback - DEBUG - Processing node 68d42a9aa5c2c05277bc796d: baseline-arm 2025-09-26 17:30:13,384 - lava_callback - ERROR - Error processing callback for node 68d42a9aa5c2c05277bc796d: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 17:30:14,077 - lava_callback - INFO - Received callback for node 68d42a9ba5c2c05277bc7970 INFO: 172.21.0.1:60250 - "POST /node/68d42a9ba5c2c05277bc7970 HTTP/1.0" 202 Accepted 2025-09-26 17:30:14,083 - lava_callback - INFO - Starting async job submit for node 68d42a9ba5c2c05277bc7970 2025-09-26 17:30:14,083 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:30:14,096 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d42a9ba5c2c05277bc7970 HTTP/1.1" 200 None 2025-09-26 17:30:14,096 - lava_callback - DEBUG - Processing node 68d42a9ba5c2c05277bc7970: baseline-arm 2025-09-26 17:30:14,097 - lava_callback - ERROR - Error processing callback for node 68d42a9ba5c2c05277bc7970: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 17:30:14,763 - lava_callback - INFO - Received callback for node 68d42a9ca5c2c05277bc7973 INFO: 172.21.0.1:60258 - "POST /node/68d42a9ca5c2c05277bc7973 HTTP/1.0" 202 Accepted 2025-09-26 17:30:14,769 - lava_callback - INFO - Starting async job submit for node 68d42a9ca5c2c05277bc7973 2025-09-26 17:30:14,769 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:30:14,782 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d42a9ca5c2c05277bc7973 HTTP/1.1" 200 None 2025-09-26 17:30:14,782 - lava_callback - DEBUG - Processing node 68d42a9ca5c2c05277bc7973: baseline-arm 2025-09-26 17:30:14,783 - lava_callback - ERROR - Error processing callback for node 68d42a9ca5c2c05277bc7973: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:60268 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60280 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:30:24,583 - lava_callback - INFO - Received callback for node 68d4334ea5c2c05277bc878c INFO: 172.21.0.1:52304 - "POST /node/68d4334ea5c2c05277bc878c HTTP/1.0" 202 Accepted 2025-09-26 17:30:24,589 - lava_callback - INFO - Starting async job submit for node 68d4334ea5c2c05277bc878c 2025-09-26 17:30:24,610 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:30:24,622 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4334ea5c2c05277bc878c HTTP/1.1" 200 None 2025-09-26 17:30:24,623 - lava_callback - DEBUG - Processing node 68d4334ea5c2c05277bc878c: baseline-arm64 2025-09-26 17:30:24,940 - lava_callback - ERROR - Error processing callback for node 68d4334ea5c2c05277bc878c: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:52308 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:30:28,607 - lava_callback - INFO - Received callback for node 68d42aa2a5c2c05277bc7983 INFO: 172.21.0.1:52324 - "POST /node/68d42aa2a5c2c05277bc7983 HTTP/1.0" 202 Accepted 2025-09-26 17:30:28,613 - lava_callback - INFO - Starting async job submit for node 68d42aa2a5c2c05277bc7983 2025-09-26 17:30:28,614 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:30:28,626 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d42aa2a5c2c05277bc7983 HTTP/1.1" 200 None 2025-09-26 17:30:28,626 - lava_callback - DEBUG - Processing node 68d42aa2a5c2c05277bc7983: ltp-fsx 2025-09-26 17:30:28,626 - lava_callback - ERROR - Error processing callback for node 68d42aa2a5c2c05277bc7983: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:52338 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:30:29,415 - lava_callback - INFO - Received callback for node 68d42aaca5c2c05277bc7992 INFO: 172.21.0.1:43962 - "POST /node/68d42aaca5c2c05277bc7992 HTTP/1.0" 202 Accepted 2025-09-26 17:30:29,421 - lava_callback - INFO - Starting async job submit for node 68d42aaca5c2c05277bc7992 2025-09-26 17:30:29,422 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:30:29,434 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d42aaca5c2c05277bc7992 HTTP/1.1" 200 None 2025-09-26 17:30:29,435 - lava_callback - DEBUG - Processing node 68d42aaca5c2c05277bc7992: ltp-cve 2025-09-26 17:30:29,435 - lava_callback - ERROR - Error processing callback for node 68d42aaca5c2c05277bc7992: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 17:30:31,887 - lava_callback - INFO - Received callback for node 68d43351a5c2c05277bc878f INFO: 172.21.0.1:43972 - "POST /node/68d43351a5c2c05277bc878f HTTP/1.0" 202 Accepted 2025-09-26 17:30:31,893 - lava_callback - INFO - Starting async job submit for node 68d43351a5c2c05277bc878f 2025-09-26 17:30:31,914 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:30:31,927 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d43351a5c2c05277bc878f HTTP/1.1" 200 None 2025-09-26 17:30:31,927 - lava_callback - DEBUG - Processing node 68d43351a5c2c05277bc878f: baseline-arm64 2025-09-26 17:30:32,221 - lava_callback - ERROR - Error processing callback for node 68d43351a5c2c05277bc878f: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:43976 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:30:38,286 - lava_callback - INFO - Received callback for node 68d48bf0a5c2c05277bce525 INFO: 172.21.0.1:43982 - "POST /node/68d48bf0a5c2c05277bce525 HTTP/1.0" 202 Accepted 2025-09-26 17:30:38,292 - lava_callback - INFO - Starting async job submit for node 68d48bf0a5c2c05277bce525 2025-09-26 17:30:38,313 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:30:38,330 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d48bf0a5c2c05277bce525 HTTP/1.1" 200 None 2025-09-26 17:30:38,330 - lava_callback - DEBUG - Processing node 68d48bf0a5c2c05277bce525: baseline-arm 2025-09-26 17:30:38,574 - lava_callback - ERROR - Error processing callback for node 68d48bf0a5c2c05277bce525: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None 2025-09-26 17:30:41,215 - lava_callback - INFO - Received callback for node 68d43356a5c2c05277bc8792 INFO: 172.21.0.1:51926 - "POST /node/68d43356a5c2c05277bc8792 HTTP/1.0" 202 Accepted 2025-09-26 17:30:41,223 - lava_callback - INFO - Starting async job submit for node 68d43356a5c2c05277bc8792 2025-09-26 17:30:41,281 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:30:41,350 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d43356a5c2c05277bc8792 HTTP/1.1" 200 None 2025-09-26 17:30:41,351 - lava_callback - DEBUG - Processing node 68d43356a5c2c05277bc8792: baseline-arm64 2025-09-26 17:30:41,756 - lava_callback - ERROR - Error processing callback for node 68d43356a5c2c05277bc8792: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:51932 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:30:48,751 - lava_callback - INFO - Received callback for node 68d42ab5a5c2c05277bc79a1 INFO: 172.21.0.1:51938 - "POST /node/68d42ab5a5c2c05277bc79a1 HTTP/1.0" 202 Accepted 2025-09-26 17:30:48,758 - lava_callback - INFO - Starting async job submit for node 68d42ab5a5c2c05277bc79a1 2025-09-26 17:30:48,758 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:30:48,771 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d42ab5a5c2c05277bc79a1 HTTP/1.1" 200 None 2025-09-26 17:30:48,771 - lava_callback - DEBUG - Processing node 68d42ab5a5c2c05277bc79a1: ltp-input 2025-09-26 17:30:48,771 - lava_callback - ERROR - Error processing callback for node 68d42ab5a5c2c05277bc79a1: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 17:30:49,490 - lava_callback - INFO - Received callback for node 68d42ac3a5c2c05277bc79b1 INFO: 172.21.0.1:38968 - "POST /node/68d42ac3a5c2c05277bc79b1 HTTP/1.0" 202 Accepted 2025-09-26 17:30:49,496 - lava_callback - INFO - Starting async job submit for node 68d42ac3a5c2c05277bc79b1 2025-09-26 17:30:49,497 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:30:49,509 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d42ac3a5c2c05277bc79b1 HTTP/1.1" 200 None 2025-09-26 17:30:49,510 - lava_callback - DEBUG - Processing node 68d42ac3a5c2c05277bc79b1: ltp-input 2025-09-26 17:30:49,510 - lava_callback - ERROR - Error processing callback for node 68d42ac3a5c2c05277bc79b1: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:38984 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38920 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38924 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48040 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48048 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49016 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49028 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:31:28,782 - lava_callback - INFO - Received callback for node 68d42ae6a5c2c05277bc79fc INFO: 172.21.0.1:49038 - "POST /node/68d42ae6a5c2c05277bc79fc HTTP/1.0" 202 Accepted 2025-09-26 17:31:28,788 - lava_callback - INFO - Starting async job submit for node 68d42ae6a5c2c05277bc79fc 2025-09-26 17:31:28,789 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:31:28,801 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d42ae6a5c2c05277bc79fc HTTP/1.1" 200 None 2025-09-26 17:31:28,802 - lava_callback - DEBUG - Processing node 68d42ae6a5c2c05277bc79fc: ltp-ipc 2025-09-26 17:31:28,802 - lava_callback - ERROR - Error processing callback for node 68d42ae6a5c2c05277bc79fc: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 17:31:29,625 - lava_callback - INFO - Received callback for node 68d42aeea5c2c05277bc7a02 INFO: 172.21.0.1:58364 - "POST /node/68d42aeea5c2c05277bc7a02 HTTP/1.0" 202 Accepted 2025-09-26 17:31:29,633 - lava_callback - INFO - Starting async job submit for node 68d42aeea5c2c05277bc7a02 2025-09-26 17:31:29,634 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:31:29,647 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d42aeea5c2c05277bc7a02 HTTP/1.1" 200 None 2025-09-26 17:31:29,647 - lava_callback - DEBUG - Processing node 68d42aeea5c2c05277bc7a02: ltp-fs-bind 2025-09-26 17:31:29,648 - lava_callback - ERROR - Error processing callback for node 68d42aeea5c2c05277bc7a02: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:58370 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:31:41,591 - lava_callback - INFO - Received callback for node 68d43359a5c2c05277bc8799 INFO: 172.21.0.1:43880 - "POST /node/68d43359a5c2c05277bc8799 HTTP/1.0" 202 Accepted 2025-09-26 17:31:41,599 - lava_callback - INFO - Starting async job submit for node 68d43359a5c2c05277bc8799 2025-09-26 17:31:41,619 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:31:41,634 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d43359a5c2c05277bc8799 HTTP/1.1" 200 None 2025-09-26 17:31:41,635 - lava_callback - DEBUG - Processing node 68d43359a5c2c05277bc8799: baseline-arm64 2025-09-26 17:31:42,444 - lava_callback - ERROR - Error processing callback for node 68d43359a5c2c05277bc8799: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:43884 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:31:48,751 - lava_callback - INFO - Received callback for node 68d42af1a5c2c05277bc7a08 INFO: 172.21.0.1:43898 - "POST /node/68d42af1a5c2c05277bc7a08 HTTP/1.0" 202 Accepted 2025-09-26 17:31:48,757 - lava_callback - INFO - Starting async job submit for node 68d42af1a5c2c05277bc7a08 2025-09-26 17:31:48,758 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:31:48,771 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d42af1a5c2c05277bc7a08 HTTP/1.1" 200 None 2025-09-26 17:31:48,771 - lava_callback - DEBUG - Processing node 68d42af1a5c2c05277bc7a08: ltp-pty 2025-09-26 17:31:48,771 - lava_callback - ERROR - Error processing callback for node 68d42af1a5c2c05277bc7a08: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 17:31:49,532 - lava_callback - INFO - Received callback for node 68d42af4a5c2c05277bc7a0b INFO: 172.21.0.1:41368 - "POST /node/68d42af4a5c2c05277bc7a0b HTTP/1.0" 202 Accepted 2025-09-26 17:31:49,538 - lava_callback - INFO - Starting async job submit for node 68d42af4a5c2c05277bc7a0b 2025-09-26 17:31:49,539 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:31:49,556 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d42af4a5c2c05277bc7a0b HTTP/1.1" 200 None 2025-09-26 17:31:49,557 - lava_callback - DEBUG - Processing node 68d42af4a5c2c05277bc7a0b: ltp-pty 2025-09-26 17:31:49,557 - lava_callback - ERROR - Error processing callback for node 68d42af4a5c2c05277bc7a0b: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 17:31:50,286 - lava_callback - INFO - Received callback for node 68d42afea5c2c05277bc7a14 INFO: 172.21.0.1:41374 - "POST /node/68d42afea5c2c05277bc7a14 HTTP/1.0" 202 Accepted 2025-09-26 17:31:50,292 - lava_callback - INFO - Starting async job submit for node 68d42afea5c2c05277bc7a14 2025-09-26 17:31:50,293 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:31:50,305 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d42afea5c2c05277bc7a14 HTTP/1.1" 200 None 2025-09-26 17:31:50,305 - lava_callback - DEBUG - Processing node 68d42afea5c2c05277bc7a14: ltp-fsx 2025-09-26 17:31:50,305 - lava_callback - ERROR - Error processing callback for node 68d42afea5c2c05277bc7a14: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 17:31:52,868 - lava_callback - INFO - Received callback for node 68d4335ca5c2c05277bc879c INFO: 172.21.0.1:41378 - "POST /node/68d4335ca5c2c05277bc879c HTTP/1.0" 202 Accepted 2025-09-26 17:31:52,874 - lava_callback - INFO - Starting async job submit for node 68d4335ca5c2c05277bc879c 2025-09-26 17:31:52,895 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:31:52,910 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4335ca5c2c05277bc879c HTTP/1.1" 200 None 2025-09-26 17:31:52,911 - lava_callback - DEBUG - Processing node 68d4335ca5c2c05277bc879c: baseline-arm64 2025-09-26 17:31:53,210 - lava_callback - ERROR - Error processing callback for node 68d4335ca5c2c05277bc879c: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None 2025-09-26 17:31:55,978 - lava_callback - INFO - Received callback for node 68d6cde73c94e462c6881d36 INFO: 172.21.0.1:41382 - "POST /node/68d6cde73c94e462c6881d36 HTTP/1.0" 202 Accepted 2025-09-26 17:31:55,987 - lava_callback - INFO - Starting async job submit for node 68d6cde73c94e462c6881d36 2025-09-26 17:31:56,005 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:31:56,027 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6cde73c94e462c6881d36 HTTP/1.1" 200 None 2025-09-26 17:31:56,027 - lava_callback - DEBUG - Processing node 68d6cde73c94e462c6881d36: baseline-x86 2025-09-26 17:31:56,185 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 17:31:56,330 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 17:31:56,331 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-68d6cde73c94e462c6881d36/log.txt.gz 2025-09-26 17:31:56,336 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 17:31:56,471 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 17:31:56,472 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-68d6cde73c94e462c6881d36/lava_callback.json.gz 2025-09-26 17:31:56,489 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:31:56,502 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6cde73c94e462c6881d36 HTTP/1.1" 200 None 2025-09-26 17:31:56,503 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:31:56,514 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6bda33c94e462c6881504 HTTP/1.1" 200 None 2025-09-26 17:31:56,515 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.21.0.1:41396 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:31:56,544 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6cde73c94e462c6881d36 HTTP/1.1" 200 None 2025-09-26 17:31:56,544 - lava_callback - INFO - Completed processing callback for node 68d6cde73c94e462c6881d36 2025-09-26 17:31:59,335 - lava_callback - INFO - Received callback for node 68d6c3313c94e462c68817c5 INFO: 172.21.0.1:50200 - "POST /node/68d6c3313c94e462c68817c5 HTTP/1.0" 202 Accepted 2025-09-26 17:31:59,343 - lava_callback - INFO - Starting async job submit for node 68d6c3313c94e462c68817c5 2025-09-26 17:31:59,364 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:31:59,382 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6c3313c94e462c68817c5 HTTP/1.1" 200 None 2025-09-26 17:31:59,382 - lava_callback - DEBUG - Processing node 68d6c3313c94e462c68817c5: blktests-ddp-x86 2025-09-26 17:32:00,146 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 17:32:00,281 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 17:32:00,282 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/blktests-ddp-x86-68d6c3313c94e462c68817c5/log.txt.gz 2025-09-26 17:32:00,301 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 17:32:00,479 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 17:32:00,479 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/blktests-ddp-x86-68d6c3313c94e462c68817c5/lava_callback.json.gz 2025-09-26 17:32:00,525 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:32:00,537 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6c3313c94e462c68817c5 HTTP/1.1" 200 None 2025-09-26 17:32:00,538 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:32:00,551 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6b9f22f83206171a92b66 HTTP/1.1" 200 None 2025-09-26 17:32:00,552 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:32:00,609 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6c3313c94e462c68817c5 HTTP/1.1" 200 None 2025-09-26 17:32:00,609 - lava_callback - INFO - Completed processing callback for node 68d6c3313c94e462c68817c5 2025-09-26 17:32:06,162 - lava_callback - INFO - Received callback for node 68d4335fa5c2c05277bc879f INFO: 172.21.0.1:50204 - "POST /node/68d4335fa5c2c05277bc879f HTTP/1.0" 202 Accepted 2025-09-26 17:32:06,168 - lava_callback - INFO - Starting async job submit for node 68d4335fa5c2c05277bc879f 2025-09-26 17:32:06,193 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:32:06,208 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4335fa5c2c05277bc879f HTTP/1.1" 200 None 2025-09-26 17:32:06,209 - lava_callback - DEBUG - Processing node 68d4335fa5c2c05277bc879f: baseline-arm64 INFO: 172.21.0.1:50220 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:32:06,536 - lava_callback - ERROR - Error processing callback for node 68d4335fa5c2c05277bc879f: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:50228 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:32:08,733 - lava_callback - INFO - Received callback for node 68d42b06a5c2c05277bc7a1a INFO: 172.21.0.1:50234 - "POST /node/68d42b06a5c2c05277bc7a1a HTTP/1.0" 202 Accepted 2025-09-26 17:32:08,739 - lava_callback - INFO - Starting async job submit for node 68d42b06a5c2c05277bc7a1a 2025-09-26 17:32:08,739 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:32:08,755 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d42b06a5c2c05277bc7a1a HTTP/1.1" 200 None 2025-09-26 17:32:08,755 - lava_callback - DEBUG - Processing node 68d42b06a5c2c05277bc7a1a: ltp-input 2025-09-26 17:32:08,755 - lava_callback - ERROR - Error processing callback for node 68d42b06a5c2c05277bc7a1a: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 17:32:09,491 - lava_callback - INFO - Received callback for node 68d42b0aa5c2c05277bc7a20 INFO: 172.21.0.1:37190 - "POST /node/68d42b0aa5c2c05277bc7a20 HTTP/1.0" 202 Accepted 2025-09-26 17:32:09,497 - lava_callback - INFO - Starting async job submit for node 68d42b0aa5c2c05277bc7a20 2025-09-26 17:32:09,498 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:32:09,511 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d42b0aa5c2c05277bc7a20 HTTP/1.1" 200 None 2025-09-26 17:32:09,511 - lava_callback - DEBUG - Processing node 68d42b0aa5c2c05277bc7a20: ltp-sched 2025-09-26 17:32:09,512 - lava_callback - ERROR - Error processing callback for node 68d42b0aa5c2c05277bc7a20: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 17:32:10,273 - lava_callback - INFO - Received callback for node 68d42b0ea5c2c05277bc7a23 INFO: 172.21.0.1:37198 - "POST /node/68d42b0ea5c2c05277bc7a23 HTTP/1.0" 202 Accepted 2025-09-26 17:32:10,279 - lava_callback - INFO - Starting async job submit for node 68d42b0ea5c2c05277bc7a23 2025-09-26 17:32:10,280 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:32:10,292 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d42b0ea5c2c05277bc7a23 HTTP/1.1" 200 None 2025-09-26 17:32:10,292 - lava_callback - DEBUG - Processing node 68d42b0ea5c2c05277bc7a23: ltp-sched 2025-09-26 17:32:10,292 - lava_callback - ERROR - Error processing callback for node 68d42b0ea5c2c05277bc7a23: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 17:32:11,007 - lava_callback - INFO - Received callback for node 68d42b11a5c2c05277bc7a29 INFO: 172.21.0.1:37214 - "POST /node/68d42b11a5c2c05277bc7a29 HTTP/1.0" 202 Accepted 2025-09-26 17:32:11,013 - lava_callback - INFO - Starting async job submit for node 68d42b11a5c2c05277bc7a29 2025-09-26 17:32:11,014 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:32:11,026 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d42b11a5c2c05277bc7a29 HTTP/1.1" 200 None 2025-09-26 17:32:11,026 - lava_callback - DEBUG - Processing node 68d42b11a5c2c05277bc7a29: ltp-ipc 2025-09-26 17:32:11,026 - lava_callback - ERROR - Error processing callback for node 68d42b11a5c2c05277bc7a29: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 17:32:11,031 - lava_callback - INFO - Received callback for node 68d6c78f3c94e462c68818c2 INFO: 172.21.0.1:37230 - "POST /node/68d6c78f3c94e462c68818c2 HTTP/1.0" 202 Accepted 2025-09-26 17:32:11,037 - lava_callback - INFO - Starting async job submit for node 68d6c78f3c94e462c68818c2 2025-09-26 17:32:11,058 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:32:11,071 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6c78f3c94e462c68818c2 HTTP/1.1" 200 None 2025-09-26 17:32:11,072 - lava_callback - DEBUG - Processing node 68d6c78f3c94e462c68818c2: baseline-arm64-broonie 2025-09-26 17:32:11,314 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 17:32:11,422 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 17:32:11,423 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-68d6c78f3c94e462c68818c2/log.txt.gz 2025-09-26 17:32:11,429 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 17:32:11,577 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 17:32:11,578 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-68d6c78f3c94e462c68818c2/lava_callback.json.gz 2025-09-26 17:32:11,600 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:32:11,613 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6c78f3c94e462c68818c2 HTTP/1.1" 200 None 2025-09-26 17:32:11,614 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:32:11,625 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6b9dd2f83206171a92b48 HTTP/1.1" 200 None 2025-09-26 17:32:11,626 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:32:11,656 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6c78f3c94e462c68818c2 HTTP/1.1" 200 None 2025-09-26 17:32:11,657 - lava_callback - INFO - Completed processing callback for node 68d6c78f3c94e462c68818c2 2025-09-26 17:32:11,824 - lava_callback - INFO - Received callback for node 68d42b19a5c2c05277bc7a2f INFO: 172.21.0.1:37238 - "POST /node/68d42b19a5c2c05277bc7a2f HTTP/1.0" 202 Accepted 2025-09-26 17:32:11,831 - lava_callback - INFO - Starting async job submit for node 68d42b19a5c2c05277bc7a2f 2025-09-26 17:32:11,832 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:32:11,845 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d42b19a5c2c05277bc7a2f HTTP/1.1" 200 None 2025-09-26 17:32:11,846 - lava_callback - DEBUG - Processing node 68d42b19a5c2c05277bc7a2f: ltp-smoketest 2025-09-26 17:32:11,846 - lava_callback - ERROR - Error processing callback for node 68d42b19a5c2c05277bc7a2f: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:37252 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37264 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48200 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48210 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:32:28,988 - lava_callback - INFO - Received callback for node 68d42b23a5c2c05277bc7a4e INFO: 172.21.0.1:48224 - "POST /node/68d42b23a5c2c05277bc7a4e HTTP/1.0" 202 Accepted 2025-09-26 17:32:28,994 - lava_callback - INFO - Starting async job submit for node 68d42b23a5c2c05277bc7a4e 2025-09-26 17:32:28,995 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:32:29,034 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d42b23a5c2c05277bc7a4e HTTP/1.1" 200 None 2025-09-26 17:32:29,035 - lava_callback - DEBUG - Processing node 68d42b23a5c2c05277bc7a4e: ltp-smoketest 2025-09-26 17:32:29,035 - lava_callback - ERROR - Error processing callback for node 68d42b23a5c2c05277bc7a4e: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 17:32:31,349 - lava_callback - INFO - Received callback for node 68d6c78b3c94e462c68818ad INFO: 172.21.0.1:56634 - "POST /node/68d6c78b3c94e462c68818ad HTTP/1.0" 202 Accepted 2025-09-26 17:32:31,355 - lava_callback - INFO - Starting async job submit for node 68d6c78b3c94e462c68818ad 2025-09-26 17:32:31,376 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:32:31,399 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6c78b3c94e462c68818ad HTTP/1.1" 200 None 2025-09-26 17:32:31,400 - lava_callback - DEBUG - Processing node 68d6c78b3c94e462c68818ad: baseline-arm64-broonie 2025-09-26 17:32:31,642 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 17:32:31,759 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 17:32:31,760 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-68d6c78b3c94e462c68818ad/log.txt.gz 2025-09-26 17:32:31,766 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 17:32:31,889 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 17:32:31,890 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-68d6c78b3c94e462c68818ad/lava_callback.json.gz 2025-09-26 17:32:31,929 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:32:31,941 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6c78b3c94e462c68818ad HTTP/1.1" 200 None 2025-09-26 17:32:31,942 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:32:31,985 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6b9dd2f83206171a92b48 HTTP/1.1" 200 None 2025-09-26 17:32:31,986 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:32:32,013 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6c78b3c94e462c68818ad HTTP/1.1" 200 None 2025-09-26 17:32:32,014 - lava_callback - INFO - Completed processing callback for node 68d6c78b3c94e462c68818ad INFO: 172.21.0.1:56650 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46844 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:32:48,356 - lava_callback - INFO - Received callback for node 68d48bfaa5c2c05277bce531 INFO: 172.21.0.1:46858 - "POST /node/68d48bfaa5c2c05277bce531 HTTP/1.0" 202 Accepted 2025-09-26 17:32:48,362 - lava_callback - INFO - Starting async job submit for node 68d48bfaa5c2c05277bce531 2025-09-26 17:32:48,381 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:32:48,397 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d48bfaa5c2c05277bce531 HTTP/1.1" 200 None 2025-09-26 17:32:48,398 - lava_callback - DEBUG - Processing node 68d48bfaa5c2c05277bce531: baseline-arm 2025-09-26 17:32:48,624 - lava_callback - ERROR - Error processing callback for node 68d48bfaa5c2c05277bce531: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None 2025-09-26 17:32:48,833 - lava_callback - INFO - Received callback for node 68d42b2ba5c2c05277bc7a54 INFO: 172.21.0.1:46862 - "POST /node/68d42b2ba5c2c05277bc7a54 HTTP/1.0" 202 Accepted 2025-09-26 17:32:48,839 - lava_callback - INFO - Starting async job submit for node 68d42b2ba5c2c05277bc7a54 2025-09-26 17:32:48,840 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:32:48,852 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d42b2ba5c2c05277bc7a54 HTTP/1.1" 200 None 2025-09-26 17:32:48,853 - lava_callback - DEBUG - Processing node 68d42b2ba5c2c05277bc7a54: ltp-pty 2025-09-26 17:32:48,853 - lava_callback - ERROR - Error processing callback for node 68d42b2ba5c2c05277bc7a54: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 17:32:49,567 - lava_callback - INFO - Received callback for node 68d42b2fa5c2c05277bc7e05 INFO: 172.21.0.1:57732 - "POST /node/68d42b2fa5c2c05277bc7e05 HTTP/1.0" 202 Accepted 2025-09-26 17:32:49,573 - lava_callback - INFO - Starting async job submit for node 68d42b2fa5c2c05277bc7e05 2025-09-26 17:32:49,574 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:32:49,587 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d42b2fa5c2c05277bc7e05 HTTP/1.1" 200 None 2025-09-26 17:32:49,588 - lava_callback - DEBUG - Processing node 68d42b2fa5c2c05277bc7e05: ltp-syscalls 2025-09-26 17:32:49,588 - lava_callback - ERROR - Error processing callback for node 68d42b2fa5c2c05277bc7e05: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 17:32:50,339 - lava_callback - INFO - Received callback for node 68d42b32a5c2c05277bc7fba INFO: 172.21.0.1:57746 - "POST /node/68d42b32a5c2c05277bc7fba HTTP/1.0" 202 Accepted 2025-09-26 17:32:50,345 - lava_callback - INFO - Starting async job submit for node 68d42b32a5c2c05277bc7fba 2025-09-26 17:32:50,346 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:32:50,358 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d42b32a5c2c05277bc7fba HTTP/1.1" 200 None 2025-09-26 17:32:50,359 - lava_callback - DEBUG - Processing node 68d42b32a5c2c05277bc7fba: ltp-syscalls 2025-09-26 17:32:50,359 - lava_callback - ERROR - Error processing callback for node 68d42b32a5c2c05277bc7fba: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 17:32:51,085 - lava_callback - INFO - Received callback for node 68d42b3ca5c2c05277bc7fc3 INFO: 172.21.0.1:57762 - "POST /node/68d42b3ca5c2c05277bc7fc3 HTTP/1.0" 202 Accepted 2025-09-26 17:32:51,091 - lava_callback - INFO - Starting async job submit for node 68d42b3ca5c2c05277bc7fc3 2025-09-26 17:32:51,092 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:32:51,104 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d42b3ca5c2c05277bc7fc3 HTTP/1.1" 200 None 2025-09-26 17:32:51,105 - lava_callback - DEBUG - Processing node 68d42b3ca5c2c05277bc7fc3: ltp-sched 2025-09-26 17:32:51,105 - lava_callback - ERROR - Error processing callback for node 68d42b3ca5c2c05277bc7fc3: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 17:32:51,375 - lava_callback - INFO - Received callback for node 68d6c78b3c94e462c68818aa INFO: 172.21.0.1:57766 - "POST /node/68d6c78b3c94e462c68818aa HTTP/1.0" 202 Accepted 2025-09-26 17:32:51,381 - lava_callback - INFO - Starting async job submit for node 68d6c78b3c94e462c68818aa 2025-09-26 17:32:51,403 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:32:51,415 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6c78b3c94e462c68818aa HTTP/1.1" 200 None 2025-09-26 17:32:51,416 - lava_callback - DEBUG - Processing node 68d6c78b3c94e462c68818aa: baseline-arm64-broonie 2025-09-26 17:32:51,670 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 17:32:51,795 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 17:32:51,795 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-68d6c78b3c94e462c68818aa/log.txt.gz 2025-09-26 17:32:51,802 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 17:32:51,946 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 17:32:51,947 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-68d6c78b3c94e462c68818aa/lava_callback.json.gz 2025-09-26 17:32:51,969 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:32:51,998 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6c78b3c94e462c68818aa HTTP/1.1" 200 None 2025-09-26 17:32:51,999 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:32:52,019 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6b9dd2f83206171a92b48 HTTP/1.1" 200 None 2025-09-26 17:32:52,020 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:32:52,056 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6c78b3c94e462c68818aa HTTP/1.1" 200 None 2025-09-26 17:32:52,062 - lava_callback - INFO - Completed processing callback for node 68d6c78b3c94e462c68818aa INFO: 172.21.0.1:57782 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:32:58,902 - lava_callback - INFO - Received callback for node 68d6c9233c94e462c688196c INFO: 172.21.0.1:57784 - "POST /node/68d6c9233c94e462c688196c HTTP/1.0" 202 Accepted 2025-09-26 17:32:58,910 - lava_callback - INFO - Starting async job submit for node 68d6c9233c94e462c688196c 2025-09-26 17:32:58,952 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:32:58,966 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6c9233c94e462c688196c HTTP/1.1" 200 None 2025-09-26 17:32:58,966 - lava_callback - DEBUG - Processing node 68d6c9233c94e462c688196c: baseline-x86 2025-09-26 17:32:59,459 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 17:32:59,608 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 17:32:59,609 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-68d6c9233c94e462c688196c/log.txt.gz 2025-09-26 17:32:59,621 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 17:32:59,753 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 17:32:59,754 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-68d6c9233c94e462c688196c/lava_callback.json.gz DEBUG: Setup failed for 68d6c9233c94e462c688196c. Transitting job node result: incomplete -> fail 2025-09-26 17:32:59,832 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:32:59,845 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6c9233c94e462c688196c HTTP/1.1" 200 None 2025-09-26 17:32:59,846 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:32:59,867 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6b9f22f83206171a92b66 HTTP/1.1" 200 None 2025-09-26 17:32:59,868 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:32:59,900 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6c9233c94e462c688196c HTTP/1.1" 200 None 2025-09-26 17:32:59,901 - lava_callback - INFO - Completed processing callback for node 68d6c9233c94e462c688196c INFO: 172.21.0.1:56720 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56726 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:33:08,808 - lava_callback - INFO - Received callback for node 68d42b43a5c2c05277bc7fc9 INFO: 172.21.0.1:56732 - "POST /node/68d42b43a5c2c05277bc7fc9 HTTP/1.0" 202 Accepted 2025-09-26 17:33:08,813 - lava_callback - INFO - Starting async job submit for node 68d42b43a5c2c05277bc7fc9 2025-09-26 17:33:08,814 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:33:08,826 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d42b43a5c2c05277bc7fc9 HTTP/1.1" 200 None 2025-09-26 17:33:08,827 - lava_callback - DEBUG - Processing node 68d42b43a5c2c05277bc7fc9: ltp-syscalls-ipc 2025-09-26 17:33:08,827 - lava_callback - ERROR - Error processing callback for node 68d42b43a5c2c05277bc7fc9: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 17:33:08,933 - lava_callback - INFO - Received callback for node 68d435a9a5c2c05277bc8aa3 INFO: 172.21.0.1:56742 - "POST /node/68d435a9a5c2c05277bc8aa3 HTTP/1.0" 202 Accepted 2025-09-26 17:33:08,939 - lava_callback - INFO - Starting async job submit for node 68d435a9a5c2c05277bc8aa3 2025-09-26 17:33:08,960 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:33:08,986 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d435a9a5c2c05277bc8aa3 HTTP/1.1" 200 None 2025-09-26 17:33:08,987 - lava_callback - DEBUG - Processing node 68d435a9a5c2c05277bc8aa3: baseline-arm64 2025-09-26 17:33:09,254 - lava_callback - INFO - Received callback for node 68d435aaa5c2c05277bc8aa6 INFO: 172.21.0.1:56750 - "POST /node/68d435aaa5c2c05277bc8aa6 HTTP/1.0" 202 Accepted 2025-09-26 17:33:09,281 - lava_callback - INFO - Starting async job submit for node 68d435aaa5c2c05277bc8aa6 2025-09-26 17:33:09,313 - lava_callback - ERROR - Error processing callback for node 68d435a9a5c2c05277bc8aa3: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None 2025-09-26 17:33:09,320 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:33:09,347 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d435aaa5c2c05277bc8aa6 HTTP/1.1" 200 None 2025-09-26 17:33:09,348 - lava_callback - DEBUG - Processing node 68d435aaa5c2c05277bc8aa6: baseline-arm64 2025-09-26 17:33:09,570 - lava_callback - INFO - Received callback for node 68d42b4aa5c2c05277bc7fcf INFO: 172.21.0.1:57134 - "POST /node/68d42b4aa5c2c05277bc7fcf HTTP/1.0" 202 Accepted 2025-09-26 17:33:09,591 - lava_callback - INFO - Starting async job submit for node 68d42b4aa5c2c05277bc7fcf 2025-09-26 17:33:09,592 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:33:09,657 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d42b4aa5c2c05277bc7fcf HTTP/1.1" 200 None 2025-09-26 17:33:09,667 - lava_callback - DEBUG - Processing node 68d42b4aa5c2c05277bc7fcf: ltp-syscalls-ipc 2025-09-26 17:33:09,667 - lava_callback - ERROR - Error processing callback for node 68d42b4aa5c2c05277bc7fcf: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 17:33:09,681 - lava_callback - ERROR - Error processing callback for node 68d435aaa5c2c05277bc8aa6: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None 2025-09-26 17:33:10,444 - lava_callback - INFO - Received callback for node 68d42b4ca5c2c05277bc7fd2 INFO: 172.21.0.1:57142 - "POST /node/68d42b4ca5c2c05277bc7fd2 HTTP/1.0" 202 Accepted 2025-09-26 17:33:10,450 - lava_callback - INFO - Starting async job submit for node 68d42b4ca5c2c05277bc7fd2 2025-09-26 17:33:10,451 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:33:10,467 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d42b4ca5c2c05277bc7fd2 HTTP/1.1" 200 None 2025-09-26 17:33:10,468 - lava_callback - DEBUG - Processing node 68d42b4ca5c2c05277bc7fd2: ltp-smoketest 2025-09-26 17:33:10,468 - lava_callback - ERROR - Error processing callback for node 68d42b4ca5c2c05277bc7fd2: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 17:33:11,194 - lava_callback - INFO - Received callback for node 68d42b4fa5c2c05277bc7fd5 INFO: 172.21.0.1:57156 - "POST /node/68d42b4fa5c2c05277bc7fd5 HTTP/1.0" 202 Accepted 2025-09-26 17:33:11,199 - lava_callback - INFO - Starting async job submit for node 68d42b4fa5c2c05277bc7fd5 2025-09-26 17:33:11,200 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:33:11,214 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d42b4fa5c2c05277bc7fd5 HTTP/1.1" 200 None 2025-09-26 17:33:11,214 - lava_callback - DEBUG - Processing node 68d42b4fa5c2c05277bc7fd5: baseline-arm-broonie 2025-09-26 17:33:11,215 - lava_callback - ERROR - Error processing callback for node 68d42b4fa5c2c05277bc7fd5: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 17:33:11,962 - lava_callback - INFO - Received callback for node 68d42b54a5c2c05277bc7fd8 INFO: 172.21.0.1:57160 - "POST /node/68d42b54a5c2c05277bc7fd8 HTTP/1.0" 202 Accepted 2025-09-26 17:33:11,968 - lava_callback - INFO - Starting async job submit for node 68d42b54a5c2c05277bc7fd8 2025-09-26 17:33:11,969 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:33:11,982 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d42b54a5c2c05277bc7fd8 HTTP/1.1" 200 None 2025-09-26 17:33:11,982 - lava_callback - DEBUG - Processing node 68d42b54a5c2c05277bc7fd8: ltp-timers 2025-09-26 17:33:11,982 - lava_callback - ERROR - Error processing callback for node 68d42b54a5c2c05277bc7fd8: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:57172 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57184 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:33:24,542 - lava_callback - INFO - Received callback for node 68d435b2a5c2c05277bc8abd INFO: 172.21.0.1:45322 - "POST /node/68d435b2a5c2c05277bc8abd HTTP/1.0" 202 Accepted 2025-09-26 17:33:24,551 - lava_callback - INFO - Starting async job submit for node 68d435b2a5c2c05277bc8abd 2025-09-26 17:33:24,572 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:33:24,584 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d435b2a5c2c05277bc8abd HTTP/1.1" 200 None 2025-09-26 17:33:24,584 - lava_callback - DEBUG - Processing node 68d435b2a5c2c05277bc8abd: baseline-arm64 2025-09-26 17:33:24,883 - lava_callback - ERROR - Error processing callback for node 68d435b2a5c2c05277bc8abd: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:45334 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45344 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:33:28,833 - lava_callback - INFO - Received callback for node 68d42b59a5c2c05277bc7ff3 INFO: 172.21.0.1:45346 - "POST /node/68d42b59a5c2c05277bc7ff3 HTTP/1.0" 202 Accepted 2025-09-26 17:33:28,839 - lava_callback - INFO - Starting async job submit for node 68d42b59a5c2c05277bc7ff3 2025-09-26 17:33:28,840 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:33:28,853 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d42b59a5c2c05277bc7ff3 HTTP/1.1" 200 None 2025-09-26 17:33:28,854 - lava_callback - DEBUG - Processing node 68d42b59a5c2c05277bc7ff3: ltp-syscalls 2025-09-26 17:33:28,854 - lava_callback - ERROR - Error processing callback for node 68d42b59a5c2c05277bc7ff3: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 17:33:29,715 - lava_callback - INFO - Received callback for node 68d42b60a5c2c05277bc7ff9 INFO: 172.21.0.1:37848 - "POST /node/68d42b60a5c2c05277bc7ff9 HTTP/1.0" 202 Accepted 2025-09-26 17:33:29,721 - lava_callback - INFO - Starting async job submit for node 68d42b60a5c2c05277bc7ff9 2025-09-26 17:33:29,722 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:33:29,734 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d42b60a5c2c05277bc7ff9 HTTP/1.1" 200 None 2025-09-26 17:33:29,735 - lava_callback - DEBUG - Processing node 68d42b60a5c2c05277bc7ff9: ltp-timers 2025-09-26 17:33:29,735 - lava_callback - ERROR - Error processing callback for node 68d42b60a5c2c05277bc7ff9: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 17:33:30,465 - lava_callback - INFO - Received callback for node 68d42b64a5c2c05277bc7ffc INFO: 172.21.0.1:37858 - "POST /node/68d42b64a5c2c05277bc7ffc HTTP/1.0" 202 Accepted 2025-09-26 17:33:30,472 - lava_callback - INFO - Starting async job submit for node 68d42b64a5c2c05277bc7ffc 2025-09-26 17:33:30,473 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:33:30,489 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d42b64a5c2c05277bc7ffc HTTP/1.1" 200 None 2025-09-26 17:33:30,489 - lava_callback - DEBUG - Processing node 68d42b64a5c2c05277bc7ffc: ltp-watchqueue 2025-09-26 17:33:30,489 - lava_callback - ERROR - Error processing callback for node 68d42b64a5c2c05277bc7ffc: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 17:33:31,212 - lava_callback - INFO - Received callback for node 68d42b65a5c2c05277bc7fff INFO: 172.21.0.1:37862 - "POST /node/68d42b65a5c2c05277bc7fff HTTP/1.0" 202 Accepted 2025-09-26 17:33:31,218 - lava_callback - INFO - Starting async job submit for node 68d42b65a5c2c05277bc7fff 2025-09-26 17:33:31,218 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:33:31,252 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d42b65a5c2c05277bc7fff HTTP/1.1" 200 None 2025-09-26 17:33:31,252 - lava_callback - DEBUG - Processing node 68d42b65a5c2c05277bc7fff: baseline-arm-broonie 2025-09-26 17:33:31,252 - lava_callback - ERROR - Error processing callback for node 68d42b65a5c2c05277bc7fff: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 17:33:31,958 - lava_callback - INFO - Received callback for node 68d42b6aa5c2c05277bc8002 INFO: 172.21.0.1:37876 - "POST /node/68d42b6aa5c2c05277bc8002 HTTP/1.0" 202 Accepted 2025-09-26 17:33:31,964 - lava_callback - INFO - Starting async job submit for node 68d42b6aa5c2c05277bc8002 2025-09-26 17:33:31,965 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:33:31,978 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d42b6aa5c2c05277bc8002 HTTP/1.1" 200 None 2025-09-26 17:33:31,979 - lava_callback - DEBUG - Processing node 68d42b6aa5c2c05277bc8002: ltp-syscalls-ipc 2025-09-26 17:33:31,979 - lava_callback - ERROR - Error processing callback for node 68d42b6aa5c2c05277bc8002: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:37886 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:33:40,440 - lava_callback - INFO - Received callback for node 68d6cde53c94e462c6881d32 INFO: 172.21.0.1:58896 - "POST /node/68d6cde53c94e462c6881d32 HTTP/1.0" 202 Accepted 2025-09-26 17:33:40,446 - lava_callback - INFO - Starting async job submit for node 68d6cde53c94e462c6881d32 2025-09-26 17:33:40,465 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:33:40,480 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6cde53c94e462c6881d32 HTTP/1.1" 200 None 2025-09-26 17:33:40,480 - lava_callback - DEBUG - Processing node 68d6cde53c94e462c6881d32: baseline-x86 2025-09-26 17:33:40,745 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 17:33:40,888 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 17:33:40,889 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-68d6cde53c94e462c6881d32/log.txt.gz 2025-09-26 17:33:40,898 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 17:33:41,052 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 17:33:41,052 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-68d6cde53c94e462c6881d32/lava_callback.json.gz 2025-09-26 17:33:41,071 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:33:41,102 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6cde53c94e462c6881d32 HTTP/1.1" 200 None 2025-09-26 17:33:41,103 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:33:41,115 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6bda33c94e462c6881504 HTTP/1.1" 200 None 2025-09-26 17:33:41,116 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:33:41,141 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6cde53c94e462c6881d32 HTTP/1.1" 200 None 2025-09-26 17:33:41,142 - lava_callback - INFO - Completed processing callback for node 68d6cde53c94e462c6881d32 INFO: 172.21.0.1:58906 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:33:48,794 - lava_callback - INFO - Received callback for node 68d42b6ba5c2c05277bc8005 INFO: 172.21.0.1:58922 - "POST /node/68d42b6ba5c2c05277bc8005 HTTP/1.0" 202 Accepted 2025-09-26 17:33:48,799 - lava_callback - INFO - Starting async job submit for node 68d42b6ba5c2c05277bc8005 2025-09-26 17:33:48,800 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:33:48,814 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d42b6ba5c2c05277bc8005 HTTP/1.1" 200 None 2025-09-26 17:33:48,815 - lava_callback - DEBUG - Processing node 68d42b6ba5c2c05277bc8005: baseline-arm-broonie 2025-09-26 17:33:48,815 - lava_callback - ERROR - Error processing callback for node 68d42b6ba5c2c05277bc8005: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 17:33:49,578 - lava_callback - INFO - Received callback for node 68d42b70a5c2c05277bc8008 INFO: 172.21.0.1:40290 - "POST /node/68d42b70a5c2c05277bc8008 HTTP/1.0" 202 Accepted 2025-09-26 17:33:49,584 - lava_callback - INFO - Starting async job submit for node 68d42b70a5c2c05277bc8008 2025-09-26 17:33:49,585 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:33:49,598 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d42b70a5c2c05277bc8008 HTTP/1.1" 200 None 2025-09-26 17:33:49,598 - lava_callback - DEBUG - Processing node 68d42b70a5c2c05277bc8008: ltp-watchqueue 2025-09-26 17:33:49,598 - lava_callback - ERROR - Error processing callback for node 68d42b70a5c2c05277bc8008: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 17:33:50,339 - lava_callback - INFO - Received callback for node 68d42b71a5c2c05277bc800b INFO: 172.21.0.1:40292 - "POST /node/68d42b71a5c2c05277bc800b HTTP/1.0" 202 Accepted 2025-09-26 17:33:50,345 - lava_callback - INFO - Starting async job submit for node 68d42b71a5c2c05277bc800b 2025-09-26 17:33:50,346 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:33:50,360 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d42b71a5c2c05277bc800b HTTP/1.1" 200 None 2025-09-26 17:33:50,360 - lava_callback - DEBUG - Processing node 68d42b71a5c2c05277bc800b: ltp-timers 2025-09-26 17:33:50,361 - lava_callback - ERROR - Error processing callback for node 68d42b71a5c2c05277bc800b: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 17:33:51,099 - lava_callback - INFO - Received callback for node 68d42b75a5c2c05277bc8012 INFO: 172.21.0.1:40308 - "POST /node/68d42b75a5c2c05277bc8012 HTTP/1.0" 202 Accepted 2025-09-26 17:33:51,105 - lava_callback - INFO - Starting async job submit for node 68d42b75a5c2c05277bc8012 2025-09-26 17:33:51,106 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:33:51,119 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d42b75a5c2c05277bc8012 HTTP/1.1" 200 None 2025-09-26 17:33:51,119 - lava_callback - DEBUG - Processing node 68d42b75a5c2c05277bc8012: ltp-watchqueue 2025-09-26 17:33:51,119 - lava_callback - ERROR - Error processing callback for node 68d42b75a5c2c05277bc8012: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:40324 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43144 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:34:08,806 - lava_callback - INFO - Received callback for node 68d42b83a5c2c05277bc8024 INFO: 172.21.0.1:43152 - "POST /node/68d42b83a5c2c05277bc8024 HTTP/1.0" 202 Accepted 2025-09-26 17:34:08,811 - lava_callback - INFO - Starting async job submit for node 68d42b83a5c2c05277bc8024 2025-09-26 17:34:08,812 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:34:08,835 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d42b83a5c2c05277bc8024 HTTP/1.1" 200 None 2025-09-26 17:34:08,836 - lava_callback - DEBUG - Processing node 68d42b83a5c2c05277bc8024: baseline-arm-broonie 2025-09-26 17:34:08,836 - lava_callback - ERROR - Error processing callback for node 68d42b83a5c2c05277bc8024: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:43156 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:34:09,569 - lava_callback - INFO - Received callback for node 68d42b85a5c2c05277bc8027 INFO: 172.21.0.1:48670 - "POST /node/68d42b85a5c2c05277bc8027 HTTP/1.0" 202 Accepted 2025-09-26 17:34:09,575 - lava_callback - INFO - Starting async job submit for node 68d42b85a5c2c05277bc8027 2025-09-26 17:34:09,575 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:34:09,613 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d42b85a5c2c05277bc8027 HTTP/1.1" 200 None 2025-09-26 17:34:09,614 - lava_callback - DEBUG - Processing node 68d42b85a5c2c05277bc8027: baseline-arm-broonie 2025-09-26 17:34:09,614 - lava_callback - ERROR - Error processing callback for node 68d42b85a5c2c05277bc8027: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 17:34:10,314 - lava_callback - INFO - Received callback for node 68d42b8aa5c2c05277bc802d INFO: 172.21.0.1:48684 - "POST /node/68d42b8aa5c2c05277bc802d HTTP/1.0" 202 Accepted 2025-09-26 17:34:10,320 - lava_callback - INFO - Starting async job submit for node 68d42b8aa5c2c05277bc802d 2025-09-26 17:34:10,321 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:34:10,333 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d42b8aa5c2c05277bc802d HTTP/1.1" 200 None 2025-09-26 17:34:10,333 - lava_callback - DEBUG - Processing node 68d42b8aa5c2c05277bc802d: baseline-arm-broonie 2025-09-26 17:34:10,333 - lava_callback - ERROR - Error processing callback for node 68d42b8aa5c2c05277bc802d: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 17:34:15,096 - lava_callback - INFO - Received callback for node 68d435b3a5c2c05277bc8ac0 INFO: 172.21.0.1:48692 - "POST /node/68d435b3a5c2c05277bc8ac0 HTTP/1.0" 202 Accepted 2025-09-26 17:34:15,102 - lava_callback - INFO - Starting async job submit for node 68d435b3a5c2c05277bc8ac0 2025-09-26 17:34:15,123 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:34:15,144 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d435b3a5c2c05277bc8ac0 HTTP/1.1" 200 None 2025-09-26 17:34:15,144 - lava_callback - DEBUG - Processing node 68d435b3a5c2c05277bc8ac0: baseline-arm64 INFO: 172.21.0.1:48694 - "GET /api/metrics HTTP/1.0" 200 OK 2025-09-26 17:34:15,463 - lava_callback - ERROR - Error processing callback for node 68d435b3a5c2c05277bc8ac0: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:48696 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:34:22,210 - lava_callback - INFO - Received callback for node 68d435b6a5c2c05277bc8ac5 INFO: 172.21.0.1:54398 - "POST /node/68d435b6a5c2c05277bc8ac5 HTTP/1.0" 202 Accepted 2025-09-26 17:34:22,216 - lava_callback - INFO - Starting async job submit for node 68d435b6a5c2c05277bc8ac5 2025-09-26 17:34:22,236 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:34:22,251 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d435b6a5c2c05277bc8ac5 HTTP/1.1" 200 None 2025-09-26 17:34:22,251 - lava_callback - DEBUG - Processing node 68d435b6a5c2c05277bc8ac5: baseline-arm64 2025-09-26 17:34:22,547 - lava_callback - ERROR - Error processing callback for node 68d435b6a5c2c05277bc8ac5: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:54410 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54412 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48884 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:34:44,726 - lava_callback - INFO - Received callback for node 68d48c7ea5c2c05277bce601 INFO: 172.21.0.1:52960 - "POST /node/68d48c7ea5c2c05277bce601 HTTP/1.0" 202 Accepted 2025-09-26 17:34:44,732 - lava_callback - INFO - Starting async job submit for node 68d48c7ea5c2c05277bce601 2025-09-26 17:34:44,752 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:34:44,764 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d48c7ea5c2c05277bce601 HTTP/1.1" 200 None 2025-09-26 17:34:44,765 - lava_callback - DEBUG - Processing node 68d48c7ea5c2c05277bce601: baseline-arm 2025-09-26 17:34:44,989 - lava_callback - ERROR - Error processing callback for node 68d48c7ea5c2c05277bce601: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None 2025-09-26 17:34:45,947 - lava_callback - INFO - Received callback for node 68d435b7a5c2c05277bc8ac8 INFO: 172.21.0.1:52970 - "POST /node/68d435b7a5c2c05277bc8ac8 HTTP/1.0" 202 Accepted 2025-09-26 17:34:45,953 - lava_callback - INFO - Starting async job submit for node 68d435b7a5c2c05277bc8ac8 2025-09-26 17:34:45,974 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:34:45,986 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d435b7a5c2c05277bc8ac8 HTTP/1.1" 200 None 2025-09-26 17:34:45,987 - lava_callback - DEBUG - Processing node 68d435b7a5c2c05277bc8ac8: baseline-arm64 2025-09-26 17:34:46,313 - lava_callback - ERROR - Error processing callback for node 68d435b7a5c2c05277bc8ac8: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:52984 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:34:48,888 - lava_callback - INFO - Received callback for node 68d42bb1a5c2c05277bc807a INFO: 172.21.0.1:52986 - "POST /node/68d42bb1a5c2c05277bc807a HTTP/1.0" 202 Accepted 2025-09-26 17:34:48,894 - lava_callback - INFO - Starting async job submit for node 68d42bb1a5c2c05277bc807a 2025-09-26 17:34:48,895 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:34:48,917 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d42bb1a5c2c05277bc807a HTTP/1.1" 200 None 2025-09-26 17:34:48,917 - lava_callback - DEBUG - Processing node 68d42bb1a5c2c05277bc807a: ltp-cap-bounds 2025-09-26 17:34:48,917 - lava_callback - ERROR - Error processing callback for node 68d42bb1a5c2c05277bc807a: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 17:34:49,664 - lava_callback - INFO - Received callback for node 68d42bb3a5c2c05277bc807d INFO: 172.21.0.1:57388 - "POST /node/68d42bb3a5c2c05277bc807d HTTP/1.0" 202 Accepted 2025-09-26 17:34:49,670 - lava_callback - INFO - Starting async job submit for node 68d42bb3a5c2c05277bc807d 2025-09-26 17:34:49,671 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:34:49,687 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d42bb3a5c2c05277bc807d HTTP/1.1" 200 None 2025-09-26 17:34:49,688 - lava_callback - DEBUG - Processing node 68d42bb3a5c2c05277bc807d: ltp-cap-bounds 2025-09-26 17:34:49,688 - lava_callback - ERROR - Error processing callback for node 68d42bb3a5c2c05277bc807d: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 17:34:50,517 - lava_callback - INFO - Received callback for node 68d42bb6a5c2c05277bc8080 INFO: 172.21.0.1:57398 - "POST /node/68d42bb6a5c2c05277bc8080 HTTP/1.0" 202 Accepted 2025-09-26 17:34:50,523 - lava_callback - INFO - Starting async job submit for node 68d42bb6a5c2c05277bc8080 2025-09-26 17:34:50,524 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:34:50,536 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d42bb6a5c2c05277bc8080 HTTP/1.1" 200 None 2025-09-26 17:34:50,536 - lava_callback - DEBUG - Processing node 68d42bb6a5c2c05277bc8080: ltp-containers 2025-09-26 17:34:50,536 - lava_callback - ERROR - Error processing callback for node 68d42bb6a5c2c05277bc8080: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 17:34:51,359 - lava_callback - INFO - Received callback for node 68d42bbaa5c2c05277bc8086 INFO: 172.21.0.1:57404 - "POST /node/68d42bbaa5c2c05277bc8086 HTTP/1.0" 202 Accepted 2025-09-26 17:34:51,365 - lava_callback - INFO - Starting async job submit for node 68d42bbaa5c2c05277bc8086 2025-09-26 17:34:51,365 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:34:51,389 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d42bbaa5c2c05277bc8086 HTTP/1.1" 200 None 2025-09-26 17:34:51,389 - lava_callback - DEBUG - Processing node 68d42bbaa5c2c05277bc8086: ltp-containers 2025-09-26 17:34:51,389 - lava_callback - ERROR - Error processing callback for node 68d42bbaa5c2c05277bc8086: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 17:34:51,477 - lava_callback - INFO - Received callback for node 68d42a60a5c2c05277bc7900 INFO: 172.21.0.1:57420 - "POST /node/68d42a60a5c2c05277bc7900 HTTP/1.0" 202 Accepted 2025-09-26 17:34:51,484 - lava_callback - INFO - Starting async job submit for node 68d42a60a5c2c05277bc7900 2025-09-26 17:34:51,585 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:34:51,602 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d42a60a5c2c05277bc7900 HTTP/1.1" 200 None 2025-09-26 17:34:51,602 - lava_callback - DEBUG - Processing node 68d42a60a5c2c05277bc7900: ltp-containers 2025-09-26 17:34:52,381 - lava_callback - ERROR - Error processing callback for node 68d42a60a5c2c05277bc7900: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:57426 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44164 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:35:09,025 - lava_callback - INFO - Received callback for node 68d42bbfa5c2c05277bc808c INFO: 172.21.0.1:44166 - "POST /node/68d42bbfa5c2c05277bc808c HTTP/1.0" 202 Accepted 2025-09-26 17:35:09,031 - lava_callback - INFO - Starting async job submit for node 68d42bbfa5c2c05277bc808c 2025-09-26 17:35:09,032 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:35:09,048 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d42bbfa5c2c05277bc808c HTTP/1.1" 200 None 2025-09-26 17:35:09,049 - lava_callback - DEBUG - Processing node 68d42bbfa5c2c05277bc808c: ltp-controllers 2025-09-26 17:35:09,049 - lava_callback - ERROR - Error processing callback for node 68d42bbfa5c2c05277bc808c: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:44168 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:35:09,778 - lava_callback - INFO - Received callback for node 68d42bc2a5c2c05277bc8093 INFO: 172.21.0.1:44980 - "POST /node/68d42bc2a5c2c05277bc8093 HTTP/1.0" 202 Accepted 2025-09-26 17:35:09,784 - lava_callback - INFO - Starting async job submit for node 68d42bc2a5c2c05277bc8093 2025-09-26 17:35:09,784 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:35:09,797 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d42bc2a5c2c05277bc8093 HTTP/1.1" 200 None 2025-09-26 17:35:09,797 - lava_callback - DEBUG - Processing node 68d42bc2a5c2c05277bc8093: ltp-cve 2025-09-26 17:35:09,797 - lava_callback - ERROR - Error processing callback for node 68d42bc2a5c2c05277bc8093: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 17:35:10,520 - lava_callback - INFO - Received callback for node 68d42bc4a5c2c05277bc8096 INFO: 172.21.0.1:44996 - "POST /node/68d42bc4a5c2c05277bc8096 HTTP/1.0" 202 Accepted 2025-09-26 17:35:10,526 - lava_callback - INFO - Starting async job submit for node 68d42bc4a5c2c05277bc8096 2025-09-26 17:35:10,526 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:35:10,546 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d42bc4a5c2c05277bc8096 HTTP/1.1" 200 None 2025-09-26 17:35:10,547 - lava_callback - DEBUG - Processing node 68d42bc4a5c2c05277bc8096: ltp-cve 2025-09-26 17:35:10,547 - lava_callback - ERROR - Error processing callback for node 68d42bc4a5c2c05277bc8096: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 17:35:11,269 - lava_callback - INFO - Received callback for node 68d42bc7a5c2c05277bc8099 INFO: 172.21.0.1:45004 - "POST /node/68d42bc7a5c2c05277bc8099 HTTP/1.0" 202 Accepted 2025-09-26 17:35:11,275 - lava_callback - INFO - Starting async job submit for node 68d42bc7a5c2c05277bc8099 2025-09-26 17:35:11,275 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:35:11,313 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d42bc7a5c2c05277bc8099 HTTP/1.1" 200 None 2025-09-26 17:35:11,313 - lava_callback - DEBUG - Processing node 68d42bc7a5c2c05277bc8099: ltp-cap-bounds 2025-09-26 17:35:11,314 - lava_callback - ERROR - Error processing callback for node 68d42bc7a5c2c05277bc8099: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 17:35:11,614 - lava_callback - INFO - Received callback for node 68d42adea5c2c05277bc79f2 INFO: 172.21.0.1:45006 - "POST /node/68d42adea5c2c05277bc79f2 HTTP/1.0" 202 Accepted 2025-09-26 17:35:11,621 - lava_callback - INFO - Starting async job submit for node 68d42adea5c2c05277bc79f2 2025-09-26 17:35:11,647 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:35:11,659 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d42adea5c2c05277bc79f2 HTTP/1.1" 200 None 2025-09-26 17:35:11,659 - lava_callback - DEBUG - Processing node 68d42adea5c2c05277bc79f2: ltp-ipc 2025-09-26 17:35:11,979 - lava_callback - ERROR - Error processing callback for node 68d42adea5c2c05277bc79f2: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None 2025-09-26 17:35:13,019 - lava_callback - INFO - Received callback for node 68d6c78e3c94e462c68818b9 INFO: 172.21.0.1:45014 - "POST /node/68d6c78e3c94e462c68818b9 HTTP/1.0" 202 Accepted 2025-09-26 17:35:13,025 - lava_callback - INFO - Starting async job submit for node 68d6c78e3c94e462c68818b9 2025-09-26 17:35:13,046 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:35:13,075 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6c78e3c94e462c68818b9 HTTP/1.1" 200 None 2025-09-26 17:35:13,076 - lava_callback - DEBUG - Processing node 68d6c78e3c94e462c68818b9: baseline-arm64-broonie 2025-09-26 17:35:13,368 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 17:35:13,535 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 17:35:13,536 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-68d6c78e3c94e462c68818b9/log.txt.gz 2025-09-26 17:35:13,543 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 17:35:13,689 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 17:35:13,690 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-68d6c78e3c94e462c68818b9/lava_callback.json.gz 2025-09-26 17:35:13,712 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:35:13,725 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6c78e3c94e462c68818b9 HTTP/1.1" 200 None 2025-09-26 17:35:13,726 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:35:13,739 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6b9dd2f83206171a92b48 HTTP/1.1" 200 None 2025-09-26 17:35:13,740 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:35:13,770 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6c78e3c94e462c68818b9 HTTP/1.1" 200 None 2025-09-26 17:35:13,771 - lava_callback - INFO - Completed processing callback for node 68d6c78e3c94e462c68818b9 INFO: 172.21.0.1:45016 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45024 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36172 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36186 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:35:29,045 - lava_callback - INFO - Received callback for node 68d42bd2a5c2c05277bc80a5 INFO: 172.21.0.1:36194 - "POST /node/68d42bd2a5c2c05277bc80a5 HTTP/1.0" 202 Accepted 2025-09-26 17:35:29,051 - lava_callback - INFO - Starting async job submit for node 68d42bd2a5c2c05277bc80a5 2025-09-26 17:35:29,052 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:35:29,066 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d42bd2a5c2c05277bc80a5 HTTP/1.1" 200 None 2025-09-26 17:35:29,066 - lava_callback - DEBUG - Processing node 68d42bd2a5c2c05277bc80a5: ltp-fs-bind 2025-09-26 17:35:29,067 - lava_callback - ERROR - Error processing callback for node 68d42bd2a5c2c05277bc80a5: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 17:35:29,882 - lava_callback - INFO - Received callback for node 68d42bd7a5c2c05277bc80bf INFO: 172.21.0.1:52296 - "POST /node/68d42bd7a5c2c05277bc80bf HTTP/1.0" 202 Accepted 2025-09-26 17:35:29,888 - lava_callback - INFO - Starting async job submit for node 68d42bd7a5c2c05277bc80bf 2025-09-26 17:35:29,889 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:35:29,910 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d42bd7a5c2c05277bc80bf HTTP/1.1" 200 None 2025-09-26 17:35:29,910 - lava_callback - DEBUG - Processing node 68d42bd7a5c2c05277bc80bf: ltp-fs-bind 2025-09-26 17:35:29,910 - lava_callback - ERROR - Error processing callback for node 68d42bd7a5c2c05277bc80bf: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 17:35:30,641 - lava_callback - INFO - Received callback for node 68d42bd8a5c2c05277bc80c2 INFO: 172.21.0.1:52300 - "POST /node/68d42bd8a5c2c05277bc80c2 HTTP/1.0" 202 Accepted 2025-09-26 17:35:30,647 - lava_callback - INFO - Starting async job submit for node 68d42bd8a5c2c05277bc80c2 2025-09-26 17:35:30,648 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:35:30,660 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d42bd8a5c2c05277bc80c2 HTTP/1.1" 200 None 2025-09-26 17:35:30,661 - lava_callback - DEBUG - Processing node 68d42bd8a5c2c05277bc80c2: ltp-controllers 2025-09-26 17:35:30,661 - lava_callback - ERROR - Error processing callback for node 68d42bd8a5c2c05277bc80c2: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 17:35:31,377 - lava_callback - INFO - Received callback for node 68d42bdca5c2c05277bc811e INFO: 172.21.0.1:52302 - "POST /node/68d42bdca5c2c05277bc811e HTTP/1.0" 202 Accepted 2025-09-26 17:35:31,383 - lava_callback - INFO - Starting async job submit for node 68d42bdca5c2c05277bc811e 2025-09-26 17:35:31,384 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:35:31,401 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d42bdca5c2c05277bc811e HTTP/1.1" 200 None 2025-09-26 17:35:31,401 - lava_callback - DEBUG - Processing node 68d42bdca5c2c05277bc811e: ltp-fsx 2025-09-26 17:35:31,401 - lava_callback - ERROR - Error processing callback for node 68d42bdca5c2c05277bc811e: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 17:35:32,154 - lava_callback - INFO - Received callback for node 68d42bdea5c2c05277bc8121 INFO: 172.21.0.1:52306 - "POST /node/68d42bdea5c2c05277bc8121 HTTP/1.0" 202 Accepted 2025-09-26 17:35:32,160 - lava_callback - INFO - Starting async job submit for node 68d42bdea5c2c05277bc8121 2025-09-26 17:35:32,161 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:35:32,175 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d42bdea5c2c05277bc8121 HTTP/1.1" 200 None 2025-09-26 17:35:32,175 - lava_callback - DEBUG - Processing node 68d42bdea5c2c05277bc8121: ltp-fsx 2025-09-26 17:35:32,175 - lava_callback - ERROR - Error processing callback for node 68d42bdea5c2c05277bc8121: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 17:35:32,902 - lava_callback - INFO - Received callback for node 68d42be2a5c2c05277bc8124 INFO: 172.21.0.1:52312 - "POST /node/68d42be2a5c2c05277bc8124 HTTP/1.0" 202 Accepted 2025-09-26 17:35:32,908 - lava_callback - INFO - Starting async job submit for node 68d42be2a5c2c05277bc8124 2025-09-26 17:35:32,909 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:35:32,922 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d42be2a5c2c05277bc8124 HTTP/1.1" 200 None 2025-09-26 17:35:32,923 - lava_callback - DEBUG - Processing node 68d42be2a5c2c05277bc8124: ltp-cve 2025-09-26 17:35:32,923 - lava_callback - ERROR - Error processing callback for node 68d42be2a5c2c05277bc8124: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 17:35:36,105 - lava_callback - INFO - Received callback for node 68d6c1263c94e462c6881665 INFO: 172.21.0.1:52322 - "POST /node/68d6c1263c94e462c6881665 HTTP/1.0" 202 Accepted 2025-09-26 17:35:36,113 - lava_callback - INFO - Starting async job submit for node 68d6c1263c94e462c6881665 2025-09-26 17:35:36,155 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:35:36,182 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6c1263c94e462c6881665 HTTP/1.1" 200 None 2025-09-26 17:35:36,182 - lava_callback - DEBUG - Processing node 68d6c1263c94e462c6881665: baseline-arm64 INFO: 172.21.0.1:52328 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:35:37,373 - lava_callback - INFO - Received callback for node 68d435bca5c2c05277bc8ace INFO: 172.21.0.1:52330 - "POST /node/68d435bca5c2c05277bc8ace HTTP/1.0" 202 Accepted 2025-09-26 17:35:37,406 - lava_callback - INFO - Starting async job submit for node 68d435bca5c2c05277bc8ace 2025-09-26 17:35:37,426 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:35:37,427 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 17:35:37,440 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d435bca5c2c05277bc8ace HTTP/1.1" 200 None 2025-09-26 17:35:37,440 - lava_callback - DEBUG - Processing node 68d435bca5c2c05277bc8ace: baseline-arm64 2025-09-26 17:35:37,646 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 17:35:37,708 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6c1263c94e462c6881665/log.txt.gz 2025-09-26 17:35:37,789 - lava_callback - ERROR - Error processing callback for node 68d435bca5c2c05277bc8ace: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None 2025-09-26 17:35:37,790 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 17:35:37,964 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 17:35:37,965 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6c1263c94e462c6881665/lava_callback.json.gz DEBUG: Setup failed for 68d6c1263c94e462c6881665. Transitting job node result: incomplete -> fail 2025-09-26 17:35:38,051 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:35:38,063 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6c1263c94e462c6881665 HTTP/1.1" 200 None 2025-09-26 17:35:38,065 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:35:38,080 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d651ddcca86fb17472aa24 HTTP/1.1" 200 None 2025-09-26 17:35:38,081 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:35:38,110 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6c1263c94e462c6881665 HTTP/1.1" 200 None 2025-09-26 17:35:38,111 - lava_callback - INFO - Completed processing callback for node 68d6c1263c94e462c6881665 INFO: 172.21.0.1:33052 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:35:48,914 - lava_callback - INFO - Received callback for node 68d42be4a5c2c05277bc8127 INFO: 172.21.0.1:33058 - "POST /node/68d42be4a5c2c05277bc8127 HTTP/1.0" 202 Accepted 2025-09-26 17:35:48,920 - lava_callback - INFO - Starting async job submit for node 68d42be4a5c2c05277bc8127 2025-09-26 17:35:48,920 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:35:48,933 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d42be4a5c2c05277bc8127 HTTP/1.1" 200 None 2025-09-26 17:35:48,933 - lava_callback - DEBUG - Processing node 68d42be4a5c2c05277bc8127: ltp-input 2025-09-26 17:35:48,933 - lava_callback - ERROR - Error processing callback for node 68d42be4a5c2c05277bc8127: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 17:35:49,669 - lava_callback - INFO - Received callback for node 68d42be7a5c2c05277bc812e INFO: 172.21.0.1:47360 - "POST /node/68d42be7a5c2c05277bc812e HTTP/1.0" 202 Accepted 2025-09-26 17:35:49,678 - lava_callback - INFO - Starting async job submit for node 68d42be7a5c2c05277bc812e 2025-09-26 17:35:49,683 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:35:49,721 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d42be7a5c2c05277bc812e HTTP/1.1" 200 None 2025-09-26 17:35:49,722 - lava_callback - DEBUG - Processing node 68d42be7a5c2c05277bc812e: ltp-input 2025-09-26 17:35:49,722 - lava_callback - ERROR - Error processing callback for node 68d42be7a5c2c05277bc812e: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 17:35:50,536 - lava_callback - INFO - Received callback for node 68d42beda5c2c05277bc8136 INFO: 172.21.0.1:47370 - "POST /node/68d42beda5c2c05277bc8136 HTTP/1.0" 202 Accepted 2025-09-26 17:35:50,542 - lava_callback - INFO - Starting async job submit for node 68d42beda5c2c05277bc8136 2025-09-26 17:35:50,543 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:35:50,557 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d42beda5c2c05277bc8136 HTTP/1.1" 200 None 2025-09-26 17:35:50,558 - lava_callback - DEBUG - Processing node 68d42beda5c2c05277bc8136: ltp-ipc 2025-09-26 17:35:50,558 - lava_callback - ERROR - Error processing callback for node 68d42beda5c2c05277bc8136: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 17:35:51,275 - lava_callback - INFO - Received callback for node 68d42bf0a5c2c05277bc813c INFO: 172.21.0.1:47382 - "POST /node/68d42bf0a5c2c05277bc813c HTTP/1.0" 202 Accepted 2025-09-26 17:35:51,281 - lava_callback - INFO - Starting async job submit for node 68d42bf0a5c2c05277bc813c 2025-09-26 17:35:51,282 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:35:51,310 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d42bf0a5c2c05277bc813c HTTP/1.1" 200 None 2025-09-26 17:35:51,310 - lava_callback - DEBUG - Processing node 68d42bf0a5c2c05277bc813c: ltp-ipc 2025-09-26 17:35:51,310 - lava_callback - ERROR - Error processing callback for node 68d42bf0a5c2c05277bc813c: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 17:35:52,046 - lava_callback - INFO - Received callback for node 68d42bf1a5c2c05277bc813f INFO: 172.21.0.1:47394 - "POST /node/68d42bf1a5c2c05277bc813f HTTP/1.0" 202 Accepted 2025-09-26 17:35:52,053 - lava_callback - INFO - Starting async job submit for node 68d42bf1a5c2c05277bc813f 2025-09-26 17:35:52,053 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:35:52,103 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d42bf1a5c2c05277bc813f HTTP/1.1" 200 None 2025-09-26 17:35:52,103 - lava_callback - DEBUG - Processing node 68d42bf1a5c2c05277bc813f: ltp-fs-bind 2025-09-26 17:35:52,103 - lava_callback - ERROR - Error processing callback for node 68d42bf1a5c2c05277bc813f: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 17:35:52,875 - lava_callback - INFO - Received callback for node 68d42bf5a5c2c05277bc8142 INFO: 172.21.0.1:47408 - "POST /node/68d42bf5a5c2c05277bc8142 HTTP/1.0" 202 Accepted 2025-09-26 17:35:52,881 - lava_callback - INFO - Starting async job submit for node 68d42bf5a5c2c05277bc8142 2025-09-26 17:35:52,882 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:35:52,894 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d42bf5a5c2c05277bc8142 HTTP/1.1" 200 None 2025-09-26 17:35:52,895 - lava_callback - DEBUG - Processing node 68d42bf5a5c2c05277bc8142: ltp-pty 2025-09-26 17:35:52,895 - lava_callback - ERROR - Error processing callback for node 68d42bf5a5c2c05277bc8142: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 17:35:53,640 - lava_callback - INFO - Received callback for node 68d42bf8a5c2c05277bc8145 INFO: 172.21.0.1:47412 - "POST /node/68d42bf8a5c2c05277bc8145 HTTP/1.0" 202 Accepted 2025-09-26 17:35:53,646 - lava_callback - INFO - Starting async job submit for node 68d42bf8a5c2c05277bc8145 2025-09-26 17:35:53,647 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:35:53,659 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d42bf8a5c2c05277bc8145 HTTP/1.1" 200 None 2025-09-26 17:35:53,659 - lava_callback - DEBUG - Processing node 68d42bf8a5c2c05277bc8145: ltp-pty 2025-09-26 17:35:53,659 - lava_callback - ERROR - Error processing callback for node 68d42bf8a5c2c05277bc8145: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:47426 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:36:02,678 - lava_callback - INFO - Received callback for node 68d435bfa5c2c05277bc8ad1 INFO: 172.21.0.1:52636 - "POST /node/68d435bfa5c2c05277bc8ad1 HTTP/1.0" 202 Accepted 2025-09-26 17:36:02,685 - lava_callback - INFO - Starting async job submit for node 68d435bfa5c2c05277bc8ad1 2025-09-26 17:36:02,707 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:36:02,719 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d435bfa5c2c05277bc8ad1 HTTP/1.1" 200 None 2025-09-26 17:36:02,719 - lava_callback - DEBUG - Processing node 68d435bfa5c2c05277bc8ad1: baseline-arm64 2025-09-26 17:36:03,024 - lava_callback - ERROR - Error processing callback for node 68d435bfa5c2c05277bc8ad1: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:52650 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:36:08,931 - lava_callback - INFO - Received callback for node 68d42bfaa5c2c05277bc8148 INFO: 172.21.0.1:52666 - "POST /node/68d42bfaa5c2c05277bc8148 HTTP/1.0" 202 Accepted 2025-09-26 17:36:08,937 - lava_callback - INFO - Starting async job submit for node 68d42bfaa5c2c05277bc8148 2025-09-26 17:36:08,937 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:36:08,950 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d42bfaa5c2c05277bc8148 HTTP/1.1" 200 None 2025-09-26 17:36:08,950 - lava_callback - DEBUG - Processing node 68d42bfaa5c2c05277bc8148: ltp-fsx 2025-09-26 17:36:08,950 - lava_callback - ERROR - Error processing callback for node 68d42bfaa5c2c05277bc8148: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 17:36:09,673 - lava_callback - INFO - Received callback for node 68d42bfda5c2c05277bc814b INFO: 172.21.0.1:52930 - "POST /node/68d42bfda5c2c05277bc814b HTTP/1.0" 202 Accepted 2025-09-26 17:36:09,679 - lava_callback - INFO - Starting async job submit for node 68d42bfda5c2c05277bc814b 2025-09-26 17:36:09,680 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:36:09,698 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d42bfda5c2c05277bc814b HTTP/1.1" 200 None 2025-09-26 17:36:09,698 - lava_callback - DEBUG - Processing node 68d42bfda5c2c05277bc814b: ltp-input 2025-09-26 17:36:09,698 - lava_callback - ERROR - Error processing callback for node 68d42bfda5c2c05277bc814b: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:52936 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:36:10,482 - lava_callback - INFO - Received callback for node 68d42bffa5c2c05277bc814e INFO: 172.21.0.1:52950 - "POST /node/68d42bffa5c2c05277bc814e HTTP/1.0" 202 Accepted 2025-09-26 17:36:10,488 - lava_callback - INFO - Starting async job submit for node 68d42bffa5c2c05277bc814e 2025-09-26 17:36:10,488 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:36:10,512 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d42bffa5c2c05277bc814e HTTP/1.1" 200 None 2025-09-26 17:36:10,513 - lava_callback - DEBUG - Processing node 68d42bffa5c2c05277bc814e: ltp-sched 2025-09-26 17:36:10,513 - lava_callback - ERROR - Error processing callback for node 68d42bffa5c2c05277bc814e: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 17:36:11,229 - lava_callback - INFO - Received callback for node 68d42c03a5c2c05277bc8151 INFO: 172.21.0.1:52966 - "POST /node/68d42c03a5c2c05277bc8151 HTTP/1.0" 202 Accepted 2025-09-26 17:36:11,236 - lava_callback - INFO - Starting async job submit for node 68d42c03a5c2c05277bc8151 2025-09-26 17:36:11,236 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:36:11,248 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d42c03a5c2c05277bc8151 HTTP/1.1" 200 None 2025-09-26 17:36:11,249 - lava_callback - DEBUG - Processing node 68d42c03a5c2c05277bc8151: ltp-sched 2025-09-26 17:36:11,249 - lava_callback - ERROR - Error processing callback for node 68d42c03a5c2c05277bc8151: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 17:36:11,982 - lava_callback - INFO - Received callback for node 68d42c05a5c2c05277bc8154 INFO: 172.21.0.1:52982 - "POST /node/68d42c05a5c2c05277bc8154 HTTP/1.0" 202 Accepted 2025-09-26 17:36:11,988 - lava_callback - INFO - Starting async job submit for node 68d42c05a5c2c05277bc8154 2025-09-26 17:36:11,989 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:36:12,002 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d42c05a5c2c05277bc8154 HTTP/1.1" 200 None 2025-09-26 17:36:12,003 - lava_callback - DEBUG - Processing node 68d42c05a5c2c05277bc8154: ltp-ipc 2025-09-26 17:36:12,003 - lava_callback - ERROR - Error processing callback for node 68d42c05a5c2c05277bc8154: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 17:36:12,741 - lava_callback - INFO - Received callback for node 68d42c08a5c2c05277bc8157 INFO: 172.21.0.1:52984 - "POST /node/68d42c08a5c2c05277bc8157 HTTP/1.0" 202 Accepted 2025-09-26 17:36:12,747 - lava_callback - INFO - Starting async job submit for node 68d42c08a5c2c05277bc8157 2025-09-26 17:36:12,747 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:36:12,760 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d42c08a5c2c05277bc8157 HTTP/1.1" 200 None 2025-09-26 17:36:12,760 - lava_callback - DEBUG - Processing node 68d42c08a5c2c05277bc8157: ltp-smoketest 2025-09-26 17:36:12,760 - lava_callback - ERROR - Error processing callback for node 68d42c08a5c2c05277bc8157: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 17:36:13,487 - lava_callback - INFO - Received callback for node 68d42c0aa5c2c05277bc815a INFO: 172.21.0.1:52988 - "POST /node/68d42c0aa5c2c05277bc815a HTTP/1.0" 202 Accepted 2025-09-26 17:36:13,494 - lava_callback - INFO - Starting async job submit for node 68d42c0aa5c2c05277bc815a 2025-09-26 17:36:13,494 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:36:13,528 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d42c0aa5c2c05277bc815a HTTP/1.1" 200 None 2025-09-26 17:36:13,528 - lava_callback - DEBUG - Processing node 68d42c0aa5c2c05277bc815a: ltp-smoketest 2025-09-26 17:36:13,528 - lava_callback - ERROR - Error processing callback for node 68d42c0aa5c2c05277bc815a: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:53004 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53016 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:36:19,801 - lava_callback - INFO - Received callback for node 68d6cab33c94e462c6881cb2 INFO: 172.21.0.1:32954 - "POST /node/68d6cab33c94e462c6881cb2 HTTP/1.0" 202 Accepted 2025-09-26 17:36:19,808 - lava_callback - INFO - Starting async job submit for node 68d6cab33c94e462c6881cb2 2025-09-26 17:36:19,838 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:36:19,852 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6cab33c94e462c6881cb2 HTTP/1.1" 200 None 2025-09-26 17:36:19,852 - lava_callback - DEBUG - Processing node 68d6cab33c94e462c6881cb2: baseline-arm 2025-09-26 17:36:20,194 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 17:36:20,304 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 17:36:20,304 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-68d6cab33c94e462c6881cb2/log.txt.gz 2025-09-26 17:36:20,313 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 17:36:20,456 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 17:36:20,457 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-68d6cab33c94e462c6881cb2/lava_callback.json.gz DEBUG: Setup failed for 68d6cab33c94e462c6881cb2. Transitting job node result: incomplete -> fail 2025-09-26 17:36:20,518 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:36:20,530 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6cab33c94e462c6881cb2 HTTP/1.1" 200 None 2025-09-26 17:36:20,531 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:36:20,543 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6bd793c94e462c68814e2 HTTP/1.1" 200 None 2025-09-26 17:36:20,544 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:36:20,571 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6cab33c94e462c6881cb2 HTTP/1.1" 200 None 2025-09-26 17:36:20,572 - lava_callback - INFO - Completed processing callback for node 68d6cab33c94e462c6881cb2 INFO: 172.21.0.1:32966 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32970 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:36:28,981 - lava_callback - INFO - Received callback for node 68d42c0ea5c2c05277bc815d INFO: 172.21.0.1:32986 - "POST /node/68d42c0ea5c2c05277bc815d HTTP/1.0" 202 Accepted 2025-09-26 17:36:28,990 - lava_callback - INFO - Starting async job submit for node 68d42c0ea5c2c05277bc815d 2025-09-26 17:36:28,991 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:36:29,008 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d42c0ea5c2c05277bc815d HTTP/1.1" 200 None 2025-09-26 17:36:29,008 - lava_callback - DEBUG - Processing node 68d42c0ea5c2c05277bc815d: ltp-pty 2025-09-26 17:36:29,008 - lava_callback - ERROR - Error processing callback for node 68d42c0ea5c2c05277bc815d: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 17:36:29,721 - lava_callback - INFO - Received callback for node 68d42c10a5c2c05277bc8164 INFO: 172.21.0.1:46302 - "POST /node/68d42c10a5c2c05277bc8164 HTTP/1.0" 202 Accepted 2025-09-26 17:36:29,727 - lava_callback - INFO - Starting async job submit for node 68d42c10a5c2c05277bc8164 2025-09-26 17:36:29,728 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:36:29,740 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d42c10a5c2c05277bc8164 HTTP/1.1" 200 None 2025-09-26 17:36:29,740 - lava_callback - DEBUG - Processing node 68d42c10a5c2c05277bc8164: ltp-syscalls 2025-09-26 17:36:29,740 - lava_callback - ERROR - Error processing callback for node 68d42c10a5c2c05277bc8164: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 17:36:30,488 - lava_callback - INFO - Received callback for node 68d42c14a5c2c05277bc817b INFO: 172.21.0.1:46316 - "POST /node/68d42c14a5c2c05277bc817b HTTP/1.0" 202 Accepted 2025-09-26 17:36:30,494 - lava_callback - INFO - Starting async job submit for node 68d42c14a5c2c05277bc817b 2025-09-26 17:36:30,495 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:36:30,508 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d42c14a5c2c05277bc817b HTTP/1.1" 200 None 2025-09-26 17:36:30,508 - lava_callback - DEBUG - Processing node 68d42c14a5c2c05277bc817b: ltp-syscalls 2025-09-26 17:36:30,508 - lava_callback - ERROR - Error processing callback for node 68d42c14a5c2c05277bc817b: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 17:36:35,338 - lava_callback - INFO - Received callback for node 68d42c16a5c2c05277bc817e INFO: 172.21.0.1:46322 - "POST /node/68d42c16a5c2c05277bc817e HTTP/1.0" 202 Accepted 2025-09-26 17:36:35,344 - lava_callback - INFO - Starting async job submit for node 68d42c16a5c2c05277bc817e 2025-09-26 17:36:35,344 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:36:35,357 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d42c16a5c2c05277bc817e HTTP/1.1" 200 None 2025-09-26 17:36:35,357 - lava_callback - DEBUG - Processing node 68d42c16a5c2c05277bc817e: ltp-sched 2025-09-26 17:36:35,357 - lava_callback - ERROR - Error processing callback for node 68d42c16a5c2c05277bc817e: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:46338 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:36:37,346 - lava_callback - INFO - Received callback for node 68d42c1aa5c2c05277bc8181 INFO: 172.21.0.1:46354 - "POST /node/68d42c1aa5c2c05277bc8181 HTTP/1.0" 202 Accepted 2025-09-26 17:36:37,352 - lava_callback - INFO - Starting async job submit for node 68d42c1aa5c2c05277bc8181 2025-09-26 17:36:37,352 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:36:37,365 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d42c1aa5c2c05277bc8181 HTTP/1.1" 200 None 2025-09-26 17:36:37,365 - lava_callback - DEBUG - Processing node 68d42c1aa5c2c05277bc8181: ltp-syscalls-ipc 2025-09-26 17:36:37,365 - lava_callback - ERROR - Error processing callback for node 68d42c1aa5c2c05277bc8181: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 17:36:38,097 - lava_callback - INFO - Received callback for node 68d42c1ca5c2c05277bc8184 INFO: 172.21.0.1:46364 - "POST /node/68d42c1ca5c2c05277bc8184 HTTP/1.0" 202 Accepted 2025-09-26 17:36:38,103 - lava_callback - INFO - Starting async job submit for node 68d42c1ca5c2c05277bc8184 2025-09-26 17:36:38,104 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:36:38,116 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d42c1ca5c2c05277bc8184 HTTP/1.1" 200 None 2025-09-26 17:36:38,117 - lava_callback - DEBUG - Processing node 68d42c1ca5c2c05277bc8184: ltp-syscalls-ipc 2025-09-26 17:36:38,117 - lava_callback - ERROR - Error processing callback for node 68d42c1ca5c2c05277bc8184: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 17:36:41,239 - lava_callback - INFO - Received callback for node 68d42c1fa5c2c05277bc8187 INFO: 172.21.0.1:43110 - "POST /node/68d42c1fa5c2c05277bc8187 HTTP/1.0" 202 Accepted 2025-09-26 17:36:41,245 - lava_callback - INFO - Starting async job submit for node 68d42c1fa5c2c05277bc8187 2025-09-26 17:36:41,246 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:36:41,258 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d42c1fa5c2c05277bc8187 HTTP/1.1" 200 None 2025-09-26 17:36:41,259 - lava_callback - DEBUG - Processing node 68d42c1fa5c2c05277bc8187: ltp-smoketest 2025-09-26 17:36:41,259 - lava_callback - ERROR - Error processing callback for node 68d42c1fa5c2c05277bc8187: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 17:36:46,156 - lava_callback - INFO - Received callback for node 68d42c21a5c2c05277bc818a INFO: 172.21.0.1:43118 - "POST /node/68d42c21a5c2c05277bc818a HTTP/1.0" 202 Accepted 2025-09-26 17:36:46,162 - lava_callback - INFO - Starting async job submit for node 68d42c21a5c2c05277bc818a 2025-09-26 17:36:46,162 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:36:46,193 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d42c21a5c2c05277bc818a HTTP/1.1" 200 None 2025-09-26 17:36:46,193 - lava_callback - DEBUG - Processing node 68d42c21a5c2c05277bc818a: ltp-timers 2025-09-26 17:36:46,193 - lava_callback - ERROR - Error processing callback for node 68d42c21a5c2c05277bc818a: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 17:36:46,252 - lava_callback - INFO - Received callback for node 68d48c7fa5c2c05277bce604 INFO: 172.21.0.1:43132 - "POST /node/68d48c7fa5c2c05277bce604 HTTP/1.0" 202 Accepted 2025-09-26 17:36:46,259 - lava_callback - INFO - Starting async job submit for node 68d48c7fa5c2c05277bce604 2025-09-26 17:36:46,279 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:36:46,292 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d48c7fa5c2c05277bce604 HTTP/1.1" 200 None 2025-09-26 17:36:46,293 - lava_callback - DEBUG - Processing node 68d48c7fa5c2c05277bce604: baseline-arm 2025-09-26 17:36:46,532 - lava_callback - ERROR - Error processing callback for node 68d48c7fa5c2c05277bce604: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:43144 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:36:47,600 - lava_callback - INFO - Received callback for node 68d42c25a5c2c05277bc818d INFO: 172.21.0.1:43148 - "POST /node/68d42c25a5c2c05277bc818d HTTP/1.0" 202 Accepted 2025-09-26 17:36:47,606 - lava_callback - INFO - Starting async job submit for node 68d42c25a5c2c05277bc818d 2025-09-26 17:36:47,607 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:36:47,627 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d42c25a5c2c05277bc818d HTTP/1.1" 200 None 2025-09-26 17:36:47,627 - lava_callback - DEBUG - Processing node 68d42c25a5c2c05277bc818d: ltp-syscalls 2025-09-26 17:36:47,627 - lava_callback - ERROR - Error processing callback for node 68d42c25a5c2c05277bc818d: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 17:36:48,421 - lava_callback - INFO - Received callback for node 68d42c27a5c2c05277bc8190 INFO: 172.21.0.1:43158 - "POST /node/68d42c27a5c2c05277bc8190 HTTP/1.0" 202 Accepted 2025-09-26 17:36:48,428 - lava_callback - INFO - Starting async job submit for node 68d42c27a5c2c05277bc8190 2025-09-26 17:36:48,428 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:36:48,454 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d42c27a5c2c05277bc8190 HTTP/1.1" 200 None 2025-09-26 17:36:48,455 - lava_callback - DEBUG - Processing node 68d42c27a5c2c05277bc8190: ltp-timers 2025-09-26 17:36:48,455 - lava_callback - ERROR - Error processing callback for node 68d42c27a5c2c05277bc8190: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 17:36:48,921 - lava_callback - INFO - Received callback for node 68d435c2a5c2c05277bc8ad4 INFO: 172.21.0.1:43160 - "POST /node/68d435c2a5c2c05277bc8ad4 HTTP/1.0" 202 Accepted 2025-09-26 17:36:48,927 - lava_callback - INFO - Starting async job submit for node 68d435c2a5c2c05277bc8ad4 2025-09-26 17:36:48,948 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:36:48,968 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d435c2a5c2c05277bc8ad4 HTTP/1.1" 200 None 2025-09-26 17:36:48,968 - lava_callback - DEBUG - Processing node 68d435c2a5c2c05277bc8ad4: baseline-arm64 2025-09-26 17:36:49,181 - lava_callback - INFO - Received callback for node 68d42c2aa5c2c05277bc8193 INFO: 172.21.0.1:43162 - "POST /node/68d42c2aa5c2c05277bc8193 HTTP/1.0" 202 Accepted 2025-09-26 17:36:49,197 - lava_callback - INFO - Starting async job submit for node 68d42c2aa5c2c05277bc8193 2025-09-26 17:36:49,208 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:36:49,271 - lava_callback - ERROR - Error processing callback for node 68d435c2a5c2c05277bc8ad4: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None 2025-09-26 17:36:49,271 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d42c2aa5c2c05277bc8193 HTTP/1.1" 200 None 2025-09-26 17:36:49,272 - lava_callback - DEBUG - Processing node 68d42c2aa5c2c05277bc8193: ltp-watchqueue 2025-09-26 17:36:49,272 - lava_callback - ERROR - Error processing callback for node 68d42c2aa5c2c05277bc8193: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 17:36:49,940 - lava_callback - INFO - Received callback for node 68d42c2da5c2c05277bc8196 INFO: 172.21.0.1:33054 - "POST /node/68d42c2da5c2c05277bc8196 HTTP/1.0" 202 Accepted 2025-09-26 17:36:49,946 - lava_callback - INFO - Starting async job submit for node 68d42c2da5c2c05277bc8196 2025-09-26 17:36:49,947 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:36:49,963 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d42c2da5c2c05277bc8196 HTTP/1.1" 200 None 2025-09-26 17:36:49,964 - lava_callback - DEBUG - Processing node 68d42c2da5c2c05277bc8196: ltp-syscalls-ipc 2025-09-26 17:36:49,964 - lava_callback - ERROR - Error processing callback for node 68d42c2da5c2c05277bc8196: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:33060 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39836 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56196 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:37:11,200 - lava_callback - INFO - Received callback for node 68d42c34a5c2c05277bc8199 INFO: 172.21.0.1:56202 - "POST /node/68d42c34a5c2c05277bc8199 HTTP/1.0" 202 Accepted 2025-09-26 17:37:11,206 - lava_callback - INFO - Starting async job submit for node 68d42c34a5c2c05277bc8199 2025-09-26 17:37:11,207 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:37:11,219 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d42c34a5c2c05277bc8199 HTTP/1.1" 200 None 2025-09-26 17:37:11,219 - lava_callback - DEBUG - Processing node 68d42c34a5c2c05277bc8199: ltp-watchqueue 2025-09-26 17:37:11,219 - lava_callback - ERROR - Error processing callback for node 68d42c34a5c2c05277bc8199: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 17:37:11,958 - lava_callback - INFO - Received callback for node 68d42c36a5c2c05277bc819c INFO: 172.21.0.1:56208 - "POST /node/68d42c36a5c2c05277bc819c HTTP/1.0" 202 Accepted 2025-09-26 17:37:11,964 - lava_callback - INFO - Starting async job submit for node 68d42c36a5c2c05277bc819c 2025-09-26 17:37:11,965 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:37:11,977 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d42c36a5c2c05277bc819c HTTP/1.1" 200 None 2025-09-26 17:37:11,978 - lava_callback - DEBUG - Processing node 68d42c36a5c2c05277bc819c: ltp-timers 2025-09-26 17:37:11,978 - lava_callback - ERROR - Error processing callback for node 68d42c36a5c2c05277bc819c: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 17:37:12,751 - lava_callback - INFO - Received callback for node 68d42c37a5c2c05277bc819f INFO: 172.21.0.1:56220 - "POST /node/68d42c37a5c2c05277bc819f HTTP/1.0" 202 Accepted 2025-09-26 17:37:12,757 - lava_callback - INFO - Starting async job submit for node 68d42c37a5c2c05277bc819f 2025-09-26 17:37:12,758 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:37:12,770 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d42c37a5c2c05277bc819f HTTP/1.1" 200 None 2025-09-26 17:37:12,770 - lava_callback - DEBUG - Processing node 68d42c37a5c2c05277bc819f: ltp-watchqueue 2025-09-26 17:37:12,770 - lava_callback - ERROR - Error processing callback for node 68d42c37a5c2c05277bc819f: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:56226 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56230 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:37:24,670 - lava_callback - INFO - Received callback for node 68d435c6a5c2c05277bc8ad7 INFO: 172.21.0.1:45884 - "POST /node/68d435c6a5c2c05277bc8ad7 HTTP/1.0" 202 Accepted 2025-09-26 17:37:24,676 - lava_callback - INFO - Starting async job submit for node 68d435c6a5c2c05277bc8ad7 2025-09-26 17:37:24,696 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:37:24,709 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d435c6a5c2c05277bc8ad7 HTTP/1.1" 200 None 2025-09-26 17:37:24,709 - lava_callback - DEBUG - Processing node 68d435c6a5c2c05277bc8ad7: baseline-arm64 2025-09-26 17:37:25,017 - lava_callback - ERROR - Error processing callback for node 68d435c6a5c2c05277bc8ad7: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:45898 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45900 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58264 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49230 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45186 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50852 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54260 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54270 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54284 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:38:17,116 - lava_callback - INFO - Received callback for node 68d435c7a5c2c05277bc8ada INFO: 172.21.0.1:54288 - "POST /node/68d435c7a5c2c05277bc8ada HTTP/1.0" 202 Accepted 2025-09-26 17:38:17,122 - lava_callback - INFO - Starting async job submit for node 68d435c7a5c2c05277bc8ada 2025-09-26 17:38:17,142 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:38:17,177 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d435c7a5c2c05277bc8ada HTTP/1.1" 200 None 2025-09-26 17:38:17,177 - lava_callback - DEBUG - Processing node 68d435c7a5c2c05277bc8ada: baseline-arm64 2025-09-26 17:38:17,478 - lava_callback - ERROR - Error processing callback for node 68d435c7a5c2c05277bc8ada: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:36372 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36388 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44452 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:38:41,074 - lava_callback - INFO - Received callback for node 68d435cca5c2c05277bc8add INFO: 172.21.0.1:35210 - "POST /node/68d435cca5c2c05277bc8add HTTP/1.0" 202 Accepted 2025-09-26 17:38:41,080 - lava_callback - INFO - Starting async job submit for node 68d435cca5c2c05277bc8add 2025-09-26 17:38:41,107 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:38:41,126 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d435cca5c2c05277bc8add HTTP/1.1" 200 None 2025-09-26 17:38:41,126 - lava_callback - DEBUG - Processing node 68d435cca5c2c05277bc8add: baseline-arm64 2025-09-26 17:38:41,420 - lava_callback - ERROR - Error processing callback for node 68d435cca5c2c05277bc8add: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None 2025-09-26 17:38:43,898 - lava_callback - INFO - Received callback for node 68d48c89a5c2c05277bce61d INFO: 172.21.0.1:35224 - "POST /node/68d48c89a5c2c05277bce61d HTTP/1.0" 202 Accepted 2025-09-26 17:38:43,904 - lava_callback - INFO - Starting async job submit for node 68d48c89a5c2c05277bce61d 2025-09-26 17:38:43,943 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:38:43,956 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d48c89a5c2c05277bce61d HTTP/1.1" 200 None 2025-09-26 17:38:43,957 - lava_callback - DEBUG - Processing node 68d48c89a5c2c05277bce61d: baseline-arm 2025-09-26 17:38:44,194 - lava_callback - ERROR - Error processing callback for node 68d48c89a5c2c05277bce61d: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:35230 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48910 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35820 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47686 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47700 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47712 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58932 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58946 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:39:31,171 - lava_callback - INFO - Received callback for node 68d435cca5c2c05277bc8ae0 INFO: 172.21.0.1:38770 - "POST /node/68d435cca5c2c05277bc8ae0 HTTP/1.0" 202 Accepted 2025-09-26 17:39:31,177 - lava_callback - INFO - Starting async job submit for node 68d435cca5c2c05277bc8ae0 2025-09-26 17:39:31,198 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:39:31,210 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d435cca5c2c05277bc8ae0 HTTP/1.1" 200 None 2025-09-26 17:39:31,210 - lava_callback - DEBUG - Processing node 68d435cca5c2c05277bc8ae0: baseline-arm64 2025-09-26 17:39:31,502 - lava_callback - ERROR - Error processing callback for node 68d435cca5c2c05277bc8ae0: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:38778 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:39:38,478 - lava_callback - INFO - Received callback for node 68d6adf32f83206171a92817 INFO: 172.21.0.1:38782 - "POST /node/68d6adf32f83206171a92817 HTTP/1.0" 202 Accepted 2025-09-26 17:39:38,485 - lava_callback - INFO - Starting async job submit for node 68d6adf32f83206171a92817 2025-09-26 17:39:38,527 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:39:38,560 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6adf32f83206171a92817 HTTP/1.1" 200 None 2025-09-26 17:39:38,561 - lava_callback - DEBUG - Processing node 68d6adf32f83206171a92817: baseline-arm64 2025-09-26 17:39:39,639 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 17:39:39,788 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 17:39:39,789 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6adf32f83206171a92817/log.txt.gz 2025-09-26 17:39:39,810 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 17:39:40,011 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 17:39:40,012 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6adf32f83206171a92817/lava_callback.json.gz DEBUG: Setup failed for 68d6adf32f83206171a92817. Transitting job node result: incomplete -> fail 2025-09-26 17:39:40,094 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:39:40,106 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6adf32f83206171a92817 HTTP/1.1" 200 None 2025-09-26 17:39:40,107 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:39:40,121 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d651ddcca86fb17472aa24 HTTP/1.1" 200 None 2025-09-26 17:39:40,122 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:39:40,150 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6adf32f83206171a92817 HTTP/1.1" 200 None 2025-09-26 17:39:40,151 - lava_callback - INFO - Completed processing callback for node 68d6adf32f83206171a92817 2025-09-26 17:39:40,178 - lava_callback - INFO - Received callback for node 68d6adf22f83206171a92814 INFO: 172.21.0.1:33858 - "POST /node/68d6adf22f83206171a92814 HTTP/1.0" 202 Accepted 2025-09-26 17:39:40,194 - lava_callback - INFO - Starting async job submit for node 68d6adf22f83206171a92814 2025-09-26 17:39:40,259 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:39:40,272 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6adf22f83206171a92814 HTTP/1.1" 200 None 2025-09-26 17:39:40,273 - lava_callback - DEBUG - Processing node 68d6adf22f83206171a92814: baseline-arm64 2025-09-26 17:39:41,505 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 17:39:41,645 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 17:39:41,645 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6adf22f83206171a92814/log.txt.gz 2025-09-26 17:39:41,667 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 17:39:41,844 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 17:39:41,845 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6adf22f83206171a92814/lava_callback.json.gz DEBUG: Setup failed for 68d6adf22f83206171a92814. Transitting job node result: incomplete -> fail 2025-09-26 17:39:41,939 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:39:41,953 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6adf22f83206171a92814 HTTP/1.1" 200 None 2025-09-26 17:39:41,954 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:39:41,967 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d651ddcca86fb17472aa24 HTTP/1.1" 200 None 2025-09-26 17:39:41,968 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:39:42,000 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6adf22f83206171a92814 HTTP/1.1" 200 None 2025-09-26 17:39:42,001 - lava_callback - INFO - Completed processing callback for node 68d6adf22f83206171a92814 2025-09-26 17:39:43,949 - lava_callback - INFO - Received callback for node 68d6ae102f83206171a92851 INFO: 172.21.0.1:33866 - "POST /node/68d6ae102f83206171a92851 HTTP/1.0" 202 Accepted 2025-09-26 17:39:43,958 - lava_callback - INFO - Starting async job submit for node 68d6ae102f83206171a92851 2025-09-26 17:39:44,010 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:39:44,027 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6ae102f83206171a92851 HTTP/1.1" 200 None 2025-09-26 17:39:44,028 - lava_callback - DEBUG - Processing node 68d6ae102f83206171a92851: baseline-arm64 2025-09-26 17:39:45,019 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 17:39:45,188 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 17:39:45,189 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6ae102f83206171a92851/log.txt.gz 2025-09-26 17:39:45,205 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 17:39:45,346 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 17:39:45,347 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6ae102f83206171a92851/lava_callback.json.gz DEBUG: Setup failed for 68d6ae102f83206171a92851. Transitting job node result: incomplete -> fail 2025-09-26 17:39:45,437 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:39:45,449 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6ae102f83206171a92851 HTTP/1.1" 200 None 2025-09-26 17:39:45,450 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:39:45,463 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d651ddcca86fb17472aa24 HTTP/1.1" 200 None 2025-09-26 17:39:45,464 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:39:45,478 - lava_callback - INFO - Received callback for node 68d6ae0f2f83206171a9284e INFO: 172.21.0.1:33870 - "POST /node/68d6ae0f2f83206171a9284e HTTP/1.0" 202 Accepted 2025-09-26 17:39:45,496 - lava_callback - INFO - Starting async job submit for node 68d6ae0f2f83206171a9284e 2025-09-26 17:39:45,504 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6ae102f83206171a92851 HTTP/1.1" 200 None 2025-09-26 17:39:45,505 - lava_callback - INFO - Completed processing callback for node 68d6ae102f83206171a92851 2025-09-26 17:39:45,570 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:39:45,587 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6ae0f2f83206171a9284e HTTP/1.1" 200 None 2025-09-26 17:39:45,587 - lava_callback - DEBUG - Processing node 68d6ae0f2f83206171a9284e: baseline-arm64 INFO: 172.21.0.1:33878 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:39:47,008 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 17:39:47,155 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 17:39:47,156 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6ae0f2f83206171a9284e/log.txt.gz 2025-09-26 17:39:47,179 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 17:39:47,414 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 17:39:47,415 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6ae0f2f83206171a9284e/lava_callback.json.gz DEBUG: Setup failed for 68d6ae0f2f83206171a9284e. Transitting job node result: incomplete -> fail 2025-09-26 17:39:47,512 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:39:47,523 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6ae0f2f83206171a9284e HTTP/1.1" 200 None 2025-09-26 17:39:47,524 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:39:47,537 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d651ddcca86fb17472aa24 HTTP/1.1" 200 None 2025-09-26 17:39:47,538 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:39:47,575 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6ae0f2f83206171a9284e HTTP/1.1" 200 None 2025-09-26 17:39:47,576 - lava_callback - INFO - Completed processing callback for node 68d6ae0f2f83206171a9284e INFO: 172.21.0.1:40220 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53264 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:40:10,079 - lava_callback - INFO - Received callback for node 68d435cfa5c2c05277bc8ae3 INFO: 172.21.0.1:37746 - "POST /node/68d435cfa5c2c05277bc8ae3 HTTP/1.0" 202 Accepted 2025-09-26 17:40:10,085 - lava_callback - INFO - Starting async job submit for node 68d435cfa5c2c05277bc8ae3 2025-09-26 17:40:10,105 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:40:10,118 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d435cfa5c2c05277bc8ae3 HTTP/1.1" 200 None 2025-09-26 17:40:10,118 - lava_callback - DEBUG - Processing node 68d435cfa5c2c05277bc8ae3: baseline-arm64 2025-09-26 17:40:10,415 - lava_callback - ERROR - Error processing callback for node 68d435cfa5c2c05277bc8ae3: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:37748 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37754 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37760 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45750 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45752 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55382 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43582 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:40:49,129 - lava_callback - INFO - Received callback for node 68d435d2a5c2c05277bc8ae6 INFO: 172.21.0.1:43588 - "POST /node/68d435d2a5c2c05277bc8ae6 HTTP/1.0" 202 Accepted 2025-09-26 17:40:49,135 - lava_callback - INFO - Starting async job submit for node 68d435d2a5c2c05277bc8ae6 2025-09-26 17:40:49,156 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:40:49,168 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d435d2a5c2c05277bc8ae6 HTTP/1.1" 200 None 2025-09-26 17:40:49,168 - lava_callback - DEBUG - Processing node 68d435d2a5c2c05277bc8ae6: baseline-arm64 2025-09-26 17:40:49,467 - lava_callback - ERROR - Error processing callback for node 68d435d2a5c2c05277bc8ae6: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:33776 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44874 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53912 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:41:11,740 - lava_callback - INFO - Received callback for node 68d42bcfa5c2c05277bc80a2 INFO: 172.21.0.1:53916 - "POST /node/68d42bcfa5c2c05277bc80a2 HTTP/1.0" 202 Accepted 2025-09-26 17:41:11,750 - lava_callback - INFO - Starting async job submit for node 68d42bcfa5c2c05277bc80a2 2025-09-26 17:41:11,871 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:41:11,884 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d42bcfa5c2c05277bc80a2 HTTP/1.1" 200 None 2025-09-26 17:41:11,884 - lava_callback - DEBUG - Processing node 68d42bcfa5c2c05277bc80a2: ltp-containers 2025-09-26 17:41:12,673 - lava_callback - ERROR - Error processing callback for node 68d42bcfa5c2c05277bc80a2: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:53930 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53934 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56052 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56056 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48738 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:41:41,693 - lava_callback - INFO - Received callback for node 68d435d6a5c2c05277bc8ae9 INFO: 172.21.0.1:50644 - "POST /node/68d435d6a5c2c05277bc8ae9 HTTP/1.0" 202 Accepted 2025-09-26 17:41:41,699 - lava_callback - INFO - Starting async job submit for node 68d435d6a5c2c05277bc8ae9 2025-09-26 17:41:41,720 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:41:41,733 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d435d6a5c2c05277bc8ae9 HTTP/1.1" 200 None 2025-09-26 17:41:41,733 - lava_callback - DEBUG - Processing node 68d435d6a5c2c05277bc8ae9: baseline-arm64 2025-09-26 17:41:42,024 - lava_callback - ERROR - Error processing callback for node 68d435d6a5c2c05277bc8ae9: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:50648 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:41:47,498 - lava_callback - INFO - Received callback for node 68d48c89a5c2c05277bce620 INFO: 172.21.0.1:50654 - "POST /node/68d48c89a5c2c05277bce620 HTTP/1.0" 202 Accepted 2025-09-26 17:41:47,504 - lava_callback - INFO - Starting async job submit for node 68d48c89a5c2c05277bce620 2025-09-26 17:41:47,525 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:41:47,667 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d48c89a5c2c05277bce620 HTTP/1.1" 200 None 2025-09-26 17:41:47,667 - lava_callback - DEBUG - Processing node 68d48c89a5c2c05277bce620: baseline-arm 2025-09-26 17:41:47,890 - lava_callback - ERROR - Error processing callback for node 68d48c89a5c2c05277bce620: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None 2025-09-26 17:41:52,107 - lava_callback - INFO - Received callback for node 68d435d7a5c2c05277bc8aec INFO: 172.21.0.1:60338 - "POST /node/68d435d7a5c2c05277bc8aec HTTP/1.0" 202 Accepted 2025-09-26 17:41:52,113 - lava_callback - INFO - Starting async job submit for node 68d435d7a5c2c05277bc8aec 2025-09-26 17:41:52,134 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:41:52,169 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d435d7a5c2c05277bc8aec HTTP/1.1" 200 None 2025-09-26 17:41:52,170 - lava_callback - DEBUG - Processing node 68d435d7a5c2c05277bc8aec: baseline-arm64 2025-09-26 17:41:52,486 - lava_callback - ERROR - Error processing callback for node 68d435d7a5c2c05277bc8aec: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:60346 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54942 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46132 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46146 - "GET /api/metrics HTTP/1.0" 200 OK 2025-09-26 17:42:15,563 - lava_callback - INFO - Received callback for node 68d6cc173c94e462c6881d10 INFO: 172.21.0.1:46158 - "POST /node/68d6cc173c94e462c6881d10 HTTP/1.0" 202 Accepted 2025-09-26 17:42:15,570 - lava_callback - INFO - Starting async job submit for node 68d6cc173c94e462c6881d10 2025-09-26 17:42:15,610 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:42:15,624 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6cc173c94e462c6881d10 HTTP/1.1" 200 None 2025-09-26 17:42:15,624 - lava_callback - DEBUG - Processing node 68d6cc173c94e462c6881d10: baseline-arm 2025-09-26 17:42:16,019 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 17:42:16,161 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 17:42:16,161 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-68d6cc173c94e462c6881d10/log.txt.gz 2025-09-26 17:42:16,172 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 17:42:16,330 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 17:42:16,330 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-68d6cc173c94e462c6881d10/lava_callback.json.gz DEBUG: Setup failed for 68d6cc173c94e462c6881d10. Transitting job node result: incomplete -> fail 2025-09-26 17:42:16,413 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:42:16,429 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6cc173c94e462c6881d10 HTTP/1.1" 200 None 2025-09-26 17:42:16,431 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:42:16,442 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6b9da2f83206171a92b45 HTTP/1.1" 200 None 2025-09-26 17:42:16,443 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:42:16,468 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6cc173c94e462c6881d10 HTTP/1.1" 200 None 2025-09-26 17:42:16,469 - lava_callback - INFO - Completed processing callback for node 68d6cc173c94e462c6881d10 INFO: 172.21.0.1:46166 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:42:25,640 - lava_callback - INFO - Received callback for node 68d435baa5c2c05277bc8acb INFO: 172.21.0.1:33010 - "POST /node/68d435baa5c2c05277bc8acb HTTP/1.0" 202 Accepted 2025-09-26 17:42:25,646 - lava_callback - INFO - Starting async job submit for node 68d435baa5c2c05277bc8acb 2025-09-26 17:42:25,676 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:42:25,689 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d435baa5c2c05277bc8acb HTTP/1.1" 200 None 2025-09-26 17:42:25,689 - lava_callback - DEBUG - Processing node 68d435baa5c2c05277bc8acb: baseline-arm64 2025-09-26 17:42:26,367 - lava_callback - ERROR - Error processing callback for node 68d435baa5c2c05277bc8acb: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:33026 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33040 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:42:32,194 - lava_callback - INFO - Received callback for node 68d42976a5c2c05277bc776d INFO: 172.21.0.1:60580 - "POST /node/68d42976a5c2c05277bc776d HTTP/1.0" 202 Accepted 2025-09-26 17:42:32,200 - lava_callback - INFO - Starting async job submit for node 68d42976a5c2c05277bc776d 2025-09-26 17:42:32,230 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:42:32,267 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d42976a5c2c05277bc776d HTTP/1.1" 200 None 2025-09-26 17:42:32,267 - lava_callback - DEBUG - Processing node 68d42976a5c2c05277bc776d: baseline-arm 2025-09-26 17:42:32,527 - lava_callback - ERROR - Error processing callback for node 68d42976a5c2c05277bc776d: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:60582 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41342 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53002 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59422 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:43:06,972 - lava_callback - INFO - Received callback for node 68d435dca5c2c05277bc8aef INFO: 172.21.0.1:59436 - "POST /node/68d435dca5c2c05277bc8aef HTTP/1.0" 202 Accepted 2025-09-26 17:43:06,978 - lava_callback - INFO - Starting async job submit for node 68d435dca5c2c05277bc8aef 2025-09-26 17:43:06,999 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:43:07,011 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d435dca5c2c05277bc8aef HTTP/1.1" 200 None 2025-09-26 17:43:07,011 - lava_callback - DEBUG - Processing node 68d435dca5c2c05277bc8aef: baseline-arm64 2025-09-26 17:43:07,322 - lava_callback - ERROR - Error processing callback for node 68d435dca5c2c05277bc8aef: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None 2025-09-26 17:43:09,058 - lava_callback - INFO - Received callback for node 68d435dfa5c2c05277bc8af2 INFO: 172.21.0.1:59444 - "POST /node/68d435dfa5c2c05277bc8af2 HTTP/1.0" 202 Accepted 2025-09-26 17:43:09,064 - lava_callback - INFO - Starting async job submit for node 68d435dfa5c2c05277bc8af2 2025-09-26 17:43:09,084 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:43:09,100 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d435dfa5c2c05277bc8af2 HTTP/1.1" 200 None 2025-09-26 17:43:09,100 - lava_callback - DEBUG - Processing node 68d435dfa5c2c05277bc8af2: baseline-arm64 2025-09-26 17:43:09,394 - lava_callback - ERROR - Error processing callback for node 68d435dfa5c2c05277bc8af2: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:50988 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51004 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51008 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51508 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51514 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44284 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:43:41,274 - lava_callback - INFO - Received callback for node 68d435e4a5c2c05277bc8af5 INFO: 172.21.0.1:47950 - "POST /node/68d435e4a5c2c05277bc8af5 HTTP/1.0" 202 Accepted 2025-09-26 17:43:41,280 - lava_callback - INFO - Starting async job submit for node 68d435e4a5c2c05277bc8af5 2025-09-26 17:43:41,303 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:43:41,331 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d435e4a5c2c05277bc8af5 HTTP/1.1" 200 None 2025-09-26 17:43:41,331 - lava_callback - DEBUG - Processing node 68d435e4a5c2c05277bc8af5: baseline-arm64 2025-09-26 17:43:41,623 - lava_callback - ERROR - Error processing callback for node 68d435e4a5c2c05277bc8af5: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:47958 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59838 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46974 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36698 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36702 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36706 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:44:21,666 - lava_callback - INFO - Received callback for node 68d435e7a5c2c05277bc8af8 INFO: 172.21.0.1:55966 - "POST /node/68d435e7a5c2c05277bc8af8 HTTP/1.0" 202 Accepted 2025-09-26 17:44:21,676 - lava_callback - INFO - Starting async job submit for node 68d435e7a5c2c05277bc8af8 2025-09-26 17:44:21,753 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:44:21,777 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d435e7a5c2c05277bc8af8 HTTP/1.1" 200 None 2025-09-26 17:44:21,778 - lava_callback - DEBUG - Processing node 68d435e7a5c2c05277bc8af8: baseline-arm64 2025-09-26 17:44:22,169 - lava_callback - ERROR - Error processing callback for node 68d435e7a5c2c05277bc8af8: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None 2025-09-26 17:44:22,370 - lava_callback - INFO - Received callback for node 68d435eda5c2c05277bc8b0d INFO: 172.21.0.1:55972 - "POST /node/68d435eda5c2c05277bc8b0d HTTP/1.0" 202 Accepted 2025-09-26 17:44:22,376 - lava_callback - INFO - Starting async job submit for node 68d435eda5c2c05277bc8b0d 2025-09-26 17:44:22,397 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:44:22,411 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d435eda5c2c05277bc8b0d HTTP/1.1" 200 None 2025-09-26 17:44:22,411 - lava_callback - DEBUG - Processing node 68d435eda5c2c05277bc8b0d: baseline-arm64 2025-09-26 17:44:22,727 - lava_callback - ERROR - Error processing callback for node 68d435eda5c2c05277bc8b0d: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:55976 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55982 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40230 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42974 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35692 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58240 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:45:09,902 - lava_callback - INFO - Received callback for node 68d435efa5c2c05277bc8b10 INFO: 172.21.0.1:43148 - "POST /node/68d435efa5c2c05277bc8b10 HTTP/1.0" 202 Accepted 2025-09-26 17:45:09,909 - lava_callback - INFO - Starting async job submit for node 68d435efa5c2c05277bc8b10 2025-09-26 17:45:09,934 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:45:09,958 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d435efa5c2c05277bc8b10 HTTP/1.1" 200 None 2025-09-26 17:45:09,958 - lava_callback - DEBUG - Processing node 68d435efa5c2c05277bc8b10: baseline-arm64 2025-09-26 17:45:10,329 - lava_callback - ERROR - Error processing callback for node 68d435efa5c2c05277bc8b10: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:43156 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43158 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43162 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57418 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57422 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:45:35,047 - lava_callback - INFO - Received callback for node 68d4381aa5c2c05277bc910b INFO: 172.21.0.1:45046 - "POST /node/68d4381aa5c2c05277bc910b HTTP/1.0" 202 Accepted 2025-09-26 17:45:35,055 - lava_callback - INFO - Starting async job submit for node 68d4381aa5c2c05277bc910b 2025-09-26 17:45:35,075 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:45:35,088 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4381aa5c2c05277bc910b HTTP/1.1" 200 None 2025-09-26 17:45:35,089 - lava_callback - DEBUG - Processing node 68d4381aa5c2c05277bc910b: baseline-arm64 2025-09-26 17:45:36,254 - lava_callback - ERROR - Error processing callback for node 68d4381aa5c2c05277bc910b: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:45054 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:45:38,571 - lava_callback - INFO - Received callback for node 68d6cde13c94e462c6881d2d INFO: 172.21.0.1:45058 - "POST /node/68d6cde13c94e462c6881d2d HTTP/1.0" 202 Accepted 2025-09-26 17:45:38,577 - lava_callback - INFO - Starting async job submit for node 68d6cde13c94e462c6881d2d 2025-09-26 17:45:38,605 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:45:38,623 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6cde13c94e462c6881d2d HTTP/1.1" 200 None 2025-09-26 17:45:38,623 - lava_callback - DEBUG - Processing node 68d6cde13c94e462c6881d2d: baseline-x86 2025-09-26 17:45:38,816 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 17:45:38,990 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 17:45:38,991 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-68d6cde13c94e462c6881d2d/log.txt.gz 2025-09-26 17:45:38,995 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 17:45:39,143 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 17:45:39,144 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-68d6cde13c94e462c6881d2d/lava_callback.json.gz 2025-09-26 17:45:39,169 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:45:39,184 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6cde13c94e462c6881d2d HTTP/1.1" 200 None 2025-09-26 17:45:39,185 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:45:39,197 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6bda33c94e462c6881504 HTTP/1.1" 200 None 2025-09-26 17:45:39,198 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:45:39,227 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6cde13c94e462c6881d2d HTTP/1.1" 200 None 2025-09-26 17:45:39,227 - lava_callback - INFO - Completed processing callback for node 68d6cde13c94e462c6881d2d 2025-09-26 17:45:46,119 - lava_callback - INFO - Received callback for node 68d4381ba5c2c05277bc910f INFO: 172.21.0.1:58598 - "POST /node/68d4381ba5c2c05277bc910f HTTP/1.0" 202 Accepted 2025-09-26 17:45:46,126 - lava_callback - INFO - Starting async job submit for node 68d4381ba5c2c05277bc910f 2025-09-26 17:45:46,147 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:45:46,160 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4381ba5c2c05277bc910f HTTP/1.1" 200 None 2025-09-26 17:45:46,161 - lava_callback - DEBUG - Processing node 68d4381ba5c2c05277bc910f: baseline-arm64 INFO: 172.21.0.1:58600 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:45:46,541 - lava_callback - ERROR - Error processing callback for node 68d4381ba5c2c05277bc910f: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:52100 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49180 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34572 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34586 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34600 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36506 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36508 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53432 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48740 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:46:49,327 - lava_callback - INFO - Received callback for node 68d43827a5c2c05277bc911d INFO: 172.21.0.1:41550 - "POST /node/68d43827a5c2c05277bc911d HTTP/1.0" 202 Accepted 2025-09-26 17:46:49,333 - lava_callback - INFO - Starting async job submit for node 68d43827a5c2c05277bc911d 2025-09-26 17:46:49,372 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:46:49,405 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d43827a5c2c05277bc911d HTTP/1.1" 200 None 2025-09-26 17:46:49,405 - lava_callback - DEBUG - Processing node 68d43827a5c2c05277bc911d: baseline-arm64 2025-09-26 17:46:49,705 - lava_callback - ERROR - Error processing callback for node 68d43827a5c2c05277bc911d: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:41566 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:47:02,406 - lava_callback - INFO - Received callback for node 68d4381fa5c2c05277bc9116 INFO: 172.21.0.1:45870 - "POST /node/68d4381fa5c2c05277bc9116 HTTP/1.0" 202 Accepted 2025-09-26 17:47:02,412 - lava_callback - INFO - Starting async job submit for node 68d4381fa5c2c05277bc9116 2025-09-26 17:47:02,433 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:47:02,447 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4381fa5c2c05277bc9116 HTTP/1.1" 200 None 2025-09-26 17:47:02,447 - lava_callback - DEBUG - Processing node 68d4381fa5c2c05277bc9116: baseline-arm64 2025-09-26 17:47:02,739 - lava_callback - ERROR - Error processing callback for node 68d4381fa5c2c05277bc9116: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:45876 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:47:11,221 - lava_callback - INFO - Received callback for node 68d4381ea5c2c05277bc9113 INFO: 172.21.0.1:41830 - "POST /node/68d4381ea5c2c05277bc9113 HTTP/1.0" 202 Accepted 2025-09-26 17:47:11,228 - lava_callback - INFO - Starting async job submit for node 68d4381ea5c2c05277bc9113 2025-09-26 17:47:11,268 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:47:11,281 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4381ea5c2c05277bc9113 HTTP/1.1" 200 None 2025-09-26 17:47:11,281 - lava_callback - DEBUG - Processing node 68d4381ea5c2c05277bc9113: baseline-arm64 2025-09-26 17:47:11,723 - lava_callback - ERROR - Error processing callback for node 68d4381ea5c2c05277bc9113: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:41846 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:47:14,513 - lava_callback - INFO - Received callback for node 68d6cde43c94e462c6881d30 INFO: 172.21.0.1:41856 - "POST /node/68d6cde43c94e462c6881d30 HTTP/1.0" 202 Accepted 2025-09-26 17:47:14,519 - lava_callback - INFO - Starting async job submit for node 68d6cde43c94e462c6881d30 2025-09-26 17:47:14,563 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:47:14,576 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6cde43c94e462c6881d30 HTTP/1.1" 200 None 2025-09-26 17:47:14,577 - lava_callback - DEBUG - Processing node 68d6cde43c94e462c6881d30: baseline-x86 INFO: 172.21.0.1:41870 - "GET /api/metrics HTTP/1.0" 200 OK 2025-09-26 17:47:15,229 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 17:47:15,382 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 17:47:15,383 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-68d6cde43c94e462c6881d30/log.txt.gz 2025-09-26 17:47:15,400 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 17:47:15,582 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 17:47:15,582 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-68d6cde43c94e462c6881d30/lava_callback.json.gz DEBUG: Setup failed for 68d6cde43c94e462c6881d30. Transitting job node result: incomplete -> fail 2025-09-26 17:47:15,669 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:47:15,684 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6cde43c94e462c6881d30 HTTP/1.1" 200 None 2025-09-26 17:47:15,685 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:47:15,696 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6bda33c94e462c6881504 HTTP/1.1" 200 None 2025-09-26 17:47:15,697 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:47:15,750 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6cde43c94e462c6881d30 HTTP/1.1" 200 None 2025-09-26 17:47:15,751 - lava_callback - INFO - Completed processing callback for node 68d6cde43c94e462c6881d30 INFO: 172.21.0.1:41872 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57412 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57422 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43836 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56490 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:47:53,945 - lava_callback - INFO - Received callback for node 68d43827a5c2c05277bc9120 INFO: 172.21.0.1:51520 - "POST /node/68d43827a5c2c05277bc9120 HTTP/1.0" 202 Accepted 2025-09-26 17:47:53,954 - lava_callback - INFO - Starting async job submit for node 68d43827a5c2c05277bc9120 2025-09-26 17:47:53,972 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:47:53,986 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d43827a5c2c05277bc9120 HTTP/1.1" 200 None 2025-09-26 17:47:53,986 - lava_callback - DEBUG - Processing node 68d43827a5c2c05277bc9120: baseline-arm64 2025-09-26 17:47:55,706 - lava_callback - ERROR - Error processing callback for node 68d43827a5c2c05277bc9120: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:51524 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47980 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40088 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40094 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40106 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:48:22,723 - lava_callback - INFO - Received callback for node 68d4382aa5c2c05277bc9123 INFO: 172.21.0.1:37690 - "POST /node/68d4382aa5c2c05277bc9123 HTTP/1.0" 202 Accepted 2025-09-26 17:48:22,729 - lava_callback - INFO - Starting async job submit for node 68d4382aa5c2c05277bc9123 2025-09-26 17:48:22,750 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:48:22,762 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4382aa5c2c05277bc9123 HTTP/1.1" 200 None 2025-09-26 17:48:22,763 - lava_callback - DEBUG - Processing node 68d4382aa5c2c05277bc9123: baseline-arm64 2025-09-26 17:48:23,069 - lava_callback - ERROR - Error processing callback for node 68d4382aa5c2c05277bc9123: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:37694 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37698 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39874 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:48:44,228 - lava_callback - INFO - Received callback for node 68d4382ba5c2c05277bc9126 INFO: 172.21.0.1:33684 - "POST /node/68d4382ba5c2c05277bc9126 HTTP/1.0" 202 Accepted 2025-09-26 17:48:44,234 - lava_callback - INFO - Starting async job submit for node 68d4382ba5c2c05277bc9126 2025-09-26 17:48:44,255 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:48:44,284 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4382ba5c2c05277bc9126 HTTP/1.1" 200 None 2025-09-26 17:48:44,285 - lava_callback - DEBUG - Processing node 68d4382ba5c2c05277bc9126: baseline-arm64 2025-09-26 17:48:44,606 - lava_callback - ERROR - Error processing callback for node 68d4382ba5c2c05277bc9126: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:33696 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:48:51,731 - lava_callback - INFO - Received callback for node 68d4321da5c2c05277bc85e6 INFO: 172.21.0.1:40712 - "POST /node/68d4321da5c2c05277bc85e6 HTTP/1.0" 202 Accepted 2025-09-26 17:48:51,737 - lava_callback - INFO - Starting async job submit for node 68d4321da5c2c05277bc85e6 2025-09-26 17:48:51,758 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:48:51,771 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4321da5c2c05277bc85e6 HTTP/1.1" 200 None 2025-09-26 17:48:51,771 - lava_callback - DEBUG - Processing node 68d4321da5c2c05277bc85e6: baseline-arm-broonie 2025-09-26 17:48:51,955 - lava_callback - ERROR - Error processing callback for node 68d4321da5c2c05277bc85e6: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:40728 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60446 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:49:11,612 - lava_callback - INFO - Received callback for node 68d43830a5c2c05277bc9129 INFO: 172.21.0.1:43920 - "POST /node/68d43830a5c2c05277bc9129 HTTP/1.0" 202 Accepted 2025-09-26 17:49:11,619 - lava_callback - INFO - Starting async job submit for node 68d43830a5c2c05277bc9129 2025-09-26 17:49:11,637 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:49:11,650 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d43830a5c2c05277bc9129 HTTP/1.1" 200 None 2025-09-26 17:49:11,651 - lava_callback - DEBUG - Processing node 68d43830a5c2c05277bc9129: baseline-arm64 2025-09-26 17:49:12,616 - lava_callback - ERROR - Error processing callback for node 68d43830a5c2c05277bc9129: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:43922 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43926 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43928 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36538 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36542 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33052 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37524 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:49:47,290 - lava_callback - INFO - Received callback for node 68d43831a5c2c05277bc912c INFO: 172.21.0.1:37532 - "POST /node/68d43831a5c2c05277bc912c HTTP/1.0" 202 Accepted 2025-09-26 17:49:47,296 - lava_callback - INFO - Starting async job submit for node 68d43831a5c2c05277bc912c 2025-09-26 17:49:47,316 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:49:47,339 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d43831a5c2c05277bc912c HTTP/1.1" 200 None 2025-09-26 17:49:47,340 - lava_callback - DEBUG - Processing node 68d43831a5c2c05277bc912c: baseline-arm64 2025-09-26 17:49:47,635 - lava_callback - ERROR - Error processing callback for node 68d43831a5c2c05277bc912c: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:54310 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58238 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:50:07,145 - lava_callback - INFO - Received callback for node 68d43834a5c2c05277bc912f INFO: 172.21.0.1:58240 - "POST /node/68d43834a5c2c05277bc912f HTTP/1.0" 202 Accepted 2025-09-26 17:50:07,151 - lava_callback - INFO - Starting async job submit for node 68d43834a5c2c05277bc912f 2025-09-26 17:50:07,172 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:50:07,189 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d43834a5c2c05277bc912f HTTP/1.1" 200 None 2025-09-26 17:50:07,190 - lava_callback - DEBUG - Processing node 68d43834a5c2c05277bc912f: baseline-arm64 2025-09-26 17:50:07,512 - lava_callback - ERROR - Error processing callback for node 68d43834a5c2c05277bc912f: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:38884 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38900 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38908 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48992 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48996 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:50:34,025 - lava_callback - INFO - Received callback for node 68d43836a5c2c05277bc9132 INFO: 172.21.0.1:53432 - "POST /node/68d43836a5c2c05277bc9132 HTTP/1.0" 202 Accepted 2025-09-26 17:50:34,031 - lava_callback - INFO - Starting async job submit for node 68d43836a5c2c05277bc9132 2025-09-26 17:50:34,051 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:50:34,072 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d43836a5c2c05277bc9132 HTTP/1.1" 200 None 2025-09-26 17:50:34,072 - lava_callback - DEBUG - Processing node 68d43836a5c2c05277bc9132: baseline-arm64 2025-09-26 17:50:34,365 - lava_callback - ERROR - Error processing callback for node 68d43836a5c2c05277bc9132: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:53448 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39394 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35882 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:51:01,834 - lava_callback - INFO - Received callback for node 68d4383ba5c2c05277bc9135 INFO: 172.21.0.1:36924 - "POST /node/68d4383ba5c2c05277bc9135 HTTP/1.0" 202 Accepted 2025-09-26 17:51:01,840 - lava_callback - INFO - Starting async job submit for node 68d4383ba5c2c05277bc9135 2025-09-26 17:51:01,860 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:51:01,881 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4383ba5c2c05277bc9135 HTTP/1.1" 200 None 2025-09-26 17:51:01,881 - lava_callback - DEBUG - Processing node 68d4383ba5c2c05277bc9135: baseline-arm64 2025-09-26 17:51:02,198 - lava_callback - ERROR - Error processing callback for node 68d4383ba5c2c05277bc9135: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:36938 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:51:12,423 - lava_callback - INFO - Received callback for node 68d42bbca5c2c05277bc8089 INFO: 172.21.0.1:47278 - "POST /node/68d42bbca5c2c05277bc8089 HTTP/1.0" 202 Accepted 2025-09-26 17:51:12,436 - lava_callback - INFO - Starting async job submit for node 68d42bbca5c2c05277bc8089 2025-09-26 17:51:12,772 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:51:12,785 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d42bbca5c2c05277bc8089 HTTP/1.1" 200 None 2025-09-26 17:51:12,786 - lava_callback - DEBUG - Processing node 68d42bbca5c2c05277bc8089: ltp-controllers INFO: 172.21.0.1:47288 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47296 - "GET /api/metrics HTTP/1.0" 200 OK 2025-09-26 17:51:15,550 - lava_callback - ERROR - Error processing callback for node 68d42bbca5c2c05277bc8089: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:47298 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:51:23,243 - lava_callback - INFO - Received callback for node 68d4383ba5c2c05277bc9138 INFO: 172.21.0.1:57848 - "POST /node/68d4383ba5c2c05277bc9138 HTTP/1.0" 202 Accepted 2025-09-26 17:51:23,249 - lava_callback - INFO - Starting async job submit for node 68d4383ba5c2c05277bc9138 2025-09-26 17:51:23,270 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:51:23,283 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4383ba5c2c05277bc9138 HTTP/1.1" 200 None 2025-09-26 17:51:23,283 - lava_callback - DEBUG - Processing node 68d4383ba5c2c05277bc9138: baseline-arm64 2025-09-26 17:51:23,593 - lava_callback - ERROR - Error processing callback for node 68d4383ba5c2c05277bc9138: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:57862 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57878 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38870 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47372 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36270 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46446 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39670 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39672 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39678 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:52:22,698 - lava_callback - INFO - Received callback for node 68d43841a5c2c05277bc913e INFO: 172.21.0.1:39812 - "POST /node/68d43841a5c2c05277bc913e HTTP/1.0" 202 Accepted 2025-09-26 17:52:22,704 - lava_callback - INFO - Starting async job submit for node 68d43841a5c2c05277bc913e 2025-09-26 17:52:22,725 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:52:22,743 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d43841a5c2c05277bc913e HTTP/1.1" 200 None 2025-09-26 17:52:22,744 - lava_callback - DEBUG - Processing node 68d43841a5c2c05277bc913e: baseline-arm64 2025-09-26 17:52:23,043 - lava_callback - ERROR - Error processing callback for node 68d43841a5c2c05277bc913e: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:39824 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39834 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47696 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:52:41,819 - lava_callback - INFO - Received callback for node 68d43844a5c2c05277bc9141 INFO: 172.21.0.1:42554 - "POST /node/68d43844a5c2c05277bc9141 HTTP/1.0" 202 Accepted 2025-09-26 17:52:41,826 - lava_callback - INFO - Starting async job submit for node 68d43844a5c2c05277bc9141 2025-09-26 17:52:41,846 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:52:41,858 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d43844a5c2c05277bc9141 HTTP/1.1" 200 None 2025-09-26 17:52:41,859 - lava_callback - DEBUG - Processing node 68d43844a5c2c05277bc9141: baseline-arm64 2025-09-26 17:52:42,176 - lava_callback - ERROR - Error processing callback for node 68d43844a5c2c05277bc9141: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:42556 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43812 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50514 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:53:09,682 - lava_callback - INFO - Received callback for node 68d4a980a5c2c05277bcf769 INFO: 172.21.0.1:41084 - "POST /node/68d4a980a5c2c05277bcf769 HTTP/1.0" 202 Accepted 2025-09-26 17:53:09,688 - lava_callback - INFO - Starting async job submit for node 68d4a980a5c2c05277bcf769 2025-09-26 17:53:09,708 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:53:09,733 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4a980a5c2c05277bcf769 HTTP/1.1" 200 None 2025-09-26 17:53:09,734 - lava_callback - DEBUG - Processing node 68d4a980a5c2c05277bcf769: baseline-arm 2025-09-26 17:53:09,956 - lava_callback - ERROR - Error processing callback for node 68d4a980a5c2c05277bcf769: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:41098 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41112 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41116 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:53:19,132 - lava_callback - INFO - Received callback for node 68d43840a5c2c05277bc913b INFO: 172.21.0.1:41124 - "POST /node/68d43840a5c2c05277bc913b HTTP/1.0" 202 Accepted 2025-09-26 17:53:19,161 - lava_callback - INFO - Starting async job submit for node 68d43840a5c2c05277bc913b 2025-09-26 17:53:19,193 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:53:19,216 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d43840a5c2c05277bc913b HTTP/1.1" 200 None 2025-09-26 17:53:19,217 - lava_callback - DEBUG - Processing node 68d43840a5c2c05277bc913b: baseline-arm64 2025-09-26 17:53:26,469 - lava_callback - ERROR - Error processing callback for node 68d43840a5c2c05277bc913b: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:56330 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56342 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:53:32,124 - lava_callback - INFO - Received callback for node 68d4325fa5c2c05277bc863b INFO: 172.21.0.1:47136 - "POST /node/68d4325fa5c2c05277bc863b HTTP/1.0" 202 Accepted 2025-09-26 17:53:32,132 - lava_callback - INFO - Starting async job submit for node 68d4325fa5c2c05277bc863b 2025-09-26 17:53:32,161 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:53:32,173 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4325fa5c2c05277bc863b HTTP/1.1" 200 None 2025-09-26 17:53:32,173 - lava_callback - DEBUG - Processing node 68d4325fa5c2c05277bc863b: ltp-cap-bounds 2025-09-26 17:53:32,495 - lava_callback - ERROR - Error processing callback for node 68d4325fa5c2c05277bc863b: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:47146 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:53:41,934 - lava_callback - INFO - Received callback for node 68d43844a5c2c05277bc9144 INFO: 172.21.0.1:50106 - "POST /node/68d43844a5c2c05277bc9144 HTTP/1.0" 202 Accepted 2025-09-26 17:53:41,940 - lava_callback - INFO - Starting async job submit for node 68d43844a5c2c05277bc9144 2025-09-26 17:53:41,960 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:53:41,972 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d43844a5c2c05277bc9144 HTTP/1.1" 200 None 2025-09-26 17:53:41,973 - lava_callback - DEBUG - Processing node 68d43844a5c2c05277bc9144: baseline-arm64 2025-09-26 17:53:42,289 - lava_callback - ERROR - Error processing callback for node 68d43844a5c2c05277bc9144: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:50112 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33054 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:54:04,705 - lava_callback - INFO - Received callback for node 68d43847a5c2c05277bc9147 INFO: 172.21.0.1:42114 - "POST /node/68d43847a5c2c05277bc9147 HTTP/1.0" 202 Accepted 2025-09-26 17:54:04,712 - lava_callback - INFO - Starting async job submit for node 68d43847a5c2c05277bc9147 2025-09-26 17:54:04,732 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:54:04,771 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d43847a5c2c05277bc9147 HTTP/1.1" 200 None 2025-09-26 17:54:04,771 - lava_callback - DEBUG - Processing node 68d43847a5c2c05277bc9147: baseline-arm64 2025-09-26 17:54:05,069 - lava_callback - ERROR - Error processing callback for node 68d43847a5c2c05277bc9147: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:42130 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45418 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45428 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45436 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51800 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51808 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43612 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51096 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:54:55,933 - lava_callback - INFO - Received callback for node 68d6cf143c94e462c6881d73 INFO: 172.21.0.1:39784 - "POST /node/68d6cf143c94e462c6881d73 HTTP/1.0" 202 Accepted 2025-09-26 17:54:55,939 - lava_callback - INFO - Starting async job submit for node 68d6cf143c94e462c6881d73 2025-09-26 17:54:55,969 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:54:55,981 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6cf143c94e462c6881d73 HTTP/1.1" 200 None 2025-09-26 17:54:55,982 - lava_callback - DEBUG - Processing node 68d6cf143c94e462c6881d73: baseline-arm 2025-09-26 17:54:56,304 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 17:54:56,448 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 17:54:56,448 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-68d6cf143c94e462c6881d73/log.txt.gz 2025-09-26 17:54:56,457 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.21.0.1:39798 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:54:56,601 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 17:54:56,601 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-68d6cf143c94e462c6881d73/lava_callback.json.gz DEBUG: Setup failed for 68d6cf143c94e462c6881d73. Transitting job node result: incomplete -> fail 2025-09-26 17:54:56,662 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:54:56,674 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6cf143c94e462c6881d73 HTTP/1.1" 200 None 2025-09-26 17:54:56,676 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:54:56,690 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6bd793c94e462c68814e2 HTTP/1.1" 200 None 2025-09-26 17:54:56,691 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:54:56,720 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6cf143c94e462c6881d73 HTTP/1.1" 200 None 2025-09-26 17:54:56,721 - lava_callback - INFO - Completed processing callback for node 68d6cf143c94e462c6881d73 2025-09-26 17:55:00,374 - lava_callback - INFO - Received callback for node 68d6ce143c94e462c6881d51 INFO: 172.21.0.1:42156 - "POST /node/68d6ce143c94e462c6881d51 HTTP/1.0" 202 Accepted 2025-09-26 17:55:00,382 - lava_callback - INFO - Starting async job submit for node 68d6ce143c94e462c6881d51 2025-09-26 17:55:00,403 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:55:00,415 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6ce143c94e462c6881d51 HTTP/1.1" 200 None 2025-09-26 17:55:00,415 - lava_callback - DEBUG - Processing node 68d6ce143c94e462c6881d51: blktests-ddp-x86 2025-09-26 17:55:01,172 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 17:55:01,353 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 17:55:01,353 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/blktests-ddp-x86-68d6ce143c94e462c6881d51/log.txt.gz 2025-09-26 17:55:01,372 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 17:55:01,536 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 17:55:01,537 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/blktests-ddp-x86-68d6ce143c94e462c6881d51/lava_callback.json.gz 2025-09-26 17:55:01,579 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:55:01,591 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6ce143c94e462c6881d51 HTTP/1.1" 200 None 2025-09-26 17:55:01,592 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:55:01,620 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6b9f22f83206171a92b66 HTTP/1.1" 200 None 2025-09-26 17:55:01,621 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:55:01,650 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6ce143c94e462c6881d51 HTTP/1.1" 200 None 2025-09-26 17:55:01,650 - lava_callback - INFO - Completed processing callback for node 68d6ce143c94e462c6881d51 2025-09-26 17:55:05,223 - lava_callback - INFO - Received callback for node 68d4384ea5c2c05277bc914d INFO: 172.21.0.1:42162 - "POST /node/68d4384ea5c2c05277bc914d HTTP/1.0" 202 Accepted 2025-09-26 17:55:05,232 - lava_callback - INFO - Starting async job submit for node 68d4384ea5c2c05277bc914d 2025-09-26 17:55:05,257 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:55:05,270 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4384ea5c2c05277bc914d HTTP/1.1" 200 None 2025-09-26 17:55:05,271 - lava_callback - DEBUG - Processing node 68d4384ea5c2c05277bc914d: baseline-arm64 2025-09-26 17:55:05,665 - lava_callback - ERROR - Error processing callback for node 68d4384ea5c2c05277bc914d: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:42172 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:55:09,331 - lava_callback - INFO - Received callback for node 68d4a980a5c2c05277bcf76c INFO: 172.21.0.1:54916 - "POST /node/68d4a980a5c2c05277bcf76c HTTP/1.0" 202 Accepted 2025-09-26 17:55:09,337 - lava_callback - INFO - Starting async job submit for node 68d4a980a5c2c05277bcf76c 2025-09-26 17:55:09,367 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:55:09,380 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4a980a5c2c05277bcf76c HTTP/1.1" 200 None 2025-09-26 17:55:09,380 - lava_callback - DEBUG - Processing node 68d4a980a5c2c05277bcf76c: baseline-arm 2025-09-26 17:55:09,610 - lava_callback - ERROR - Error processing callback for node 68d4a980a5c2c05277bcf76c: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:54920 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54934 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54936 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:55:25,933 - lava_callback - INFO - Received callback for node 68d4384fa5c2c05277bc9150 INFO: 172.21.0.1:44906 - "POST /node/68d4384fa5c2c05277bc9150 HTTP/1.0" 202 Accepted 2025-09-26 17:55:25,939 - lava_callback - INFO - Starting async job submit for node 68d4384fa5c2c05277bc9150 2025-09-26 17:55:25,959 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:55:25,971 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4384fa5c2c05277bc9150 HTTP/1.1" 200 None 2025-09-26 17:55:25,972 - lava_callback - DEBUG - Processing node 68d4384fa5c2c05277bc9150: baseline-arm64 2025-09-26 17:55:26,288 - lava_callback - ERROR - Error processing callback for node 68d4384fa5c2c05277bc9150: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:44922 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44934 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42356 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52874 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42800 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43588 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49976 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49978 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49984 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45190 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45198 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55074 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:56:42,168 - lava_callback - INFO - Received callback for node 68d43855a5c2c05277bc9166 INFO: 172.21.0.1:56054 - "POST /node/68d43855a5c2c05277bc9166 HTTP/1.0" 202 Accepted 2025-09-26 17:56:42,174 - lava_callback - INFO - Starting async job submit for node 68d43855a5c2c05277bc9166 2025-09-26 17:56:42,195 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:56:42,207 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d43855a5c2c05277bc9166 HTTP/1.1" 200 None 2025-09-26 17:56:42,207 - lava_callback - DEBUG - Processing node 68d43855a5c2c05277bc9166: baseline-arm64 2025-09-26 17:56:42,498 - lava_callback - ERROR - Error processing callback for node 68d43855a5c2c05277bc9166: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:56056 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33114 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41428 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:57:09,543 - lava_callback - INFO - Received callback for node 68d4a982a5c2c05277bcf76f INFO: 172.21.0.1:37318 - "POST /node/68d4a982a5c2c05277bcf76f HTTP/1.0" 202 Accepted 2025-09-26 17:57:09,549 - lava_callback - INFO - Starting async job submit for node 68d4a982a5c2c05277bcf76f 2025-09-26 17:57:09,570 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:57:09,583 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4a982a5c2c05277bcf76f HTTP/1.1" 200 None 2025-09-26 17:57:09,583 - lava_callback - DEBUG - Processing node 68d4a982a5c2c05277bcf76f: baseline-arm 2025-09-26 17:57:09,815 - lava_callback - ERROR - Error processing callback for node 68d4a982a5c2c05277bcf76f: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None 2025-09-26 17:57:12,455 - lava_callback - INFO - Received callback for node 68d43267a5c2c05277bc8648 INFO: 172.21.0.1:37334 - "POST /node/68d43267a5c2c05277bc8648 HTTP/1.0" 202 Accepted 2025-09-26 17:57:12,463 - lava_callback - INFO - Starting async job submit for node 68d43267a5c2c05277bc8648 2025-09-26 17:57:12,564 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:57:12,577 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d43267a5c2c05277bc8648 HTTP/1.1" 200 None 2025-09-26 17:57:12,578 - lava_callback - DEBUG - Processing node 68d43267a5c2c05277bc8648: ltp-containers 2025-09-26 17:57:13,366 - lava_callback - ERROR - Error processing callback for node 68d43267a5c2c05277bc8648: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:37342 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37352 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37356 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:57:21,499 - lava_callback - INFO - Received callback for node 68d43853a5c2c05277bc9163 INFO: 172.21.0.1:47350 - "POST /node/68d43853a5c2c05277bc9163 HTTP/1.0" 202 Accepted 2025-09-26 17:57:21,505 - lava_callback - INFO - Starting async job submit for node 68d43853a5c2c05277bc9163 2025-09-26 17:57:21,526 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:57:21,539 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d43853a5c2c05277bc9163 HTTP/1.1" 200 None 2025-09-26 17:57:21,539 - lava_callback - DEBUG - Processing node 68d43853a5c2c05277bc9163: baseline-arm64 2025-09-26 17:57:21,854 - lava_callback - ERROR - Error processing callback for node 68d43853a5c2c05277bc9163: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:47360 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47362 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57358 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46386 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38608 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:58:04,611 - lava_callback - INFO - Received callback for node 68d43b6ba5c2c05277bc951a INFO: 172.21.0.1:48544 - "POST /node/68d43b6ba5c2c05277bc951a HTTP/1.0" 202 Accepted 2025-09-26 17:58:04,622 - lava_callback - INFO - Starting async job submit for node 68d43b6ba5c2c05277bc951a 2025-09-26 17:58:04,658 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:58:04,674 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d43b6ba5c2c05277bc951a HTTP/1.1" 200 None 2025-09-26 17:58:04,675 - lava_callback - DEBUG - Processing node 68d43b6ba5c2c05277bc951a: baseline-arm64 2025-09-26 17:58:05,123 - lava_callback - ERROR - Error processing callback for node 68d43b6ba5c2c05277bc951a: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:48556 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36454 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36462 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36468 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53634 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53646 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32902 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:58:43,844 - lava_callback - INFO - Received callback for node 68d43b6ea5c2c05277bc951d INFO: 172.21.0.1:58668 - "POST /node/68d43b6ea5c2c05277bc951d HTTP/1.0" 202 Accepted 2025-09-26 17:58:43,855 - lava_callback - INFO - Starting async job submit for node 68d43b6ea5c2c05277bc951d 2025-09-26 17:58:43,890 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:58:43,913 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d43b6ea5c2c05277bc951d HTTP/1.1" 200 None 2025-09-26 17:58:43,914 - lava_callback - DEBUG - Processing node 68d43b6ea5c2c05277bc951d: baseline-arm64 2025-09-26 17:58:44,440 - lava_callback - ERROR - Error processing callback for node 68d43b6ea5c2c05277bc951d: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:58684 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41320 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:59:03,426 - lava_callback - INFO - Received callback for node 68d4a982a5c2c05277bcf772 INFO: 172.21.0.1:45332 - "POST /node/68d4a982a5c2c05277bcf772 HTTP/1.0" 202 Accepted 2025-09-26 17:59:03,432 - lava_callback - INFO - Starting async job submit for node 68d4a982a5c2c05277bcf772 2025-09-26 17:59:03,452 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:59:03,465 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4a982a5c2c05277bcf772 HTTP/1.1" 200 None 2025-09-26 17:59:03,466 - lava_callback - DEBUG - Processing node 68d4a982a5c2c05277bcf772: baseline-arm 2025-09-26 17:59:03,686 - lava_callback - ERROR - Error processing callback for node 68d4a982a5c2c05277bcf772: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:45338 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54502 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54510 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54526 - "GET / HTTP/1.0" 200 OK 2025-09-26 17:59:21,924 - lava_callback - INFO - Received callback for node 68d43b75a5c2c05277bc952f INFO: 172.21.0.1:50716 - "POST /node/68d43b75a5c2c05277bc952f HTTP/1.0" 202 Accepted 2025-09-26 17:59:21,930 - lava_callback - INFO - Starting async job submit for node 68d43b75a5c2c05277bc952f 2025-09-26 17:59:21,951 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 17:59:21,964 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d43b75a5c2c05277bc952f HTTP/1.1" 200 None 2025-09-26 17:59:21,964 - lava_callback - DEBUG - Processing node 68d43b75a5c2c05277bc952f: baseline-arm64 2025-09-26 17:59:22,274 - lava_callback - ERROR - Error processing callback for node 68d43b75a5c2c05277bc952f: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:50718 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50734 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34656 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55856 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39568 - "GET / HTTP/1.0" 200 OK 2025-09-26 18:00:02,487 - lava_callback - INFO - Received callback for node 68d43b77a5c2c05277bc9532 INFO: 172.21.0.1:33238 - "POST /node/68d43b77a5c2c05277bc9532 HTTP/1.0" 202 Accepted 2025-09-26 18:00:02,493 - lava_callback - INFO - Starting async job submit for node 68d43b77a5c2c05277bc9532 2025-09-26 18:00:02,514 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:00:02,535 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d43b77a5c2c05277bc9532 HTTP/1.1" 200 None 2025-09-26 18:00:02,535 - lava_callback - DEBUG - Processing node 68d43b77a5c2c05277bc9532: baseline-arm64 2025-09-26 18:00:02,872 - lava_callback - ERROR - Error processing callback for node 68d43b77a5c2c05277bc9532: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None 2025-09-26 18:00:06,391 - lava_callback - INFO - Received callback for node 68d6d3793c94e462c6881db8 INFO: 172.21.0.1:33252 - "POST /node/68d6d3793c94e462c6881db8 HTTP/1.0" 202 Accepted 2025-09-26 18:00:06,404 - lava_callback - INFO - Starting async job submit for node 68d6d3793c94e462c6881db8 2025-09-26 18:00:06,439 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.21.0.1:33256 - "GET / HTTP/1.0" 200 OK 2025-09-26 18:00:06,582 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6d3793c94e462c6881db8 HTTP/1.1" 200 None 2025-09-26 18:00:06,583 - lava_callback - DEBUG - Processing node 68d6d3793c94e462c6881db8: blktests-ddp-x86 2025-09-26 18:00:07,285 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 18:00:07,412 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 18:00:07,413 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/blktests-ddp-x86-68d6d3793c94e462c6881db8/log.txt.gz 2025-09-26 18:00:07,430 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 18:00:07,547 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 18:00:07,548 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/blktests-ddp-x86-68d6d3793c94e462c6881db8/lava_callback.json.gz 2025-09-26 18:00:07,568 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:00:07,585 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6d3793c94e462c6881db8 HTTP/1.1" 200 None 2025-09-26 18:00:07,586 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:00:07,605 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6b9f22f83206171a92b66 HTTP/1.1" 200 None 2025-09-26 18:00:07,606 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:00:07,635 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6d3793c94e462c6881db8 HTTP/1.1" 200 None 2025-09-26 18:00:07,635 - lava_callback - INFO - Completed processing callback for node 68d6d3793c94e462c6881db8 INFO: 172.21.0.1:52986 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52992 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53004 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50846 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50850 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43942 - "GET / HTTP/1.0" 200 OK 2025-09-26 18:00:42,131 - lava_callback - INFO - Received callback for node 68d43b7aa5c2c05277bc9535 INFO: 172.21.0.1:53304 - "POST /node/68d43b7aa5c2c05277bc9535 HTTP/1.0" 202 Accepted 2025-09-26 18:00:42,137 - lava_callback - INFO - Starting async job submit for node 68d43b7aa5c2c05277bc9535 2025-09-26 18:00:42,169 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:00:42,182 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d43b7aa5c2c05277bc9535 HTTP/1.1" 200 None 2025-09-26 18:00:42,183 - lava_callback - DEBUG - Processing node 68d43b7aa5c2c05277bc9535: baseline-arm64 2025-09-26 18:00:42,596 - lava_callback - ERROR - Error processing callback for node 68d43b7aa5c2c05277bc9535: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:53312 - "GET / HTTP/1.0" 200 OK 2025-09-26 18:00:56,111 - lava_callback - INFO - Received callback for node 68d6d4983c94e462c6881e78 INFO: 172.21.0.1:35582 - "POST /node/68d6d4983c94e462c6881e78 HTTP/1.0" 202 Accepted 2025-09-26 18:00:56,118 - lava_callback - INFO - Starting async job submit for node 68d6d4983c94e462c6881e78 2025-09-26 18:00:56,146 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:00:56,158 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6d4983c94e462c6881e78 HTTP/1.1" 200 None 2025-09-26 18:00:56,159 - lava_callback - DEBUG - Processing node 68d6d4983c94e462c6881e78: baseline-arm64-foundriesio 2025-09-26 18:00:56,507 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.21.0.1:35598 - "GET / HTTP/1.0" 200 OK 2025-09-26 18:00:56,646 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 18:00:56,647 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-68d6d4983c94e462c6881e78/log.txt.gz 2025-09-26 18:00:56,657 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 18:00:56,796 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 18:00:56,796 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-68d6d4983c94e462c6881e78/lava_callback.json.gz 2025-09-26 18:00:56,826 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:00:56,847 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6d4983c94e462c6881e78 HTTP/1.1" 200 None 2025-09-26 18:00:56,848 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:00:56,863 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6bd8e3c94e462c68814f1 HTTP/1.1" 200 None 2025-09-26 18:00:56,864 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:00:56,892 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6d4983c94e462c6881e78 HTTP/1.1" 200 None 2025-09-26 18:00:56,892 - lava_callback - INFO - Completed processing callback for node 68d6d4983c94e462c6881e78 2025-09-26 18:00:57,585 - lava_callback - INFO - Received callback for node 68d6d4953c94e462c6881e6c INFO: 172.21.0.1:35614 - "POST /node/68d6d4953c94e462c6881e6c HTTP/1.0" 202 Accepted 2025-09-26 18:00:57,591 - lava_callback - INFO - Starting async job submit for node 68d6d4953c94e462c6881e6c 2025-09-26 18:00:57,610 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:00:57,626 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6d4953c94e462c6881e6c HTTP/1.1" 200 None 2025-09-26 18:00:57,627 - lava_callback - DEBUG - Processing node 68d6d4953c94e462c6881e6c: baseline-arm64-pengutronix 2025-09-26 18:00:57,821 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 18:00:57,993 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 18:00:57,994 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-68d6d4953c94e462c6881e6c/log.txt.gz 2025-09-26 18:00:57,999 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 18:00:58,116 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 18:00:58,116 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-68d6d4953c94e462c6881e6c/lava_callback.json.gz 2025-09-26 18:00:58,136 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:00:58,148 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6d4953c94e462c6881e6c HTTP/1.1" 200 None 2025-09-26 18:00:58,149 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:00:58,160 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6bd8e3c94e462c68814f1 HTTP/1.1" 200 None 2025-09-26 18:00:58,161 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:00:58,193 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6d4953c94e462c6881e6c HTTP/1.1" 200 None 2025-09-26 18:00:58,193 - lava_callback - INFO - Completed processing callback for node 68d6d4953c94e462c6881e6c 2025-09-26 18:01:02,704 - lava_callback - INFO - Received callback for node 68d6d4973c94e462c6881e75 INFO: 172.21.0.1:43574 - "POST /node/68d6d4973c94e462c6881e75 HTTP/1.0" 202 Accepted 2025-09-26 18:01:02,710 - lava_callback - INFO - Starting async job submit for node 68d6d4973c94e462c6881e75 2025-09-26 18:01:02,730 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:01:02,747 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6d4973c94e462c6881e75 HTTP/1.1" 200 None 2025-09-26 18:01:02,748 - lava_callback - DEBUG - Processing node 68d6d4973c94e462c6881e75: baseline-arm64-foundriesio 2025-09-26 18:01:02,947 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 18:01:03,059 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 18:01:03,060 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-68d6d4973c94e462c6881e75/log.txt.gz 2025-09-26 18:01:03,066 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 18:01:03,173 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 18:01:03,174 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-68d6d4973c94e462c6881e75/lava_callback.json.gz 2025-09-26 18:01:03,197 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:01:03,213 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6d4973c94e462c6881e75 HTTP/1.1" 200 None 2025-09-26 18:01:03,214 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:01:03,230 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6bd8e3c94e462c68814f1 HTTP/1.1" 200 None 2025-09-26 18:01:03,231 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:01:03,259 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6d4973c94e462c6881e75 HTTP/1.1" 200 None 2025-09-26 18:01:03,260 - lava_callback - INFO - Completed processing callback for node 68d6d4973c94e462c6881e75 2025-09-26 18:01:06,098 - lava_callback - INFO - Received callback for node 68d6d4973c94e462c6881e72 INFO: 172.21.0.1:43576 - "POST /node/68d6d4973c94e462c6881e72 HTTP/1.0" 202 Accepted 2025-09-26 18:01:06,104 - lava_callback - INFO - Starting async job submit for node 68d6d4973c94e462c6881e72 2025-09-26 18:01:06,123 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:01:06,135 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6d4973c94e462c6881e72 HTTP/1.1" 200 None 2025-09-26 18:01:06,136 - lava_callback - DEBUG - Processing node 68d6d4973c94e462c6881e72: baseline-arm64-pengutronix 2025-09-26 18:01:06,330 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 18:01:06,450 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 18:01:06,451 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-68d6d4973c94e462c6881e72/log.txt.gz 2025-09-26 18:01:06,457 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.21.0.1:43578 - "GET / HTTP/1.0" 200 OK 2025-09-26 18:01:06,630 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 18:01:06,631 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-68d6d4973c94e462c6881e72/lava_callback.json.gz 2025-09-26 18:01:06,651 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:01:06,665 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6d4973c94e462c6881e72 HTTP/1.1" 200 None 2025-09-26 18:01:06,666 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:01:06,678 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6bd8e3c94e462c68814f1 HTTP/1.1" 200 None 2025-09-26 18:01:06,679 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:01:06,709 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6d4973c94e462c6881e72 HTTP/1.1" 200 None 2025-09-26 18:01:06,710 - lava_callback - INFO - Completed processing callback for node 68d6d4973c94e462c6881e72 2025-09-26 18:01:10,311 - lava_callback - INFO - Received callback for node 68d4a9bda5c2c05277bcf7ae INFO: 172.21.0.1:34700 - "POST /node/68d4a9bda5c2c05277bcf7ae HTTP/1.0" 202 Accepted 2025-09-26 18:01:10,318 - lava_callback - INFO - Starting async job submit for node 68d4a9bda5c2c05277bcf7ae 2025-09-26 18:01:10,342 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:01:10,361 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4a9bda5c2c05277bcf7ae HTTP/1.1" 200 None 2025-09-26 18:01:10,362 - lava_callback - DEBUG - Processing node 68d4a9bda5c2c05277bcf7ae: baseline-arm 2025-09-26 18:01:10,427 - lava_callback - INFO - Received callback for node 68d43849a5c2c05277bc914a INFO: 172.21.0.1:34712 - "POST /node/68d43849a5c2c05277bc914a HTTP/1.0" 202 Accepted 2025-09-26 18:01:10,460 - lava_callback - INFO - Starting async job submit for node 68d43849a5c2c05277bc914a 2025-09-26 18:01:10,516 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:01:10,582 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d43849a5c2c05277bc914a HTTP/1.1" 200 None 2025-09-26 18:01:10,593 - lava_callback - DEBUG - Processing node 68d43849a5c2c05277bc914a: baseline-arm64 2025-09-26 18:01:10,710 - lava_callback - ERROR - Error processing callback for node 68d4a9bda5c2c05277bcf7ae: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None 2025-09-26 18:01:11,360 - lava_callback - ERROR - Error processing callback for node 68d43849a5c2c05277bc914a: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None 2025-09-26 18:01:11,771 - lava_callback - INFO - Received callback for node 68d431dba5c2c05277bc857b INFO: 172.21.0.1:34724 - "POST /node/68d431dba5c2c05277bc857b HTTP/1.0" 202 Accepted 2025-09-26 18:01:11,776 - lava_callback - INFO - Starting async job submit for node 68d431dba5c2c05277bc857b 2025-09-26 18:01:11,777 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:01:11,790 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d431dba5c2c05277bc857b HTTP/1.1" 200 None 2025-09-26 18:01:11,790 - lava_callback - DEBUG - Processing node 68d431dba5c2c05277bc857b: baseline-arm 2025-09-26 18:01:11,790 - lava_callback - ERROR - Error processing callback for node 68d431dba5c2c05277bc857b: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:34726 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34728 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34730 - "GET / HTTP/1.0" 200 OK 2025-09-26 18:01:20,290 - lava_callback - INFO - Received callback for node 68d6d4943c94e462c6881e66 INFO: 172.21.0.1:42190 - "POST /node/68d6d4943c94e462c6881e66 HTTP/1.0" 202 Accepted 2025-09-26 18:01:20,296 - lava_callback - INFO - Starting async job submit for node 68d6d4943c94e462c6881e66 2025-09-26 18:01:20,318 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:01:20,357 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6d4943c94e462c6881e66 HTTP/1.1" 200 None 2025-09-26 18:01:20,357 - lava_callback - DEBUG - Processing node 68d6d4943c94e462c6881e66: baseline-arm64-kontron 2025-09-26 18:01:20,634 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 18:01:20,747 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 18:01:20,747 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-kontron-68d6d4943c94e462c6881e66/log.txt.gz 2025-09-26 18:01:20,754 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 18:01:20,861 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 18:01:20,862 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-kontron-68d6d4943c94e462c6881e66/lava_callback.json.gz 2025-09-26 18:01:20,884 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:01:20,905 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6d4943c94e462c6881e66 HTTP/1.1" 200 None 2025-09-26 18:01:20,906 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:01:20,920 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6bd8e3c94e462c68814f1 HTTP/1.1" 200 None 2025-09-26 18:01:20,922 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:01:20,959 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6d4943c94e462c6881e66 HTTP/1.1" 200 None 2025-09-26 18:01:20,960 - lava_callback - INFO - Completed processing callback for node 68d6d4943c94e462c6881e66 2025-09-26 18:01:21,625 - lava_callback - INFO - Received callback for node 68d43b7ca5c2c05277bc9538 INFO: 172.21.0.1:42194 - "POST /node/68d43b7ca5c2c05277bc9538 HTTP/1.0" 202 Accepted 2025-09-26 18:01:21,631 - lava_callback - INFO - Starting async job submit for node 68d43b7ca5c2c05277bc9538 2025-09-26 18:01:21,653 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:01:21,667 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d43b7ca5c2c05277bc9538 HTTP/1.1" 200 None 2025-09-26 18:01:21,668 - lava_callback - DEBUG - Processing node 68d43b7ca5c2c05277bc9538: baseline-arm64 2025-09-26 18:01:21,974 - lava_callback - ERROR - Error processing callback for node 68d43b7ca5c2c05277bc9538: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:42208 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51112 - "GET / HTTP/1.0" 200 OK 2025-09-26 18:01:31,470 - lava_callback - INFO - Received callback for node 68d431eda5c2c05277bc8596 INFO: 172.21.0.1:51122 - "POST /node/68d431eda5c2c05277bc8596 HTTP/1.0" 202 Accepted 2025-09-26 18:01:31,475 - lava_callback - INFO - Starting async job submit for node 68d431eda5c2c05277bc8596 2025-09-26 18:01:31,476 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:01:31,489 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d431eda5c2c05277bc8596 HTTP/1.1" 200 None 2025-09-26 18:01:31,490 - lava_callback - DEBUG - Processing node 68d431eda5c2c05277bc8596: baseline-arm 2025-09-26 18:01:31,490 - lava_callback - ERROR - Error processing callback for node 68d431eda5c2c05277bc8596: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 18:01:32,168 - lava_callback - INFO - Received callback for node 68d431f0a5c2c05277bc8599 INFO: 172.21.0.1:51132 - "POST /node/68d431f0a5c2c05277bc8599 HTTP/1.0" 202 Accepted 2025-09-26 18:01:32,173 - lava_callback - INFO - Starting async job submit for node 68d431f0a5c2c05277bc8599 2025-09-26 18:01:32,174 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:01:32,186 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d431f0a5c2c05277bc8599 HTTP/1.1" 200 None 2025-09-26 18:01:32,187 - lava_callback - DEBUG - Processing node 68d431f0a5c2c05277bc8599: baseline-arm 2025-09-26 18:01:32,187 - lava_callback - ERROR - Error processing callback for node 68d431f0a5c2c05277bc8599: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 18:01:32,291 - lava_callback - INFO - Received callback for node 68d6d48d3c94e462c6881e3f INFO: 172.21.0.1:51148 - "POST /node/68d6d48d3c94e462c6881e3f HTTP/1.0" 202 Accepted 2025-09-26 18:01:32,297 - lava_callback - INFO - Starting async job submit for node 68d6d48d3c94e462c6881e3f 2025-09-26 18:01:32,320 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:01:32,333 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6d48d3c94e462c6881e3f HTTP/1.1" 200 None 2025-09-26 18:01:32,334 - lava_callback - DEBUG - Processing node 68d6d48d3c94e462c6881e3f: baseline-arm64-broonie 2025-09-26 18:01:32,583 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 18:01:32,703 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 18:01:32,704 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-68d6d48d3c94e462c6881e3f/log.txt.gz 2025-09-26 18:01:32,710 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 18:01:32,836 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 18:01:32,836 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-68d6d48d3c94e462c6881e3f/lava_callback.json.gz 2025-09-26 18:01:32,860 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:01:32,864 - lava_callback - INFO - Received callback for node 68d431f2a5c2c05277bc859c INFO: 172.21.0.1:51164 - "POST /node/68d431f2a5c2c05277bc859c HTTP/1.0" 202 Accepted 2025-09-26 18:01:32,870 - lava_callback - INFO - Starting async job submit for node 68d431f2a5c2c05277bc859c 2025-09-26 18:01:32,871 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:01:32,887 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6d48d3c94e462c6881e3f HTTP/1.1" 200 None 2025-09-26 18:01:32,889 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:01:32,891 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d431f2a5c2c05277bc859c HTTP/1.1" 200 None 2025-09-26 18:01:32,891 - lava_callback - DEBUG - Processing node 68d431f2a5c2c05277bc859c: baseline-arm 2025-09-26 18:01:32,891 - lava_callback - ERROR - Error processing callback for node 68d431f2a5c2c05277bc859c: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 18:01:32,911 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6bd8e3c94e462c68814f1 HTTP/1.1" 200 None 2025-09-26 18:01:32,912 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:01:32,965 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6d48d3c94e462c6881e3f HTTP/1.1" 200 None 2025-09-26 18:01:32,966 - lava_callback - INFO - Completed processing callback for node 68d6d48d3c94e462c6881e3f 2025-09-26 18:01:33,568 - lava_callback - INFO - Received callback for node 68d431f3a5c2c05277bc859f INFO: 172.21.0.1:51170 - "POST /node/68d431f3a5c2c05277bc859f HTTP/1.0" 202 Accepted 2025-09-26 18:01:33,574 - lava_callback - INFO - Starting async job submit for node 68d431f3a5c2c05277bc859f 2025-09-26 18:01:33,575 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:01:33,587 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d431f3a5c2c05277bc859f HTTP/1.1" 200 None 2025-09-26 18:01:33,587 - lava_callback - DEBUG - Processing node 68d431f3a5c2c05277bc859f: baseline-arm 2025-09-26 18:01:33,587 - lava_callback - ERROR - Error processing callback for node 68d431f3a5c2c05277bc859f: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:51174 - "GET / HTTP/1.0" 200 OK 2025-09-26 18:01:38,778 - lava_callback - INFO - Received callback for node 68d6d4903c94e462c6881e4b INFO: 172.21.0.1:51184 - "POST /node/68d6d4903c94e462c6881e4b HTTP/1.0" 202 Accepted 2025-09-26 18:01:38,784 - lava_callback - INFO - Starting async job submit for node 68d6d4903c94e462c6881e4b 2025-09-26 18:01:38,805 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:01:38,817 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6d4903c94e462c6881e4b HTTP/1.1" 200 None 2025-09-26 18:01:38,818 - lava_callback - DEBUG - Processing node 68d6d4903c94e462c6881e4b: baseline-arm64-clabbe 2025-09-26 18:01:39,144 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 18:01:39,272 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 18:01:39,273 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-clabbe-68d6d4903c94e462c6881e4b/log.txt.gz 2025-09-26 18:01:39,281 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 18:01:39,423 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 18:01:39,424 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-clabbe-68d6d4903c94e462c6881e4b/lava_callback.json.gz 2025-09-26 18:01:39,445 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:01:39,488 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6d4903c94e462c6881e4b HTTP/1.1" 200 None 2025-09-26 18:01:39,489 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:01:39,501 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6bd8e3c94e462c68814f1 HTTP/1.1" 200 None 2025-09-26 18:01:39,502 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:01:39,547 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6d4903c94e462c6881e4b HTTP/1.1" 200 None 2025-09-26 18:01:39,548 - lava_callback - INFO - Completed processing callback for node 68d6d4903c94e462c6881e4b 2025-09-26 18:01:40,345 - lava_callback - INFO - Received callback for node 68d6d4943c94e462c6881e63 INFO: 172.21.0.1:56210 - "POST /node/68d6d4943c94e462c6881e63 HTTP/1.0" 202 Accepted 2025-09-26 18:01:40,352 - lava_callback - INFO - Starting async job submit for node 68d6d4943c94e462c6881e63 2025-09-26 18:01:40,393 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:01:40,410 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6d4943c94e462c6881e63 HTTP/1.1" 200 None 2025-09-26 18:01:40,410 - lava_callback - DEBUG - Processing node 68d6d4943c94e462c6881e63: baseline-arm64-kontron 2025-09-26 18:01:40,656 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 18:01:40,759 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 18:01:40,760 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-kontron-68d6d4943c94e462c6881e63/log.txt.gz 2025-09-26 18:01:40,766 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 18:01:40,879 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 18:01:40,879 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-kontron-68d6d4943c94e462c6881e63/lava_callback.json.gz 2025-09-26 18:01:40,914 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:01:40,927 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6d4943c94e462c6881e63 HTTP/1.1" 200 None 2025-09-26 18:01:40,928 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:01:40,940 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6bd8e3c94e462c68814f1 HTTP/1.1" 200 None 2025-09-26 18:01:40,941 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:01:40,963 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6d4943c94e462c6881e63 HTTP/1.1" 200 None 2025-09-26 18:01:40,963 - lava_callback - INFO - Completed processing callback for node 68d6d4943c94e462c6881e63 INFO: 172.21.0.1:56222 - "GET / HTTP/1.0" 200 OK 2025-09-26 18:01:52,260 - lava_callback - INFO - Received callback for node 68d6d4893c94e462c6881e24 INFO: 172.21.0.1:42138 - "POST /node/68d6d4893c94e462c6881e24 HTTP/1.0" 202 Accepted 2025-09-26 18:01:52,266 - lava_callback - INFO - Starting async job submit for node 68d6d4893c94e462c6881e24 2025-09-26 18:01:52,289 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:01:52,303 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6d4893c94e462c6881e24 HTTP/1.1" 200 None 2025-09-26 18:01:52,303 - lava_callback - DEBUG - Processing node 68d6d4893c94e462c6881e24: baseline-arm64-broonie 2025-09-26 18:01:52,606 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 18:01:52,752 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 18:01:52,753 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-68d6d4893c94e462c6881e24/log.txt.gz 2025-09-26 18:01:52,761 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 18:01:52,879 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 18:01:52,879 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-68d6d4893c94e462c6881e24/lava_callback.json.gz 2025-09-26 18:01:52,904 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:01:52,941 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6d4893c94e462c6881e24 HTTP/1.1" 200 None 2025-09-26 18:01:52,942 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:01:52,954 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6bd8e3c94e462c68814f1 HTTP/1.1" 200 None 2025-09-26 18:01:52,955 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:01:52,990 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6d4893c94e462c6881e24 HTTP/1.1" 200 None 2025-09-26 18:01:52,991 - lava_callback - INFO - Completed processing callback for node 68d6d4893c94e462c6881e24 2025-09-26 18:01:53,839 - lava_callback - INFO - Received callback for node 68d6d48a3c94e462c6881e27 INFO: 172.21.0.1:42146 - "POST /node/68d6d48a3c94e462c6881e27 HTTP/1.0" 202 Accepted 2025-09-26 18:01:53,845 - lava_callback - INFO - Starting async job submit for node 68d6d48a3c94e462c6881e27 2025-09-26 18:01:53,868 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:01:53,880 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6d48a3c94e462c6881e27 HTTP/1.1" 200 None 2025-09-26 18:01:53,881 - lava_callback - DEBUG - Processing node 68d6d48a3c94e462c6881e27: baseline-arm64-broonie 2025-09-26 18:01:54,131 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 18:01:54,253 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 18:01:54,254 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-68d6d48a3c94e462c6881e27/log.txt.gz 2025-09-26 18:01:54,260 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 18:01:54,371 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 18:01:54,372 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-68d6d48a3c94e462c6881e27/lava_callback.json.gz 2025-09-26 18:01:54,394 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:01:54,409 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6d48a3c94e462c6881e27 HTTP/1.1" 200 None 2025-09-26 18:01:54,410 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:01:54,422 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6bd8e3c94e462c68814f1 HTTP/1.1" 200 None 2025-09-26 18:01:54,423 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:01:54,449 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6d48a3c94e462c6881e27 HTTP/1.1" 200 None 2025-09-26 18:01:54,450 - lava_callback - INFO - Completed processing callback for node 68d6d48a3c94e462c6881e27 2025-09-26 18:01:55,340 - lava_callback - INFO - Received callback for node 68d6d48e3c94e462c6881e42 INFO: 172.21.0.1:42154 - "POST /node/68d6d48e3c94e462c6881e42 HTTP/1.0" 202 Accepted 2025-09-26 18:01:55,346 - lava_callback - INFO - Starting async job submit for node 68d6d48e3c94e462c6881e42 2025-09-26 18:01:55,368 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:01:55,385 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6d48e3c94e462c6881e42 HTTP/1.1" 200 None 2025-09-26 18:01:55,385 - lava_callback - DEBUG - Processing node 68d6d48e3c94e462c6881e42: baseline-arm64-broonie 2025-09-26 18:01:55,648 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 18:01:55,755 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 18:01:55,756 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-68d6d48e3c94e462c6881e42/log.txt.gz 2025-09-26 18:01:55,763 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 18:01:55,875 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 18:01:55,876 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-68d6d48e3c94e462c6881e42/lava_callback.json.gz 2025-09-26 18:01:55,898 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:01:55,910 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6d48e3c94e462c6881e42 HTTP/1.1" 200 None 2025-09-26 18:01:55,911 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:01:55,922 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6bd8e3c94e462c68814f1 HTTP/1.1" 200 None 2025-09-26 18:01:55,923 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:01:55,950 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6d48e3c94e462c6881e42 HTTP/1.1" 200 None 2025-09-26 18:01:55,951 - lava_callback - INFO - Completed processing callback for node 68d6d48e3c94e462c6881e42 INFO: 172.21.0.1:42164 - "GET / HTTP/1.0" 200 OK 2025-09-26 18:02:03,583 - lava_callback - INFO - Received callback for node 68d6d4933c94e462c6881e60 INFO: 172.21.0.1:43952 - "POST /node/68d6d4933c94e462c6881e60 HTTP/1.0" 202 Accepted 2025-09-26 18:02:03,589 - lava_callback - INFO - Starting async job submit for node 68d6d4933c94e462c6881e60 2025-09-26 18:02:03,611 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:02:03,623 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6d4933c94e462c6881e60 HTTP/1.1" 200 None 2025-09-26 18:02:03,624 - lava_callback - DEBUG - Processing node 68d6d4933c94e462c6881e60: baseline-arm64-clabbe 2025-09-26 18:02:03,868 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 18:02:04,008 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 18:02:04,009 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-clabbe-68d6d4933c94e462c6881e60/log.txt.gz 2025-09-26 18:02:04,015 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 18:02:04,160 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 18:02:04,161 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-clabbe-68d6d4933c94e462c6881e60/lava_callback.json.gz 2025-09-26 18:02:04,182 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:02:04,221 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6d4933c94e462c6881e60 HTTP/1.1" 200 None 2025-09-26 18:02:04,222 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:02:04,233 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6bd8e3c94e462c68814f1 HTTP/1.1" 200 None 2025-09-26 18:02:04,234 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:02:04,273 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6d4933c94e462c6881e60 HTTP/1.1" 200 None 2025-09-26 18:02:04,273 - lava_callback - INFO - Completed processing callback for node 68d6d4933c94e462c6881e60 INFO: 172.21.0.1:43958 - "GET / HTTP/1.0" 200 OK 2025-09-26 18:02:10,037 - lava_callback - INFO - Received callback for node 68d43b7fa5c2c05277bc953b INFO: 172.21.0.1:40652 - "POST /node/68d43b7fa5c2c05277bc953b HTTP/1.0" 202 Accepted 2025-09-26 18:02:10,043 - lava_callback - INFO - Starting async job submit for node 68d43b7fa5c2c05277bc953b 2025-09-26 18:02:10,064 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:02:10,076 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d43b7fa5c2c05277bc953b HTTP/1.1" 200 None 2025-09-26 18:02:10,076 - lava_callback - DEBUG - Processing node 68d43b7fa5c2c05277bc953b: baseline-arm64 2025-09-26 18:02:10,372 - lava_callback - ERROR - Error processing callback for node 68d43b7fa5c2c05277bc953b: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None 2025-09-26 18:02:12,236 - lava_callback - INFO - Received callback for node 68d6d48c3c94e462c6881e33 INFO: 172.21.0.1:40660 - "POST /node/68d6d48c3c94e462c6881e33 HTTP/1.0" 202 Accepted 2025-09-26 18:02:12,243 - lava_callback - INFO - Starting async job submit for node 68d6d48c3c94e462c6881e33 2025-09-26 18:02:12,264 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:02:12,284 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6d48c3c94e462c6881e33 HTTP/1.1" 200 None 2025-09-26 18:02:12,285 - lava_callback - DEBUG - Processing node 68d6d48c3c94e462c6881e33: baseline-arm64-broonie 2025-09-26 18:02:12,547 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 18:02:12,674 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 18:02:12,674 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-68d6d48c3c94e462c6881e33/log.txt.gz 2025-09-26 18:02:12,681 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 18:02:12,814 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 18:02:12,815 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-68d6d48c3c94e462c6881e33/lava_callback.json.gz 2025-09-26 18:02:12,836 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:02:12,850 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6d48c3c94e462c6881e33 HTTP/1.1" 200 None 2025-09-26 18:02:12,851 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:02:12,862 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6bd8e3c94e462c68814f1 HTTP/1.1" 200 None 2025-09-26 18:02:12,863 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:02:12,891 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6d48c3c94e462c6881e33 HTTP/1.1" 200 None 2025-09-26 18:02:12,891 - lava_callback - INFO - Completed processing callback for node 68d6d48c3c94e462c6881e33 INFO: 172.21.0.1:40674 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40684 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40690 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51104 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53908 - "GET / HTTP/1.0" 200 OK 2025-09-26 18:02:32,603 - lava_callback - INFO - Received callback for node 68d43225a5c2c05277bc85ef INFO: 172.21.0.1:53918 - "POST /node/68d43225a5c2c05277bc85ef HTTP/1.0" 202 Accepted 2025-09-26 18:02:32,608 - lava_callback - INFO - Starting async job submit for node 68d43225a5c2c05277bc85ef 2025-09-26 18:02:32,609 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:02:32,611 - lava_callback - INFO - Received callback for node 68d43233a5c2c05277bc85fb INFO: 172.21.0.1:53930 - "POST /node/68d43233a5c2c05277bc85fb HTTP/1.0" 202 Accepted 2025-09-26 18:02:32,616 - lava_callback - INFO - Starting async job submit for node 68d43233a5c2c05277bc85fb 2025-09-26 18:02:32,617 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:02:32,627 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d43225a5c2c05277bc85ef HTTP/1.1" 200 None 2025-09-26 18:02:32,627 - lava_callback - DEBUG - Processing node 68d43225a5c2c05277bc85ef: baseline-arm-broonie 2025-09-26 18:02:32,627 - lava_callback - ERROR - Error processing callback for node 68d43225a5c2c05277bc85ef: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 18:02:32,643 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d43233a5c2c05277bc85fb HTTP/1.1" 200 None 2025-09-26 18:02:32,643 - lava_callback - DEBUG - Processing node 68d43233a5c2c05277bc85fb: baseline-arm-clabbe 2025-09-26 18:02:32,643 - lava_callback - ERROR - Error processing callback for node 68d43233a5c2c05277bc85fb: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 18:02:34,188 - lava_callback - INFO - Received callback for node 68d43b82a5c2c05277bc953e INFO: 172.21.0.1:53946 - "POST /node/68d43b82a5c2c05277bc953e HTTP/1.0" 202 Accepted 2025-09-26 18:02:34,195 - lava_callback - INFO - Starting async job submit for node 68d43b82a5c2c05277bc953e 2025-09-26 18:02:34,213 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:02:34,226 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d43b82a5c2c05277bc953e HTTP/1.1" 200 None 2025-09-26 18:02:34,226 - lava_callback - DEBUG - Processing node 68d43b82a5c2c05277bc953e: baseline-arm64 2025-09-26 18:02:35,214 - lava_callback - ERROR - Error processing callback for node 68d43b82a5c2c05277bc953e: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:53950 - "GET / HTTP/1.0" 200 OK 2025-09-26 18:02:41,445 - lava_callback - INFO - Received callback for node 68d43b89a5c2c05277bc9544 INFO: 172.21.0.1:46974 - "POST /node/68d43b89a5c2c05277bc9544 HTTP/1.0" 202 Accepted 2025-09-26 18:02:41,452 - lava_callback - INFO - Starting async job submit for node 68d43b89a5c2c05277bc9544 2025-09-26 18:02:41,472 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:02:41,484 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d43b89a5c2c05277bc9544 HTTP/1.1" 200 None 2025-09-26 18:02:41,485 - lava_callback - DEBUG - Processing node 68d43b89a5c2c05277bc9544: baseline-arm64 2025-09-26 18:02:41,783 - lava_callback - ERROR - Error processing callback for node 68d43b89a5c2c05277bc9544: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:46990 - "GET / HTTP/1.0" 200 OK 2025-09-26 18:02:52,634 - lava_callback - INFO - Received callback for node 68d4323da5c2c05277bc8607 INFO: 172.21.0.1:54656 - "POST /node/68d4323da5c2c05277bc8607 HTTP/1.0" 202 Accepted 2025-09-26 18:02:52,641 - lava_callback - INFO - Starting async job submit for node 68d4323da5c2c05277bc8607 2025-09-26 18:02:52,642 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:02:52,655 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4323da5c2c05277bc8607 HTTP/1.1" 200 None 2025-09-26 18:02:52,655 - lava_callback - DEBUG - Processing node 68d4323da5c2c05277bc8607: baseline-arm-clabbe 2025-09-26 18:02:52,655 - lava_callback - ERROR - Error processing callback for node 68d4323da5c2c05277bc8607: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 18:02:53,299 - lava_callback - INFO - Received callback for node 68d43240a5c2c05277bc860a INFO: 172.21.0.1:54668 - "POST /node/68d43240a5c2c05277bc860a HTTP/1.0" 202 Accepted 2025-09-26 18:02:53,305 - lava_callback - INFO - Starting async job submit for node 68d43240a5c2c05277bc860a 2025-09-26 18:02:53,306 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:02:53,318 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d43240a5c2c05277bc860a HTTP/1.1" 200 None 2025-09-26 18:02:53,319 - lava_callback - DEBUG - Processing node 68d43240a5c2c05277bc860a: baseline-arm-clabbe 2025-09-26 18:02:53,319 - lava_callback - ERROR - Error processing callback for node 68d43240a5c2c05277bc860a: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:54674 - "GET / HTTP/1.0" 200 OK 2025-09-26 18:03:06,348 - lava_callback - INFO - Received callback for node 68d6d4833c94e462c6881df7 INFO: 172.21.0.1:48568 - "POST /node/68d6d4833c94e462c6881df7 HTTP/1.0" 202 Accepted 2025-09-26 18:03:06,354 - lava_callback - INFO - Starting async job submit for node 68d6d4833c94e462c6881df7 2025-09-26 18:03:06,375 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:03:06,388 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6d4833c94e462c6881df7 HTTP/1.1" 200 None 2025-09-26 18:03:06,388 - lava_callback - DEBUG - Processing node 68d6d4833c94e462c6881df7: baseline-arm64-baylibre INFO: 172.21.0.1:48574 - "GET / HTTP/1.0" 200 OK 2025-09-26 18:03:06,651 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 18:03:06,800 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 18:03:06,801 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-baylibre-68d6d4833c94e462c6881df7/log.txt.gz 2025-09-26 18:03:06,808 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 18:03:06,955 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 18:03:06,956 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-baylibre-68d6d4833c94e462c6881df7/lava_callback.json.gz 2025-09-26 18:03:06,977 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:03:07,005 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6d4833c94e462c6881df7 HTTP/1.1" 200 None 2025-09-26 18:03:07,006 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:03:07,018 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6bd8e3c94e462c68814f1 HTTP/1.1" 200 None 2025-09-26 18:03:07,019 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:03:07,065 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6d4833c94e462c6881df7 HTTP/1.1" 200 None 2025-09-26 18:03:07,066 - lava_callback - INFO - Completed processing callback for node 68d6d4833c94e462c6881df7 2025-09-26 18:03:12,654 - lava_callback - INFO - Received callback for node 68d43253a5c2c05277bc862f INFO: 172.21.0.1:36192 - "POST /node/68d43253a5c2c05277bc862f HTTP/1.0" 202 Accepted 2025-09-26 18:03:12,660 - lava_callback - INFO - Starting async job submit for node 68d43253a5c2c05277bc862f 2025-09-26 18:03:12,660 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:03:12,678 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d43253a5c2c05277bc862f HTTP/1.1" 200 None 2025-09-26 18:03:12,678 - lava_callback - DEBUG - Processing node 68d43253a5c2c05277bc862f: baseline-arm-clabbe 2025-09-26 18:03:12,678 - lava_callback - ERROR - Error processing callback for node 68d43253a5c2c05277bc862f: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:36202 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36204 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36214 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43862 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55222 - "GET / HTTP/1.0" 200 OK 2025-09-26 18:03:29,445 - lava_callback - INFO - Received callback for node 68d4a9bda5c2c05277bcf7b1 INFO: 172.21.0.1:55228 - "POST /node/68d4a9bda5c2c05277bcf7b1 HTTP/1.0" 202 Accepted 2025-09-26 18:03:29,451 - lava_callback - INFO - Starting async job submit for node 68d4a9bda5c2c05277bcf7b1 2025-09-26 18:03:29,472 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:03:29,484 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4a9bda5c2c05277bcf7b1 HTTP/1.1" 200 None 2025-09-26 18:03:29,485 - lava_callback - DEBUG - Processing node 68d4a9bda5c2c05277bcf7b1: baseline-arm 2025-09-26 18:03:29,747 - lava_callback - ERROR - Error processing callback for node 68d4a9bda5c2c05277bcf7b1: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None 2025-09-26 18:03:29,955 - lava_callback - INFO - Received callback for node 68d6d4853c94e462c6881e03 INFO: 172.21.0.1:55232 - "POST /node/68d6d4853c94e462c6881e03 HTTP/1.0" 202 Accepted 2025-09-26 18:03:29,961 - lava_callback - INFO - Starting async job submit for node 68d6d4853c94e462c6881e03 2025-09-26 18:03:29,982 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:03:29,995 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6d4853c94e462c6881e03 HTTP/1.1" 200 None 2025-09-26 18:03:29,995 - lava_callback - DEBUG - Processing node 68d6d4853c94e462c6881e03: baseline-arm64-baylibre 2025-09-26 18:03:30,253 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 18:03:30,376 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 18:03:30,377 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-baylibre-68d6d4853c94e462c6881e03/log.txt.gz 2025-09-26 18:03:30,384 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 18:03:30,497 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 18:03:30,497 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-baylibre-68d6d4853c94e462c6881e03/lava_callback.json.gz 2025-09-26 18:03:30,518 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:03:30,534 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6d4853c94e462c6881e03 HTTP/1.1" 200 None 2025-09-26 18:03:30,534 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:03:30,546 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6bd8e3c94e462c68814f1 HTTP/1.1" 200 None 2025-09-26 18:03:30,547 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:03:30,573 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6d4853c94e462c6881e03 HTTP/1.1" 200 None 2025-09-26 18:03:30,574 - lava_callback - INFO - Completed processing callback for node 68d6d4853c94e462c6881e03 INFO: 172.21.0.1:55236 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39952 - "GET / HTTP/1.0" 200 OK 2025-09-26 18:03:50,717 - lava_callback - INFO - Received callback for node 68d43b8ba5c2c05277bc9547 INFO: 172.21.0.1:56028 - "POST /node/68d43b8ba5c2c05277bc9547 HTTP/1.0" 202 Accepted 2025-09-26 18:03:50,724 - lava_callback - INFO - Starting async job submit for node 68d43b8ba5c2c05277bc9547 2025-09-26 18:03:50,744 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:03:50,758 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d43b8ba5c2c05277bc9547 HTTP/1.1" 200 None 2025-09-26 18:03:50,758 - lava_callback - DEBUG - Processing node 68d43b8ba5c2c05277bc9547: baseline-arm64 2025-09-26 18:03:51,056 - lava_callback - ERROR - Error processing callback for node 68d43b8ba5c2c05277bc9547: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None 2025-09-26 18:03:52,634 - lava_callback - INFO - Received callback for node 68d4327ba5c2c05277bc8662 INFO: 172.21.0.1:56030 - "POST /node/68d4327ba5c2c05277bc8662 HTTP/1.0" 202 Accepted 2025-09-26 18:03:52,640 - lava_callback - INFO - Starting async job submit for node 68d4327ba5c2c05277bc8662 2025-09-26 18:03:52,641 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:03:52,653 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4327ba5c2c05277bc8662 HTTP/1.1" 200 None 2025-09-26 18:03:52,653 - lava_callback - DEBUG - Processing node 68d4327ba5c2c05277bc8662: ltp-fs-bind 2025-09-26 18:03:52,653 - lava_callback - ERROR - Error processing callback for node 68d4327ba5c2c05277bc8662: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 18:03:53,414 - lava_callback - INFO - Received callback for node 68d4327ca5c2c05277bc8665 INFO: 172.21.0.1:56040 - "POST /node/68d4327ca5c2c05277bc8665 HTTP/1.0" 202 Accepted 2025-09-26 18:03:53,420 - lava_callback - INFO - Starting async job submit for node 68d4327ca5c2c05277bc8665 2025-09-26 18:03:53,421 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:03:53,433 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4327ca5c2c05277bc8665 HTTP/1.1" 200 None 2025-09-26 18:03:53,434 - lava_callback - DEBUG - Processing node 68d4327ca5c2c05277bc8665: ltp-fsx 2025-09-26 18:03:53,434 - lava_callback - ERROR - Error processing callback for node 68d4327ca5c2c05277bc8665: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 18:03:54,220 - lava_callback - INFO - Received callback for node 68d43280a5c2c05277bc8668 INFO: 172.21.0.1:56050 - "POST /node/68d43280a5c2c05277bc8668 HTTP/1.0" 202 Accepted 2025-09-26 18:03:54,226 - lava_callback - INFO - Starting async job submit for node 68d43280a5c2c05277bc8668 2025-09-26 18:03:54,227 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:03:54,239 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d43280a5c2c05277bc8668 HTTP/1.1" 200 None 2025-09-26 18:03:54,239 - lava_callback - DEBUG - Processing node 68d43280a5c2c05277bc8668: ltp-input 2025-09-26 18:03:54,240 - lava_callback - ERROR - Error processing callback for node 68d43280a5c2c05277bc8668: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 18:03:56,432 - lava_callback - INFO - Received callback for node 68d43b8fa5c2c05277bc954a INFO: 172.21.0.1:56058 - "POST /node/68d43b8fa5c2c05277bc954a HTTP/1.0" 202 Accepted 2025-09-26 18:03:56,438 - lava_callback - INFO - Starting async job submit for node 68d43b8fa5c2c05277bc954a 2025-09-26 18:03:56,459 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:03:56,473 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d43b8fa5c2c05277bc954a HTTP/1.1" 200 None 2025-09-26 18:03:56,474 - lava_callback - DEBUG - Processing node 68d43b8fa5c2c05277bc954a: baseline-arm64 INFO: 172.21.0.1:56066 - "GET / HTTP/1.0" 200 OK 2025-09-26 18:03:56,798 - lava_callback - ERROR - Error processing callback for node 68d43b8fa5c2c05277bc954a: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None 2025-09-26 18:04:00,403 - lava_callback - INFO - Received callback for node 68d6d5053c94e462c6881e9d INFO: 172.21.0.1:42212 - "POST /node/68d6d5053c94e462c6881e9d HTTP/1.0" 202 Accepted 2025-09-26 18:04:00,409 - lava_callback - INFO - Starting async job submit for node 68d6d5053c94e462c6881e9d 2025-09-26 18:04:00,443 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:04:00,455 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6d5053c94e462c6881e9d HTTP/1.1" 200 None 2025-09-26 18:04:00,456 - lava_callback - DEBUG - Processing node 68d6d5053c94e462c6881e9d: baseline-arm64-kontron 2025-09-26 18:04:00,764 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 18:04:00,870 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 18:04:00,871 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-kontron-68d6d5053c94e462c6881e9d/log.txt.gz 2025-09-26 18:04:00,877 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 18:04:01,026 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 18:04:01,027 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-kontron-68d6d5053c94e462c6881e9d/lava_callback.json.gz 2025-09-26 18:04:01,061 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:04:01,074 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6d5053c94e462c6881e9d HTTP/1.1" 200 None 2025-09-26 18:04:01,075 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:04:01,086 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6bd8e3c94e462c68814f1 HTTP/1.1" 200 None 2025-09-26 18:04:01,087 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:04:01,112 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6d5053c94e462c6881e9d HTTP/1.1" 200 None 2025-09-26 18:04:01,113 - lava_callback - INFO - Completed processing callback for node 68d6d5053c94e462c6881e9d 2025-09-26 18:04:02,853 - lava_callback - INFO - Received callback for node 68d43b92a5c2c05277bc954d INFO: 172.21.0.1:42218 - "POST /node/68d43b92a5c2c05277bc954d HTTP/1.0" 202 Accepted 2025-09-26 18:04:02,859 - lava_callback - INFO - Starting async job submit for node 68d43b92a5c2c05277bc954d 2025-09-26 18:04:02,882 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:04:02,899 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d43b92a5c2c05277bc954d HTTP/1.1" 200 None 2025-09-26 18:04:02,900 - lava_callback - DEBUG - Processing node 68d43b92a5c2c05277bc954d: baseline-arm64 2025-09-26 18:04:03,268 - lava_callback - ERROR - Error processing callback for node 68d43b92a5c2c05277bc954d: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:42228 - "GET / HTTP/1.0" 200 OK 2025-09-26 18:04:12,135 - lava_callback - INFO - Received callback for node 68d6d4843c94e462c6881dfd INFO: 172.21.0.1:58262 - "POST /node/68d6d4843c94e462c6881dfd HTTP/1.0" 202 Accepted 2025-09-26 18:04:12,141 - lava_callback - INFO - Starting async job submit for node 68d6d4843c94e462c6881dfd 2025-09-26 18:04:12,161 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:04:12,194 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6d4843c94e462c6881dfd HTTP/1.1" 200 None 2025-09-26 18:04:12,195 - lava_callback - DEBUG - Processing node 68d6d4843c94e462c6881dfd: baseline-arm64-baylibre 2025-09-26 18:04:12,384 - lava_callback - INFO - Received callback for node 68d431d7a5c2c05277bc8574 INFO: 172.21.0.1:58268 - "POST /node/68d431d7a5c2c05277bc8574 HTTP/1.0" 202 Accepted 2025-09-26 18:04:12,421 - lava_callback - INFO - Starting async job submit for node 68d431d7a5c2c05277bc8574 2025-09-26 18:04:12,482 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:04:12,543 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d431d7a5c2c05277bc8574 HTTP/1.1" 200 None 2025-09-26 18:04:12,543 - lava_callback - DEBUG - Processing node 68d431d7a5c2c05277bc8574: baseline-arm 2025-09-26 18:04:12,596 - lava_callback - INFO - Received callback for node 68d4328aa5c2c05277bc8671 INFO: 172.21.0.1:58280 - "POST /node/68d4328aa5c2c05277bc8671 HTTP/1.0" 202 Accepted 2025-09-26 18:04:12,613 - lava_callback - INFO - Starting async job submit for node 68d4328aa5c2c05277bc8671 2025-09-26 18:04:12,614 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:04:12,629 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 18:04:12,707 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4328aa5c2c05277bc8671 HTTP/1.1" 200 None 2025-09-26 18:04:12,723 - lava_callback - DEBUG - Processing node 68d4328aa5c2c05277bc8671: ltp-ipc 2025-09-26 18:04:12,723 - lava_callback - ERROR - Error processing callback for node 68d4328aa5c2c05277bc8671: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 18:04:12,828 - lava_callback - ERROR - Error processing callback for node 68d431d7a5c2c05277bc8574: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None 2025-09-26 18:04:12,859 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 18:04:12,860 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-baylibre-68d6d4843c94e462c6881dfd/log.txt.gz 2025-09-26 18:04:12,867 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 18:04:12,988 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 18:04:12,989 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-baylibre-68d6d4843c94e462c6881dfd/lava_callback.json.gz 2025-09-26 18:04:13,012 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:04:13,024 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6d4843c94e462c6881dfd HTTP/1.1" 200 None 2025-09-26 18:04:13,025 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:04:13,047 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6bd8e3c94e462c68814f1 HTTP/1.1" 200 None 2025-09-26 18:04:13,048 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:04:13,080 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6d4843c94e462c6881dfd HTTP/1.1" 200 None 2025-09-26 18:04:13,080 - lava_callback - INFO - Completed processing callback for node 68d6d4843c94e462c6881dfd 2025-09-26 18:04:13,401 - lava_callback - INFO - Received callback for node 68d4328fa5c2c05277bc868b INFO: 172.21.0.1:58286 - "POST /node/68d4328fa5c2c05277bc868b HTTP/1.0" 202 Accepted 2025-09-26 18:04:13,407 - lava_callback - INFO - Starting async job submit for node 68d4328fa5c2c05277bc868b 2025-09-26 18:04:13,408 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:04:13,419 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4328fa5c2c05277bc868b HTTP/1.1" 200 None 2025-09-26 18:04:13,420 - lava_callback - DEBUG - Processing node 68d4328fa5c2c05277bc868b: ltp-pty 2025-09-26 18:04:13,420 - lava_callback - ERROR - Error processing callback for node 68d4328fa5c2c05277bc868b: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 18:04:14,152 - lava_callback - INFO - Received callback for node 68d43298a5c2c05277bc8694 INFO: 172.21.0.1:58290 - "POST /node/68d43298a5c2c05277bc8694 HTTP/1.0" 202 Accepted 2025-09-26 18:04:14,158 - lava_callback - INFO - Starting async job submit for node 68d43298a5c2c05277bc8694 2025-09-26 18:04:14,159 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:04:14,171 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d43298a5c2c05277bc8694 HTTP/1.1" 200 None 2025-09-26 18:04:14,171 - lava_callback - DEBUG - Processing node 68d43298a5c2c05277bc8694: ltp-sched 2025-09-26 18:04:14,171 - lava_callback - ERROR - Error processing callback for node 68d43298a5c2c05277bc8694: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 18:04:14,888 - lava_callback - INFO - Received callback for node 68d4329ca5c2c05277bc8697 INFO: 172.21.0.1:58298 - "POST /node/68d4329ca5c2c05277bc8697 HTTP/1.0" 202 Accepted 2025-09-26 18:04:14,893 - lava_callback - INFO - Starting async job submit for node 68d4329ca5c2c05277bc8697 2025-09-26 18:04:14,894 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:04:14,906 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4329ca5c2c05277bc8697 HTTP/1.1" 200 None 2025-09-26 18:04:14,906 - lava_callback - DEBUG - Processing node 68d4329ca5c2c05277bc8697: ltp-smoketest 2025-09-26 18:04:14,906 - lava_callback - ERROR - Error processing callback for node 68d4329ca5c2c05277bc8697: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:58314 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58328 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58330 - "GET / HTTP/1.0" 200 OK 2025-09-26 18:04:17,680 - lava_callback - INFO - Received callback for node 68d6d4863c94e462c6881e12 INFO: 172.21.0.1:58342 - "POST /node/68d6d4863c94e462c6881e12 HTTP/1.0" 202 Accepted 2025-09-26 18:04:17,686 - lava_callback - INFO - Starting async job submit for node 68d6d4863c94e462c6881e12 2025-09-26 18:04:17,706 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:04:17,763 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6d4863c94e462c6881e12 HTTP/1.1" 200 None 2025-09-26 18:04:17,763 - lava_callback - DEBUG - Processing node 68d6d4863c94e462c6881e12: baseline-arm64-baylibre 2025-09-26 18:04:18,040 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 18:04:18,181 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 18:04:18,182 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-baylibre-68d6d4863c94e462c6881e12/log.txt.gz 2025-09-26 18:04:18,188 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 18:04:18,301 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 18:04:18,302 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-baylibre-68d6d4863c94e462c6881e12/lava_callback.json.gz 2025-09-26 18:04:18,323 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:04:18,335 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6d4863c94e462c6881e12 HTTP/1.1" 200 None 2025-09-26 18:04:18,336 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:04:18,347 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6bd8e3c94e462c68814f1 HTTP/1.1" 200 None 2025-09-26 18:04:18,348 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:04:18,372 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6d4863c94e462c6881e12 HTTP/1.1" 200 None 2025-09-26 18:04:18,373 - lava_callback - INFO - Completed processing callback for node 68d6d4863c94e462c6881e12 2025-09-26 18:04:19,529 - lava_callback - INFO - Received callback for node 68d6d4833c94e462c6881dfa INFO: 172.21.0.1:36170 - "POST /node/68d6d4833c94e462c6881dfa HTTP/1.0" 202 Accepted 2025-09-26 18:04:19,535 - lava_callback - INFO - Starting async job submit for node 68d6d4833c94e462c6881dfa 2025-09-26 18:04:19,555 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:04:19,567 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6d4833c94e462c6881dfa HTTP/1.1" 200 None 2025-09-26 18:04:19,568 - lava_callback - DEBUG - Processing node 68d6d4833c94e462c6881dfa: baseline-arm64-baylibre 2025-09-26 18:04:19,824 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 18:04:19,943 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 18:04:19,944 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-baylibre-68d6d4833c94e462c6881dfa/log.txt.gz 2025-09-26 18:04:19,951 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 18:04:20,060 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 18:04:20,061 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-baylibre-68d6d4833c94e462c6881dfa/lava_callback.json.gz 2025-09-26 18:04:20,083 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:04:20,095 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6d4833c94e462c6881dfa HTTP/1.1" 200 None 2025-09-26 18:04:20,096 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:04:20,116 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6bd8e3c94e462c68814f1 HTTP/1.1" 200 None 2025-09-26 18:04:20,118 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:04:20,152 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6d4833c94e462c6881dfa HTTP/1.1" 200 None 2025-09-26 18:04:20,153 - lava_callback - INFO - Completed processing callback for node 68d6d4833c94e462c6881dfa INFO: 172.21.0.1:36176 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60264 - "GET / HTTP/1.0" 200 OK 2025-09-26 18:04:32,574 - lava_callback - INFO - Received callback for node 68d4329ea5c2c05277bc869a INFO: 172.21.0.1:60274 - "POST /node/68d4329ea5c2c05277bc869a HTTP/1.0" 202 Accepted 2025-09-26 18:04:32,580 - lava_callback - INFO - Starting async job submit for node 68d4329ea5c2c05277bc869a 2025-09-26 18:04:32,581 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:04:32,594 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4329ea5c2c05277bc869a HTTP/1.1" 200 None 2025-09-26 18:04:32,594 - lava_callback - DEBUG - Processing node 68d4329ea5c2c05277bc869a: ltp-syscalls 2025-09-26 18:04:32,594 - lava_callback - ERROR - Error processing callback for node 68d4329ea5c2c05277bc869a: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 18:04:33,317 - lava_callback - INFO - Received callback for node 68d432a1a5c2c05277bc869d INFO: 172.21.0.1:60278 - "POST /node/68d432a1a5c2c05277bc869d HTTP/1.0" 202 Accepted 2025-09-26 18:04:33,323 - lava_callback - INFO - Starting async job submit for node 68d432a1a5c2c05277bc869d 2025-09-26 18:04:33,324 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:04:33,336 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d432a1a5c2c05277bc869d HTTP/1.1" 200 None 2025-09-26 18:04:33,337 - lava_callback - DEBUG - Processing node 68d432a1a5c2c05277bc869d: ltp-syscalls-ipc 2025-09-26 18:04:33,337 - lava_callback - ERROR - Error processing callback for node 68d432a1a5c2c05277bc869d: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 18:04:33,698 - lava_callback - INFO - Received callback for node 68d6d47f3c94e462c6881ddc INFO: 172.21.0.1:60286 - "POST /node/68d6d47f3c94e462c6881ddc HTTP/1.0" 202 Accepted 2025-09-26 18:04:33,704 - lava_callback - INFO - Starting async job submit for node 68d6d47f3c94e462c6881ddc 2025-09-26 18:04:33,724 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:04:33,746 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6d47f3c94e462c6881ddc HTTP/1.1" 200 None 2025-09-26 18:04:33,746 - lava_callback - DEBUG - Processing node 68d6d47f3c94e462c6881ddc: baseline-arm64 2025-09-26 18:04:33,988 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 18:04:34,124 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 18:04:34,125 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6d47f3c94e462c6881ddc/log.txt.gz 2025-09-26 18:04:34,131 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 18:04:34,163 - lava_callback - INFO - Received callback for node 68d432a3a5c2c05277bc86a0 INFO: 172.21.0.1:60300 - "POST /node/68d432a3a5c2c05277bc86a0 HTTP/1.0" 202 Accepted 2025-09-26 18:04:34,172 - lava_callback - INFO - Starting async job submit for node 68d432a3a5c2c05277bc86a0 2025-09-26 18:04:34,174 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:04:34,186 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d432a3a5c2c05277bc86a0 HTTP/1.1" 200 None 2025-09-26 18:04:34,186 - lava_callback - DEBUG - Processing node 68d432a3a5c2c05277bc86a0: ltp-timers 2025-09-26 18:04:34,186 - lava_callback - ERROR - Error processing callback for node 68d432a3a5c2c05277bc86a0: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 18:04:34,274 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 18:04:34,275 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6d47f3c94e462c6881ddc/lava_callback.json.gz 2025-09-26 18:04:34,296 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:04:34,307 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6d47f3c94e462c6881ddc HTTP/1.1" 200 None 2025-09-26 18:04:34,308 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:04:34,320 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6bd8e3c94e462c68814f1 HTTP/1.1" 200 None 2025-09-26 18:04:34,321 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:04:34,349 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6d47f3c94e462c6881ddc HTTP/1.1" 200 None 2025-09-26 18:04:34,350 - lava_callback - INFO - Completed processing callback for node 68d6d47f3c94e462c6881ddc 2025-09-26 18:04:34,941 - lava_callback - INFO - Received callback for node 68d432a6a5c2c05277bc86a3 INFO: 172.21.0.1:60302 - "POST /node/68d432a6a5c2c05277bc86a3 HTTP/1.0" 202 Accepted 2025-09-26 18:04:34,947 - lava_callback - INFO - Starting async job submit for node 68d432a6a5c2c05277bc86a3 2025-09-26 18:04:34,948 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:04:34,960 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d432a6a5c2c05277bc86a3 HTTP/1.1" 200 None 2025-09-26 18:04:34,960 - lava_callback - DEBUG - Processing node 68d432a6a5c2c05277bc86a3: ltp-watchqueue 2025-09-26 18:04:34,960 - lava_callback - ERROR - Error processing callback for node 68d432a6a5c2c05277bc86a3: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:60306 - "GET / HTTP/1.0" 200 OK 2025-09-26 18:04:44,426 - lava_callback - INFO - Received callback for node 68d6d47d3c94e462c6881dd3 INFO: 172.21.0.1:46208 - "POST /node/68d6d47d3c94e462c6881dd3 HTTP/1.0" 202 Accepted 2025-09-26 18:04:44,432 - lava_callback - INFO - Starting async job submit for node 68d6d47d3c94e462c6881dd3 2025-09-26 18:04:44,453 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:04:44,465 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6d47d3c94e462c6881dd3 HTTP/1.1" 200 None 2025-09-26 18:04:44,466 - lava_callback - DEBUG - Processing node 68d6d47d3c94e462c6881dd3: baseline-arm64 2025-09-26 18:04:44,815 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 18:04:44,928 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 18:04:44,928 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6d47d3c94e462c6881dd3/log.txt.gz 2025-09-26 18:04:44,936 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 18:04:45,045 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 18:04:45,046 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6d47d3c94e462c6881dd3/lava_callback.json.gz 2025-09-26 18:04:45,067 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:04:45,079 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6d47d3c94e462c6881dd3 HTTP/1.1" 200 None 2025-09-26 18:04:45,080 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:04:45,092 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6bd8e3c94e462c68814f1 HTTP/1.1" 200 None 2025-09-26 18:04:45,093 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:04:45,116 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6d47d3c94e462c6881dd3 HTTP/1.1" 200 None 2025-09-26 18:04:45,117 - lava_callback - INFO - Completed processing callback for node 68d6d47d3c94e462c6881dd3 INFO: 172.21.0.1:46220 - "GET / HTTP/1.0" 200 OK 2025-09-26 18:04:47,589 - lava_callback - INFO - Received callback for node 68d6d47b3c94e462c6881dc1 INFO: 172.21.0.1:46228 - "POST /node/68d6d47b3c94e462c6881dc1 HTTP/1.0" 202 Accepted 2025-09-26 18:04:47,595 - lava_callback - INFO - Starting async job submit for node 68d6d47b3c94e462c6881dc1 2025-09-26 18:04:47,616 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:04:47,636 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6d47b3c94e462c6881dc1 HTTP/1.1" 200 None 2025-09-26 18:04:47,637 - lava_callback - DEBUG - Processing node 68d6d47b3c94e462c6881dc1: baseline-arm64 2025-09-26 18:04:47,890 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 18:04:47,972 - lava_callback - INFO - Received callback for node 68d6d4813c94e462c6881deb INFO: 172.21.0.1:46234 - "POST /node/68d6d4813c94e462c6881deb HTTP/1.0" 202 Accepted 2025-09-26 18:04:47,979 - lava_callback - INFO - Starting async job submit for node 68d6d4813c94e462c6881deb 2025-09-26 18:04:48,004 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:04:48,004 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 18:04:48,005 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6d47b3c94e462c6881dc1/log.txt.gz 2025-09-26 18:04:48,012 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 18:04:48,020 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6d4813c94e462c6881deb HTTP/1.1" 200 None 2025-09-26 18:04:48,021 - lava_callback - DEBUG - Processing node 68d6d4813c94e462c6881deb: baseline-arm64 2025-09-26 18:04:48,153 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 18:04:48,195 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6d47b3c94e462c6881dc1/lava_callback.json.gz 2025-09-26 18:04:48,231 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:04:48,303 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6d47b3c94e462c6881dc1 HTTP/1.1" 200 None 2025-09-26 18:04:48,325 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:04:48,336 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 18:04:48,346 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6bd8e3c94e462c68814f1 HTTP/1.1" 200 None 2025-09-26 18:04:48,347 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:04:48,380 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6d47b3c94e462c6881dc1 HTTP/1.1" 200 None 2025-09-26 18:04:48,381 - lava_callback - INFO - Completed processing callback for node 68d6d47b3c94e462c6881dc1 2025-09-26 18:04:48,595 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 18:04:48,596 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6d4813c94e462c6881deb/log.txt.gz 2025-09-26 18:04:48,604 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 18:04:48,682 - lava_callback - INFO - Received callback for node 68d6d4853c94e462c6881e06 INFO: 172.21.0.1:46250 - "POST /node/68d6d4853c94e462c6881e06 HTTP/1.0" 202 Accepted 2025-09-26 18:04:48,691 - lava_callback - INFO - Starting async job submit for node 68d6d4853c94e462c6881e06 2025-09-26 18:04:48,725 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:04:48,737 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 18:04:48,738 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6d4813c94e462c6881deb/lava_callback.json.gz 2025-09-26 18:04:48,764 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6d4853c94e462c6881e06 HTTP/1.1" 200 None 2025-09-26 18:04:48,765 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:04:48,766 - lava_callback - DEBUG - Processing node 68d6d4853c94e462c6881e06: baseline-arm64-baylibre 2025-09-26 18:04:48,867 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6d4813c94e462c6881deb HTTP/1.1" 200 None 2025-09-26 18:04:48,884 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:04:48,950 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6bd8e3c94e462c68814f1 HTTP/1.1" 200 None 2025-09-26 18:04:48,971 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:04:49,080 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6d4813c94e462c6881deb HTTP/1.1" 200 None 2025-09-26 18:04:49,080 - lava_callback - INFO - Completed processing callback for node 68d6d4813c94e462c6881deb 2025-09-26 18:04:49,106 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 18:04:49,231 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 18:04:49,232 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-baylibre-68d6d4853c94e462c6881e06/log.txt.gz 2025-09-26 18:04:49,239 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 18:04:49,371 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 18:04:49,372 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-baylibre-68d6d4853c94e462c6881e06/lava_callback.json.gz 2025-09-26 18:04:49,410 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:04:49,430 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6d4853c94e462c6881e06 HTTP/1.1" 200 None 2025-09-26 18:04:49,431 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:04:49,443 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6bd8e3c94e462c68814f1 HTTP/1.1" 200 None 2025-09-26 18:04:49,444 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:04:49,474 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6d4853c94e462c6881e06 HTTP/1.1" 200 None 2025-09-26 18:04:49,475 - lava_callback - INFO - Completed processing callback for node 68d6d4853c94e462c6881e06 2025-09-26 18:04:54,289 - lava_callback - INFO - Received callback for node 68d6d47a3c94e462c6881dbe INFO: 172.21.0.1:52732 - "POST /node/68d6d47a3c94e462c6881dbe HTTP/1.0" 202 Accepted 2025-09-26 18:04:54,295 - lava_callback - INFO - Starting async job submit for node 68d6d47a3c94e462c6881dbe 2025-09-26 18:04:54,315 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:04:54,327 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6d47a3c94e462c6881dbe HTTP/1.1" 200 None 2025-09-26 18:04:54,328 - lava_callback - DEBUG - Processing node 68d6d47a3c94e462c6881dbe: baseline-arm64 2025-09-26 18:04:54,564 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 18:04:54,682 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 18:04:54,683 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6d47a3c94e462c6881dbe/log.txt.gz 2025-09-26 18:04:54,689 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 18:04:54,806 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 18:04:54,807 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6d47a3c94e462c6881dbe/lava_callback.json.gz 2025-09-26 18:04:54,828 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:04:54,842 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6d47a3c94e462c6881dbe HTTP/1.1" 200 None 2025-09-26 18:04:54,843 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:04:54,854 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6bd8e3c94e462c68814f1 HTTP/1.1" 200 None 2025-09-26 18:04:54,855 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:04:54,890 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6d47a3c94e462c6881dbe HTTP/1.1" 200 None 2025-09-26 18:04:54,891 - lava_callback - INFO - Completed processing callback for node 68d6d47a3c94e462c6881dbe 2025-09-26 18:04:55,177 - lava_callback - INFO - Received callback for node 68d6d47e3c94e462c6881dd9 INFO: 172.21.0.1:52746 - "POST /node/68d6d47e3c94e462c6881dd9 HTTP/1.0" 202 Accepted 2025-09-26 18:04:55,183 - lava_callback - INFO - Starting async job submit for node 68d6d47e3c94e462c6881dd9 2025-09-26 18:04:55,204 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:04:55,216 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6d47e3c94e462c6881dd9 HTTP/1.1" 200 None 2025-09-26 18:04:55,217 - lava_callback - DEBUG - Processing node 68d6d47e3c94e462c6881dd9: baseline-arm64 2025-09-26 18:04:55,449 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 18:04:55,571 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 18:04:55,572 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6d47e3c94e462c6881dd9/log.txt.gz 2025-09-26 18:04:55,578 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 18:04:55,681 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 18:04:55,682 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6d47e3c94e462c6881dd9/lava_callback.json.gz 2025-09-26 18:04:55,704 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:04:55,718 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6d47e3c94e462c6881dd9 HTTP/1.1" 200 None 2025-09-26 18:04:55,719 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:04:55,733 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6bd8e3c94e462c68814f1 HTTP/1.1" 200 None 2025-09-26 18:04:55,734 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:04:55,763 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6d47e3c94e462c6881dd9 HTTP/1.1" 200 None 2025-09-26 18:04:55,764 - lava_callback - INFO - Completed processing callback for node 68d6d47e3c94e462c6881dd9 INFO: 172.21.0.1:52762 - "GET / HTTP/1.0" 200 OK 2025-09-26 18:04:57,727 - lava_callback - INFO - Received callback for node 68d6d4813c94e462c6881dee INFO: 172.21.0.1:52768 - "POST /node/68d6d4813c94e462c6881dee HTTP/1.0" 202 Accepted 2025-09-26 18:04:57,733 - lava_callback - INFO - Starting async job submit for node 68d6d4813c94e462c6881dee 2025-09-26 18:04:57,774 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:04:57,786 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6d4813c94e462c6881dee HTTP/1.1" 200 None 2025-09-26 18:04:57,786 - lava_callback - DEBUG - Processing node 68d6d4813c94e462c6881dee: baseline-arm64 2025-09-26 18:04:58,070 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 18:04:58,184 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 18:04:58,185 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6d4813c94e462c6881dee/log.txt.gz 2025-09-26 18:04:58,192 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 18:04:58,316 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 18:04:58,316 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6d4813c94e462c6881dee/lava_callback.json.gz 2025-09-26 18:04:58,337 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:04:58,350 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6d4813c94e462c6881dee HTTP/1.1" 200 None 2025-09-26 18:04:58,351 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:04:58,363 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6bd8e3c94e462c68814f1 HTTP/1.1" 200 None 2025-09-26 18:04:58,364 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:04:58,396 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6d4813c94e462c6881dee HTTP/1.1" 200 None 2025-09-26 18:04:58,397 - lava_callback - INFO - Completed processing callback for node 68d6d4813c94e462c6881dee 2025-09-26 18:04:59,401 - lava_callback - INFO - Received callback for node 68d6d47d3c94e462c6881dcd INFO: 172.21.0.1:59672 - "POST /node/68d6d47d3c94e462c6881dcd HTTP/1.0" 202 Accepted 2025-09-26 18:04:59,407 - lava_callback - INFO - Starting async job submit for node 68d6d47d3c94e462c6881dcd 2025-09-26 18:04:59,426 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:04:59,439 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6d47d3c94e462c6881dcd HTTP/1.1" 200 None 2025-09-26 18:04:59,440 - lava_callback - DEBUG - Processing node 68d6d47d3c94e462c6881dcd: baseline-arm64 2025-09-26 18:04:59,775 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 18:04:59,887 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 18:04:59,888 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6d47d3c94e462c6881dcd/log.txt.gz 2025-09-26 18:04:59,897 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 18:05:00,003 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 18:05:00,004 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6d47d3c94e462c6881dcd/lava_callback.json.gz 2025-09-26 18:05:00,024 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:05:00,036 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6d47d3c94e462c6881dcd HTTP/1.1" 200 None 2025-09-26 18:05:00,037 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:05:00,051 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6bd8e3c94e462c68814f1 HTTP/1.1" 200 None 2025-09-26 18:05:00,052 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:05:00,090 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6d47d3c94e462c6881dcd HTTP/1.1" 200 None 2025-09-26 18:05:00,091 - lava_callback - INFO - Completed processing callback for node 68d6d47d3c94e462c6881dcd 2025-09-26 18:05:01,122 - lava_callback - INFO - Received callback for node 68d6d47d3c94e462c6881dd0 INFO: 172.21.0.1:59676 - "POST /node/68d6d47d3c94e462c6881dd0 HTTP/1.0" 202 Accepted 2025-09-26 18:05:01,128 - lava_callback - INFO - Starting async job submit for node 68d6d47d3c94e462c6881dd0 2025-09-26 18:05:01,149 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:05:01,161 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6d47d3c94e462c6881dd0 HTTP/1.1" 200 None 2025-09-26 18:05:01,162 - lava_callback - DEBUG - Processing node 68d6d47d3c94e462c6881dd0: baseline-arm64 2025-09-26 18:05:01,458 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 18:05:01,588 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 18:05:01,589 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6d47d3c94e462c6881dd0/log.txt.gz 2025-09-26 18:05:01,597 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 18:05:01,731 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 18:05:01,731 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6d47d3c94e462c6881dd0/lava_callback.json.gz 2025-09-26 18:05:01,752 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:05:01,764 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6d47d3c94e462c6881dd0 HTTP/1.1" 200 None 2025-09-26 18:05:01,765 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:05:01,776 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6bd8e3c94e462c68814f1 HTTP/1.1" 200 None 2025-09-26 18:05:01,777 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:05:01,805 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6d47d3c94e462c6881dd0 HTTP/1.1" 200 None 2025-09-26 18:05:01,807 - lava_callback - INFO - Completed processing callback for node 68d6d47d3c94e462c6881dd0 INFO: 172.21.0.1:59690 - "GET / HTTP/1.0" 200 OK 2025-09-26 18:05:11,306 - lava_callback - INFO - Received callback for node 68d43b9ba5c2c05277bc9556 INFO: 172.21.0.1:41880 - "POST /node/68d43b9ba5c2c05277bc9556 HTTP/1.0" 202 Accepted 2025-09-26 18:05:11,315 - lava_callback - INFO - Starting async job submit for node 68d43b9ba5c2c05277bc9556 2025-09-26 18:05:11,343 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:05:11,356 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d43b9ba5c2c05277bc9556 HTTP/1.1" 200 None 2025-09-26 18:05:11,357 - lava_callback - DEBUG - Processing node 68d43b9ba5c2c05277bc9556: baseline-arm64 2025-09-26 18:05:11,654 - lava_callback - ERROR - Error processing callback for node 68d43b9ba5c2c05277bc9556: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None 2025-09-26 18:05:12,690 - lava_callback - INFO - Received callback for node 68d6d4803c94e462c6881de8 INFO: 172.21.0.1:41896 - "POST /node/68d6d4803c94e462c6881de8 HTTP/1.0" 202 Accepted 2025-09-26 18:05:12,696 - lava_callback - INFO - Starting async job submit for node 68d6d4803c94e462c6881de8 2025-09-26 18:05:12,715 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:05:12,727 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6d4803c94e462c6881de8 HTTP/1.1" 200 None 2025-09-26 18:05:12,728 - lava_callback - DEBUG - Processing node 68d6d4803c94e462c6881de8: baseline-arm64 2025-09-26 18:05:12,755 - lava_callback - INFO - Received callback for node 68d6c7833c94e462c6881877 INFO: 172.21.0.1:41904 - "POST /node/68d6c7833c94e462c6881877 HTTP/1.0" 202 Accepted 2025-09-26 18:05:12,787 - lava_callback - INFO - Starting async job submit for node 68d6c7833c94e462c6881877 2025-09-26 18:05:12,844 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:05:12,939 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6c7833c94e462c6881877 HTTP/1.1" 200 None 2025-09-26 18:05:12,954 - lava_callback - DEBUG - Processing node 68d6c7833c94e462c6881877: baseline-arm64 2025-09-26 18:05:13,355 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 18:05:13,597 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 18:05:13,598 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6d4803c94e462c6881de8/log.txt.gz 2025-09-26 18:05:13,601 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 18:05:13,607 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 18:05:13,940 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 18:05:13,941 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 18:05:13,942 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6d4803c94e462c6881de8/lava_callback.json.gz 2025-09-26 18:05:13,942 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6c7833c94e462c6881877/log.txt.gz 2025-09-26 18:05:13,962 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:05:13,971 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 18:05:13,996 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6d4803c94e462c6881de8 HTTP/1.1" 200 None 2025-09-26 18:05:13,997 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:05:14,023 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6bd8e3c94e462c68814f1 HTTP/1.1" 200 None 2025-09-26 18:05:14,024 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:05:14,087 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6d4803c94e462c6881de8 HTTP/1.1" 200 None 2025-09-26 18:05:14,088 - lava_callback - INFO - Completed processing callback for node 68d6d4803c94e462c6881de8 2025-09-26 18:05:14,147 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 18:05:14,148 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6c7833c94e462c6881877/lava_callback.json.gz 2025-09-26 18:05:14,178 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:05:14,202 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6c7833c94e462c6881877 HTTP/1.1" 200 None 2025-09-26 18:05:14,203 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:05:14,215 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6b9dd2f83206171a92b48 HTTP/1.1" 200 None 2025-09-26 18:05:14,216 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:05:14,250 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6c7833c94e462c6881877 HTTP/1.1" 200 None 2025-09-26 18:05:14,251 - lava_callback - INFO - Completed processing callback for node 68d6c7833c94e462c6881877 2025-09-26 18:05:14,281 - lava_callback - INFO - Received callback for node 68d6d4863c94e462c6881e0f INFO: 172.21.0.1:41914 - "POST /node/68d6d4863c94e462c6881e0f HTTP/1.0" 202 Accepted 2025-09-26 18:05:14,293 - lava_callback - INFO - Starting async job submit for node 68d6d4863c94e462c6881e0f 2025-09-26 18:05:14,321 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:05:14,336 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6d4863c94e462c6881e0f HTTP/1.1" 200 None 2025-09-26 18:05:14,337 - lava_callback - DEBUG - Processing node 68d6d4863c94e462c6881e0f: baseline-arm64-baylibre 2025-09-26 18:05:14,644 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 18:05:14,759 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 18:05:14,760 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-baylibre-68d6d4863c94e462c6881e0f/log.txt.gz 2025-09-26 18:05:14,767 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 18:05:14,887 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 18:05:14,888 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-baylibre-68d6d4863c94e462c6881e0f/lava_callback.json.gz 2025-09-26 18:05:14,909 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:05:14,925 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6d4863c94e462c6881e0f HTTP/1.1" 200 None 2025-09-26 18:05:14,926 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:05:14,937 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6bd8e3c94e462c68814f1 HTTP/1.1" 200 None 2025-09-26 18:05:14,938 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:05:14,964 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6d4863c94e462c6881e0f HTTP/1.1" 200 None 2025-09-26 18:05:14,965 - lava_callback - INFO - Completed processing callback for node 68d6d4863c94e462c6881e0f INFO: 172.21.0.1:41928 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41934 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41946 - "GET / HTTP/1.0" 200 OK 2025-09-26 18:05:25,115 - lava_callback - INFO - Received callback for node 68d43b98a5c2c05277bc9553 INFO: 172.21.0.1:56728 - "POST /node/68d43b98a5c2c05277bc9553 HTTP/1.0" 202 Accepted 2025-09-26 18:05:25,122 - lava_callback - INFO - Starting async job submit for node 68d43b98a5c2c05277bc9553 2025-09-26 18:05:25,142 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:05:25,155 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d43b98a5c2c05277bc9553 HTTP/1.1" 200 None 2025-09-26 18:05:25,156 - lava_callback - DEBUG - Processing node 68d43b98a5c2c05277bc9553: baseline-arm64 2025-09-26 18:05:25,467 - lava_callback - ERROR - Error processing callback for node 68d43b98a5c2c05277bc9553: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:56732 - "GET / HTTP/1.0" 200 OK 2025-09-26 18:05:26,922 - lava_callback - INFO - Received callback for node 68d43b97a5c2c05277bc9550 INFO: 172.21.0.1:56742 - "POST /node/68d43b97a5c2c05277bc9550 HTTP/1.0" 202 Accepted 2025-09-26 18:05:26,928 - lava_callback - INFO - Starting async job submit for node 68d43b97a5c2c05277bc9550 2025-09-26 18:05:26,949 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:05:26,961 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d43b97a5c2c05277bc9550 HTTP/1.1" 200 None 2025-09-26 18:05:26,961 - lava_callback - DEBUG - Processing node 68d43b97a5c2c05277bc9550: baseline-arm64 2025-09-26 18:05:27,264 - lava_callback - ERROR - Error processing callback for node 68d43b97a5c2c05277bc9550: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:56268 - "GET / HTTP/1.0" 200 OK 2025-09-26 18:05:29,883 - lava_callback - INFO - Received callback for node 68d4a9bfa5c2c05277bcf7b4 INFO: 172.21.0.1:56274 - "POST /node/68d4a9bfa5c2c05277bcf7b4 HTTP/1.0" 202 Accepted 2025-09-26 18:05:29,889 - lava_callback - INFO - Starting async job submit for node 68d4a9bfa5c2c05277bcf7b4 2025-09-26 18:05:29,910 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:05:29,924 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4a9bfa5c2c05277bcf7b4 HTTP/1.1" 200 None 2025-09-26 18:05:29,924 - lava_callback - DEBUG - Processing node 68d4a9bfa5c2c05277bcf7b4: baseline-arm 2025-09-26 18:05:30,184 - lava_callback - ERROR - Error processing callback for node 68d4a9bfa5c2c05277bcf7b4: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:56286 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51698 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36422 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53930 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48632 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48638 - "GET / HTTP/1.0" 200 OK 2025-09-26 18:06:17,330 - lava_callback - INFO - Received callback for node 68d6d1a73c94e462c6881dab INFO: 172.21.0.1:48652 - "POST /node/68d6d1a73c94e462c6881dab HTTP/1.0" 202 Accepted 2025-09-26 18:06:17,337 - lava_callback - INFO - Starting async job submit for node 68d6d1a73c94e462c6881dab 2025-09-26 18:06:17,380 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:06:17,393 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6d1a73c94e462c6881dab HTTP/1.1" 200 None 2025-09-26 18:06:17,393 - lava_callback - DEBUG - Processing node 68d6d1a73c94e462c6881dab: baseline-x86 INFO: 172.21.0.1:48666 - "GET / HTTP/1.0" 200 OK 2025-09-26 18:06:18,018 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 18:06:18,154 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 18:06:18,155 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-68d6d1a73c94e462c6881dab/log.txt.gz 2025-09-26 18:06:18,172 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 18:06:18,356 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 18:06:18,357 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-68d6d1a73c94e462c6881dab/lava_callback.json.gz DEBUG: Setup failed for 68d6d1a73c94e462c6881dab. Transitting job node result: incomplete -> fail 2025-09-26 18:06:18,443 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:06:18,484 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6d1a73c94e462c6881dab HTTP/1.1" 200 None 2025-09-26 18:06:18,485 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:06:18,496 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6bda33c94e462c6881504 HTTP/1.1" 200 None 2025-09-26 18:06:18,497 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:06:18,527 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6d1a73c94e462c6881dab HTTP/1.1" 200 None 2025-09-26 18:06:18,528 - lava_callback - INFO - Completed processing callback for node 68d6d1a73c94e462c6881dab 2025-09-26 18:06:20,551 - lava_callback - INFO - Received callback for node 68d6d5913c94e462c6881ebd INFO: 172.21.0.1:55236 - "POST /node/68d6d5913c94e462c6881ebd HTTP/1.0" 202 Accepted 2025-09-26 18:06:20,557 - lava_callback - INFO - Starting async job submit for node 68d6d5913c94e462c6881ebd 2025-09-26 18:06:20,626 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:06:20,638 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6d5913c94e462c6881ebd HTTP/1.1" 200 None 2025-09-26 18:06:20,639 - lava_callback - DEBUG - Processing node 68d6d5913c94e462c6881ebd: baseline-arm64-kontron 2025-09-26 18:06:20,939 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 18:06:21,090 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 18:06:21,091 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-kontron-68d6d5913c94e462c6881ebd/log.txt.gz 2025-09-26 18:06:21,098 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 18:06:21,250 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 18:06:21,251 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-kontron-68d6d5913c94e462c6881ebd/lava_callback.json.gz 2025-09-26 18:06:21,292 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:06:21,306 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6d5913c94e462c6881ebd HTTP/1.1" 200 None 2025-09-26 18:06:21,307 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:06:21,321 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6bd8e3c94e462c68814f1 HTTP/1.1" 200 None 2025-09-26 18:06:21,323 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:06:21,362 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6d5913c94e462c6881ebd HTTP/1.1" 200 None 2025-09-26 18:06:21,364 - lava_callback - INFO - Completed processing callback for node 68d6d5913c94e462c6881ebd INFO: 172.21.0.1:55240 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60328 - "GET / HTTP/1.0" 200 OK 2025-09-26 18:06:31,781 - lava_callback - INFO - Received callback for node 68d43baea5c2c05277bc9577 INFO: 172.21.0.1:60342 - "POST /node/68d43baea5c2c05277bc9577 HTTP/1.0" 202 Accepted 2025-09-26 18:06:31,787 - lava_callback - INFO - Starting async job submit for node 68d43baea5c2c05277bc9577 2025-09-26 18:06:31,808 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:06:31,820 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d43baea5c2c05277bc9577 HTTP/1.1" 200 None 2025-09-26 18:06:31,820 - lava_callback - DEBUG - Processing node 68d43baea5c2c05277bc9577: baseline-arm64 2025-09-26 18:06:32,115 - lava_callback - ERROR - Error processing callback for node 68d43baea5c2c05277bc9577: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:60356 - "GET / HTTP/1.0" 200 OK 2025-09-26 18:06:46,162 - lava_callback - INFO - Received callback for node 68d6c7873c94e462c6881892 INFO: 172.21.0.1:57348 - "POST /node/68d6c7873c94e462c6881892 HTTP/1.0" 202 Accepted 2025-09-26 18:06:46,168 - lava_callback - INFO - Starting async job submit for node 68d6c7873c94e462c6881892 2025-09-26 18:06:46,188 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:06:46,201 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6c7873c94e462c6881892 HTTP/1.1" 200 None 2025-09-26 18:06:46,202 - lava_callback - DEBUG - Processing node 68d6c7873c94e462c6881892: baseline-arm64 INFO: 172.21.0.1:57362 - "GET / HTTP/1.0" 200 OK 2025-09-26 18:06:46,626 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 18:06:46,771 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 18:06:46,772 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6c7873c94e462c6881892/log.txt.gz 2025-09-26 18:06:46,781 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 18:06:46,900 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 18:06:46,901 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6c7873c94e462c6881892/lava_callback.json.gz 2025-09-26 18:06:46,922 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:06:46,934 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6c7873c94e462c6881892 HTTP/1.1" 200 None 2025-09-26 18:06:46,935 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:06:46,949 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6b9dd2f83206171a92b48 HTTP/1.1" 200 None 2025-09-26 18:06:46,950 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:06:46,979 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6c7873c94e462c6881892 HTTP/1.1" 200 None 2025-09-26 18:06:46,980 - lava_callback - INFO - Completed processing callback for node 68d6c7873c94e462c6881892 2025-09-26 18:06:47,603 - lava_callback - INFO - Received callback for node 68d43b9da5c2c05277bc9559 INFO: 172.21.0.1:57374 - "POST /node/68d43b9da5c2c05277bc9559 HTTP/1.0" 202 Accepted 2025-09-26 18:06:47,609 - lava_callback - INFO - Starting async job submit for node 68d43b9da5c2c05277bc9559 2025-09-26 18:06:47,630 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:06:47,644 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d43b9da5c2c05277bc9559 HTTP/1.1" 200 None 2025-09-26 18:06:47,644 - lava_callback - DEBUG - Processing node 68d43b9da5c2c05277bc9559: baseline-arm64 2025-09-26 18:06:47,940 - lava_callback - ERROR - Error processing callback for node 68d43b9da5c2c05277bc9559: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None 2025-09-26 18:06:49,673 - lava_callback - INFO - Received callback for node 68d6cde83c94e462c6881d39 INFO: 172.21.0.1:42998 - "POST /node/68d6cde83c94e462c6881d39 HTTP/1.0" 202 Accepted 2025-09-26 18:06:49,682 - lava_callback - INFO - Starting async job submit for node 68d6cde83c94e462c6881d39 2025-09-26 18:06:49,706 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:06:49,723 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6cde83c94e462c6881d39 HTTP/1.1" 200 None 2025-09-26 18:06:49,723 - lava_callback - DEBUG - Processing node 68d6cde83c94e462c6881d39: baseline-x86 2025-09-26 18:06:50,022 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 18:06:50,184 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 18:06:50,184 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-68d6cde83c94e462c6881d39/log.txt.gz 2025-09-26 18:06:50,192 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 18:06:50,301 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 18:06:50,302 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-68d6cde83c94e462c6881d39/lava_callback.json.gz 2025-09-26 18:06:50,319 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:06:50,331 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6cde83c94e462c6881d39 HTTP/1.1" 200 None 2025-09-26 18:06:50,332 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:06:50,343 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6bda33c94e462c6881504 HTTP/1.1" 200 None 2025-09-26 18:06:50,344 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:06:50,368 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6cde83c94e462c6881d39 HTTP/1.1" 200 None 2025-09-26 18:06:50,369 - lava_callback - INFO - Completed processing callback for node 68d6cde83c94e462c6881d39 2025-09-26 18:06:50,422 - lava_callback - INFO - Received callback for node 68d43baba5c2c05277bc9562 INFO: 172.21.0.1:43002 - "POST /node/68d43baba5c2c05277bc9562 HTTP/1.0" 202 Accepted 2025-09-26 18:06:50,430 - lava_callback - INFO - Starting async job submit for node 68d43baba5c2c05277bc9562 2025-09-26 18:06:50,477 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:06:50,500 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d43baba5c2c05277bc9562 HTTP/1.1" 200 None 2025-09-26 18:06:50,500 - lava_callback - DEBUG - Processing node 68d43baba5c2c05277bc9562: baseline-arm64 2025-09-26 18:06:50,835 - lava_callback - ERROR - Error processing callback for node 68d43baba5c2c05277bc9562: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:43008 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54766 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53566 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53568 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53578 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37272 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33490 - "GET / HTTP/1.0" 200 OK 2025-09-26 18:07:29,684 - lava_callback - INFO - Received callback for node 68d4a9c0a5c2c05277bcf7b7 INFO: 172.21.0.1:33498 - "POST /node/68d4a9c0a5c2c05277bcf7b7 HTTP/1.0" 202 Accepted 2025-09-26 18:07:29,690 - lava_callback - INFO - Starting async job submit for node 68d4a9c0a5c2c05277bcf7b7 2025-09-26 18:07:29,710 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:07:29,752 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4a9c0a5c2c05277bcf7b7 HTTP/1.1" 200 None 2025-09-26 18:07:29,752 - lava_callback - DEBUG - Processing node 68d4a9c0a5c2c05277bcf7b7: baseline-arm 2025-09-26 18:07:29,978 - lava_callback - ERROR - Error processing callback for node 68d4a9c0a5c2c05277bcf7b7: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:33508 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50882 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52558 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45696 - "GET / HTTP/1.0" 200 OK 2025-09-26 18:08:12,560 - lava_callback - INFO - Received callback for node 68d43bb9a5c2c05277bc9583 INFO: 172.21.0.1:45014 - "POST /node/68d43bb9a5c2c05277bc9583 HTTP/1.0" 202 Accepted 2025-09-26 18:08:12,566 - lava_callback - INFO - Starting async job submit for node 68d43bb9a5c2c05277bc9583 2025-09-26 18:08:12,587 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:08:12,614 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d43bb9a5c2c05277bc9583 HTTP/1.1" 200 None 2025-09-26 18:08:12,615 - lava_callback - DEBUG - Processing node 68d43bb9a5c2c05277bc9583: baseline-arm64 2025-09-26 18:08:12,937 - lava_callback - ERROR - Error processing callback for node 68d43bb9a5c2c05277bc9583: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:45028 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45040 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45056 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57402 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56300 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56304 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49212 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52228 - "GET / HTTP/1.0" 200 OK 2025-09-26 18:09:00,676 - lava_callback - INFO - Received callback for node 68d6d63d3c94e462c6881f10 INFO: 172.21.0.1:44636 - "POST /node/68d6d63d3c94e462c6881f10 HTTP/1.0" 202 Accepted 2025-09-26 18:09:00,684 - lava_callback - INFO - Starting async job submit for node 68d6d63d3c94e462c6881f10 2025-09-26 18:09:00,724 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:09:00,751 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6d63d3c94e462c6881f10 HTTP/1.1" 200 None 2025-09-26 18:09:00,752 - lava_callback - DEBUG - Processing node 68d6d63d3c94e462c6881f10: baseline-arm64-kontron 2025-09-26 18:09:01,069 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 18:09:01,197 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 18:09:01,197 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-kontron-68d6d63d3c94e462c6881f10/log.txt.gz 2025-09-26 18:09:01,203 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 18:09:01,328 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 18:09:01,328 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-kontron-68d6d63d3c94e462c6881f10/lava_callback.json.gz 2025-09-26 18:09:01,365 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:09:01,380 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6d63d3c94e462c6881f10 HTTP/1.1" 200 None 2025-09-26 18:09:01,381 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:09:01,393 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6bd8e3c94e462c68814f1 HTTP/1.1" 200 None 2025-09-26 18:09:01,395 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:09:01,424 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6d63d3c94e462c6881f10 HTTP/1.1" 200 None 2025-09-26 18:09:01,424 - lava_callback - INFO - Completed processing callback for node 68d6d63d3c94e462c6881f10 INFO: 172.21.0.1:44638 - "GET / HTTP/1.0" 200 OK 2025-09-26 18:09:09,973 - lava_callback - INFO - Received callback for node 68d43bb1a5c2c05277bc957a INFO: 172.21.0.1:54256 - "POST /node/68d43bb1a5c2c05277bc957a HTTP/1.0" 202 Accepted 2025-09-26 18:09:09,979 - lava_callback - INFO - Starting async job submit for node 68d43bb1a5c2c05277bc957a 2025-09-26 18:09:10,001 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:09:10,023 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d43bb1a5c2c05277bc957a HTTP/1.1" 200 None 2025-09-26 18:09:10,023 - lava_callback - DEBUG - Processing node 68d43bb1a5c2c05277bc957a: baseline-arm64 2025-09-26 18:09:10,423 - lava_callback - ERROR - Error processing callback for node 68d43bb1a5c2c05277bc957a: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:54258 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54264 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54274 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52016 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47430 - "GET / HTTP/1.0" 200 OK 2025-09-26 18:09:29,537 - lava_callback - INFO - Received callback for node 68d43bb4a5c2c05277bc957d INFO: 172.21.0.1:47440 - "POST /node/68d43bb4a5c2c05277bc957d HTTP/1.0" 202 Accepted 2025-09-26 18:09:29,614 - lava_callback - INFO - Starting async job submit for node 68d43bb4a5c2c05277bc957d 2025-09-26 18:09:29,640 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:09:29,653 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d43bb4a5c2c05277bc957d HTTP/1.1" 200 None 2025-09-26 18:09:29,653 - lava_callback - DEBUG - Processing node 68d43bb4a5c2c05277bc957d: baseline-arm64 INFO: 172.21.0.1:47446 - "GET / HTTP/1.0" 200 OK 2025-09-26 18:09:45,306 - lava_callback - INFO - Received callback for node 68d43bb9a5c2c05277bc9586 INFO: 172.21.0.1:46928 - "POST /node/68d43bb9a5c2c05277bc9586 HTTP/1.0" 202 Accepted 2025-09-26 18:09:45,337 - lava_callback - INFO - Starting async job submit for node 68d43bb9a5c2c05277bc9586 2025-09-26 18:09:45,377 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:09:45,439 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d43bb9a5c2c05277bc9586 HTTP/1.1" 200 None 2025-09-26 18:09:45,450 - lava_callback - DEBUG - Processing node 68d43bb9a5c2c05277bc9586: baseline-arm64 2025-09-26 18:09:46,154 - lava_callback - ERROR - Error processing callback for node 68d43bb9a5c2c05277bc9586: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:46942 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55674 - "GET / HTTP/1.0" 200 OK 2025-09-26 18:09:59,108 - lava_callback - ERROR - Error processing callback for node 68d43bb4a5c2c05277bc957d: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:60342 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41074 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41088 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41102 - "GET / HTTP/1.0" 200 OK 2025-09-26 18:10:23,226 - lava_callback - INFO - Received callback for node 68d43bbca5c2c05277bc9589 INFO: 172.21.0.1:56832 - "POST /node/68d43bbca5c2c05277bc9589 HTTP/1.0" 202 Accepted 2025-09-26 18:10:23,234 - lava_callback - INFO - Starting async job submit for node 68d43bbca5c2c05277bc9589 2025-09-26 18:10:23,262 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:10:23,278 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d43bbca5c2c05277bc9589 HTTP/1.1" 200 None 2025-09-26 18:10:23,278 - lava_callback - DEBUG - Processing node 68d43bbca5c2c05277bc9589: baseline-arm64 2025-09-26 18:10:23,580 - lava_callback - ERROR - Error processing callback for node 68d43bbca5c2c05277bc9589: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:56844 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53396 - "GET / HTTP/1.0" 200 OK 2025-09-26 18:10:30,089 - lava_callback - INFO - Received callback for node 68d43bbda5c2c05277bc958c INFO: 172.21.0.1:53400 - "POST /node/68d43bbda5c2c05277bc958c HTTP/1.0" 202 Accepted 2025-09-26 18:10:30,095 - lava_callback - INFO - Starting async job submit for node 68d43bbda5c2c05277bc958c 2025-09-26 18:10:30,127 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:10:30,140 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d43bbda5c2c05277bc958c HTTP/1.1" 200 None 2025-09-26 18:10:30,140 - lava_callback - DEBUG - Processing node 68d43bbda5c2c05277bc958c: baseline-arm64 2025-09-26 18:10:30,487 - lava_callback - ERROR - Error processing callback for node 68d43bbda5c2c05277bc958c: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None 2025-09-26 18:10:33,286 - lava_callback - INFO - Received callback for node 68d4326da5c2c05277bc8655 INFO: 172.21.0.1:53414 - "POST /node/68d4326da5c2c05277bc8655 HTTP/1.0" 202 Accepted 2025-09-26 18:10:33,297 - lava_callback - INFO - Starting async job submit for node 68d4326da5c2c05277bc8655 2025-09-26 18:10:33,670 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:10:33,682 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4326da5c2c05277bc8655 HTTP/1.1" 200 None 2025-09-26 18:10:33,682 - lava_callback - DEBUG - Processing node 68d4326da5c2c05277bc8655: ltp-controllers INFO: 172.21.0.1:53420 - "GET / HTTP/1.0" 200 OK 2025-09-26 18:10:36,803 - lava_callback - ERROR - Error processing callback for node 68d4326da5c2c05277bc8655: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:60916 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53824 - "GET / HTTP/1.0" 200 OK 2025-09-26 18:10:56,562 - lava_callback - INFO - Received callback for node 68d6d47b3c94e462c6881dc4 INFO: 172.21.0.1:53830 - "POST /node/68d6d47b3c94e462c6881dc4 HTTP/1.0" 202 Accepted 2025-09-26 18:10:56,569 - lava_callback - INFO - Starting async job submit for node 68d6d47b3c94e462c6881dc4 2025-09-26 18:10:56,590 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:10:56,603 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6d47b3c94e462c6881dc4 HTTP/1.1" 200 None 2025-09-26 18:10:56,603 - lava_callback - DEBUG - Processing node 68d6d47b3c94e462c6881dc4: baseline-arm64 2025-09-26 18:10:56,966 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 18:10:57,160 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 18:10:57,161 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6d47b3c94e462c6881dc4/log.txt.gz 2025-09-26 18:10:57,170 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 18:10:57,324 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 18:10:57,325 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6d47b3c94e462c6881dc4/lava_callback.json.gz 2025-09-26 18:10:57,346 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:10:57,358 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6d47b3c94e462c6881dc4 HTTP/1.1" 200 None 2025-09-26 18:10:57,359 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:10:57,375 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6bd8e3c94e462c68814f1 HTTP/1.1" 200 None 2025-09-26 18:10:57,377 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:10:57,415 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6d47b3c94e462c6881dc4 HTTP/1.1" 200 None 2025-09-26 18:10:57,416 - lava_callback - INFO - Completed processing callback for node 68d6d47b3c94e462c6881dc4 INFO: 172.21.0.1:49024 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42542 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42558 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42570 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55992 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42832 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42838 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48044 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51504 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34072 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35824 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35828 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45846 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45860 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40242 - "GET / HTTP/1.0" 200 OK 2025-09-26 18:12:31,120 - lava_callback - INFO - Received callback for node 68d6d47f3c94e462c6881ddf INFO: 172.21.0.1:40256 - "POST /node/68d6d47f3c94e462c6881ddf HTTP/1.0" 202 Accepted 2025-09-26 18:12:31,126 - lava_callback - INFO - Starting async job submit for node 68d6d47f3c94e462c6881ddf 2025-09-26 18:12:31,146 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:12:31,170 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6d47f3c94e462c6881ddf HTTP/1.1" 200 None 2025-09-26 18:12:31,170 - lava_callback - DEBUG - Processing node 68d6d47f3c94e462c6881ddf: baseline-arm64 2025-09-26 18:12:31,558 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 18:12:31,708 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 18:12:31,708 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6d47f3c94e462c6881ddf/log.txt.gz 2025-09-26 18:12:31,716 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 18:12:31,849 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 18:12:31,850 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6d47f3c94e462c6881ddf/lava_callback.json.gz 2025-09-26 18:12:31,872 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:12:31,884 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6d47f3c94e462c6881ddf HTTP/1.1" 200 None 2025-09-26 18:12:31,885 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:12:31,896 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6bd8e3c94e462c68814f1 HTTP/1.1" 200 None 2025-09-26 18:12:31,897 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:12:31,920 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6d47f3c94e462c6881ddf HTTP/1.1" 200 None 2025-09-26 18:12:31,921 - lava_callback - INFO - Completed processing callback for node 68d6d47f3c94e462c6881ddf INFO: 172.21.0.1:40262 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51042 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40392 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45728 - "GET / HTTP/1.0" 200 OK 2025-09-26 18:13:12,675 - lava_callback - INFO - Received callback for node 68d437fca5c2c05277bc9091 INFO: 172.21.0.1:52110 - "POST /node/68d437fca5c2c05277bc9091 HTTP/1.0" 202 Accepted 2025-09-26 18:13:12,681 - lava_callback - INFO - Starting async job submit for node 68d437fca5c2c05277bc9091 2025-09-26 18:13:12,702 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:13:12,716 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d437fca5c2c05277bc9091 HTTP/1.1" 200 None 2025-09-26 18:13:12,717 - lava_callback - DEBUG - Processing node 68d437fca5c2c05277bc9091: baseline-arm-broonie 2025-09-26 18:13:12,910 - lava_callback - ERROR - Error processing callback for node 68d437fca5c2c05277bc9091: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:52120 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52126 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57090 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57104 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39578 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39592 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47892 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44276 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52592 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52908 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52920 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46704 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46716 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43460 - "GET / HTTP/1.0" 200 OK 2025-09-26 18:14:30,891 - lava_callback - INFO - Received callback for node 68d4c294a5c2c05277bcfc6e INFO: 172.21.0.1:43466 - "POST /node/68d4c294a5c2c05277bcfc6e HTTP/1.0" 202 Accepted 2025-09-26 18:14:30,897 - lava_callback - INFO - Starting async job submit for node 68d4c294a5c2c05277bcfc6e 2025-09-26 18:14:30,918 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:14:30,930 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4c294a5c2c05277bcfc6e HTTP/1.1" 200 None 2025-09-26 18:14:30,931 - lava_callback - DEBUG - Processing node 68d4c294a5c2c05277bcfc6e: baseline-arm 2025-09-26 18:14:31,163 - lava_callback - ERROR - Error processing callback for node 68d4c294a5c2c05277bcfc6e: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:43482 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51544 - "GET / HTTP/1.0" 200 OK 2025-09-26 18:14:54,721 - lava_callback - INFO - Received callback for node 68d6d1693c94e462c6881da4 INFO: 172.21.0.1:45664 - "POST /node/68d6d1693c94e462c6881da4 HTTP/1.0" 202 Accepted 2025-09-26 18:14:54,727 - lava_callback - INFO - Starting async job submit for node 68d6d1693c94e462c6881da4 2025-09-26 18:14:54,757 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:14:54,769 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6d1693c94e462c6881da4 HTTP/1.1" 200 None 2025-09-26 18:14:54,770 - lava_callback - DEBUG - Processing node 68d6d1693c94e462c6881da4: baseline-x86 2025-09-26 18:14:55,084 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 18:14:55,233 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 18:14:55,233 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-68d6d1693c94e462c6881da4/log.txt.gz 2025-09-26 18:14:55,241 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 18:14:55,380 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 18:14:55,381 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-68d6d1693c94e462c6881da4/lava_callback.json.gz DEBUG: Setup failed for 68d6d1693c94e462c6881da4. Transitting job node result: incomplete -> fail 2025-09-26 18:14:55,441 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:14:55,455 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6d1693c94e462c6881da4 HTTP/1.1" 200 None 2025-09-26 18:14:55,456 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:14:55,493 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6bda33c94e462c6881504 HTTP/1.1" 200 None 2025-09-26 18:14:55,494 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:14:55,522 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6d1693c94e462c6881da4 HTTP/1.1" 200 None 2025-09-26 18:14:55,523 - lava_callback - INFO - Completed processing callback for node 68d6d1693c94e462c6881da4 INFO: 172.21.0.1:45680 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39484 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40246 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40248 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60852 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60854 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54764 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54766 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51038 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51744 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37722 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47758 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47766 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49472 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49486 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47230 - "GET / HTTP/1.0" 200 OK 2025-09-26 18:16:32,559 - lava_callback - INFO - Received callback for node 68d4c295a5c2c05277bcfc71 INFO: 172.21.0.1:47240 - "POST /node/68d4c295a5c2c05277bcfc71 HTTP/1.0" 202 Accepted 2025-09-26 18:16:32,565 - lava_callback - INFO - Starting async job submit for node 68d4c295a5c2c05277bcfc71 2025-09-26 18:16:32,585 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:16:32,599 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4c295a5c2c05277bcfc71 HTTP/1.1" 200 None 2025-09-26 18:16:32,599 - lava_callback - DEBUG - Processing node 68d4c295a5c2c05277bcfc71: baseline-arm 2025-09-26 18:16:32,834 - lava_callback - ERROR - Error processing callback for node 68d4c295a5c2c05277bcfc71: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:47246 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39678 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56270 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48456 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52018 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52032 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56892 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56906 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52652 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52654 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39524 - "GET / HTTP/1.0" 200 OK 2025-09-26 18:17:53,053 - lava_callback - INFO - Received callback for node 68d43809a5c2c05277bc90c0 INFO: 172.21.0.1:54530 - "POST /node/68d43809a5c2c05277bc90c0 HTTP/1.0" 202 Accepted 2025-09-26 18:17:53,060 - lava_callback - INFO - Starting async job submit for node 68d43809a5c2c05277bc90c0 2025-09-26 18:17:53,096 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:17:53,128 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d43809a5c2c05277bc90c0 HTTP/1.1" 200 None 2025-09-26 18:17:53,129 - lava_callback - DEBUG - Processing node 68d43809a5c2c05277bc90c0: ltp-cap-bounds 2025-09-26 18:17:53,452 - lava_callback - ERROR - Error processing callback for node 68d43809a5c2c05277bc90c0: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:54534 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49966 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35328 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35334 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48612 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48624 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43322 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43332 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59468 - "GET / HTTP/1.0" 200 OK 2025-09-26 18:18:50,581 - lava_callback - INFO - Received callback for node 68d4610aa5c2c05277bcaf1c INFO: 172.21.0.1:42002 - "POST /node/68d4610aa5c2c05277bcaf1c HTTP/1.0" 202 Accepted 2025-09-26 18:18:50,588 - lava_callback - INFO - Starting async job submit for node 68d4610aa5c2c05277bcaf1c 2025-09-26 18:18:50,608 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:18:50,622 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4610aa5c2c05277bcaf1c HTTP/1.1" 200 None 2025-09-26 18:18:50,622 - lava_callback - DEBUG - Processing node 68d4610aa5c2c05277bcaf1c: baseline-arm64 2025-09-26 18:18:50,941 - lava_callback - ERROR - Error processing callback for node 68d4610aa5c2c05277bcaf1c: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None 2025-09-26 18:18:51,660 - lava_callback - INFO - Received callback for node 68d4c297a5c2c05277bcfc74 INFO: 172.21.0.1:42008 - "POST /node/68d4c297a5c2c05277bcfc74 HTTP/1.0" 202 Accepted 2025-09-26 18:18:51,666 - lava_callback - INFO - Starting async job submit for node 68d4c297a5c2c05277bcfc74 2025-09-26 18:18:51,687 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:18:51,709 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4c297a5c2c05277bcfc74 HTTP/1.1" 200 None 2025-09-26 18:18:51,710 - lava_callback - DEBUG - Processing node 68d4c297a5c2c05277bcfc74: baseline-arm 2025-09-26 18:18:51,933 - lava_callback - ERROR - Error processing callback for node 68d4c297a5c2c05277bcfc74: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:42016 - "GET / HTTP/1.0" 200 OK 2025-09-26 18:19:02,078 - lava_callback - INFO - Received callback for node 68d4610da5c2c05277bcaf1f INFO: 172.21.0.1:39570 - "POST /node/68d4610da5c2c05277bcaf1f HTTP/1.0" 202 Accepted 2025-09-26 18:19:02,084 - lava_callback - INFO - Starting async job submit for node 68d4610da5c2c05277bcaf1f 2025-09-26 18:19:02,105 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:19:02,122 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4610da5c2c05277bcaf1f HTTP/1.1" 200 None 2025-09-26 18:19:02,123 - lava_callback - DEBUG - Processing node 68d4610da5c2c05277bcaf1f: baseline-arm64 2025-09-26 18:19:02,447 - lava_callback - ERROR - Error processing callback for node 68d4610da5c2c05277bcaf1f: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:39576 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39578 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39960 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39964 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45290 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45304 - "GET /favicon.ico HTTP/1.0" 404 Not Found INFO: 172.21.0.1:45314 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56220 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56236 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45252 - "GET / HTTP/1.0" 200 OK 2025-09-26 18:19:53,325 - lava_callback - INFO - Received callback for node 68d46112a5c2c05277bcaf35 INFO: 172.21.0.1:54992 - "POST /node/68d46112a5c2c05277bcaf35 HTTP/1.0" 202 Accepted 2025-09-26 18:19:53,331 - lava_callback - INFO - Starting async job submit for node 68d46112a5c2c05277bcaf35 2025-09-26 18:19:53,352 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:19:53,366 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d46112a5c2c05277bcaf35 HTTP/1.1" 200 None 2025-09-26 18:19:53,366 - lava_callback - DEBUG - Processing node 68d46112a5c2c05277bcaf35: baseline-arm64 2025-09-26 18:19:53,662 - lava_callback - ERROR - Error processing callback for node 68d46112a5c2c05277bcaf35: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:54998 - "GET / HTTP/1.0" 200 OK 2025-09-26 18:20:01,821 - lava_callback - INFO - Received callback for node 68d46115a5c2c05277bcaf38 INFO: 172.21.0.1:41660 - "POST /node/68d46115a5c2c05277bcaf38 HTTP/1.0" 202 Accepted 2025-09-26 18:20:01,827 - lava_callback - INFO - Starting async job submit for node 68d46115a5c2c05277bcaf38 2025-09-26 18:20:01,847 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:20:01,881 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d46115a5c2c05277bcaf38 HTTP/1.1" 200 None 2025-09-26 18:20:01,882 - lava_callback - DEBUG - Processing node 68d46115a5c2c05277bcaf38: baseline-arm64 2025-09-26 18:20:02,191 - lava_callback - ERROR - Error processing callback for node 68d46115a5c2c05277bcaf38: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:41662 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42292 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42300 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58118 - "GET / HTTP/1.0" 200 OK 2025-09-26 18:20:21,772 - lava_callback - INFO - Received callback for node 68d4611aa5c2c05277bcaf3b INFO: 172.21.0.1:58134 - "POST /node/68d4611aa5c2c05277bcaf3b HTTP/1.0" 202 Accepted 2025-09-26 18:20:21,778 - lava_callback - INFO - Starting async job submit for node 68d4611aa5c2c05277bcaf3b 2025-09-26 18:20:21,798 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:20:21,811 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4611aa5c2c05277bcaf3b HTTP/1.1" 200 None 2025-09-26 18:20:21,811 - lava_callback - DEBUG - Processing node 68d4611aa5c2c05277bcaf3b: baseline-arm64 2025-09-26 18:20:22,131 - lava_callback - ERROR - Error processing callback for node 68d4611aa5c2c05277bcaf3b: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:58142 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59254 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59258 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49470 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56384 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52588 - "GET / HTTP/1.0" 200 OK 2025-09-26 18:21:09,418 - lava_callback - INFO - Received callback for node 68d4c297a5c2c05277bcfc77 INFO: 172.21.0.1:37078 - "POST /node/68d4c297a5c2c05277bcfc77 HTTP/1.0" 202 Accepted 2025-09-26 18:21:09,424 - lava_callback - INFO - Starting async job submit for node 68d4c297a5c2c05277bcfc77 2025-09-26 18:21:09,442 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:21:09,454 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4c297a5c2c05277bcfc77 HTTP/1.1" 200 None 2025-09-26 18:21:09,455 - lava_callback - DEBUG - Processing node 68d4c297a5c2c05277bcfc77: baseline-arm 2025-09-26 18:21:09,682 - lava_callback - ERROR - Error processing callback for node 68d4c297a5c2c05277bcfc77: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:37092 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37102 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35900 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35902 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41928 - "GET / HTTP/1.0" 200 OK 2025-09-26 18:21:30,532 - lava_callback - INFO - Received callback for node 68d4611aa5c2c05277bcaf3e INFO: 172.21.0.1:41944 - "POST /node/68d4611aa5c2c05277bcaf3e HTTP/1.0" 202 Accepted 2025-09-26 18:21:30,538 - lava_callback - INFO - Starting async job submit for node 68d4611aa5c2c05277bcaf3e 2025-09-26 18:21:30,559 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:21:30,595 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4611aa5c2c05277bcaf3e HTTP/1.1" 200 None 2025-09-26 18:21:30,596 - lava_callback - DEBUG - Processing node 68d4611aa5c2c05277bcaf3e: baseline-arm64 2025-09-26 18:21:30,886 - lava_callback - ERROR - Error processing callback for node 68d4611aa5c2c05277bcaf3e: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None 2025-09-26 18:21:32,195 - lava_callback - INFO - Received callback for node 68d4611da5c2c05277bcaf41 INFO: 172.21.0.1:41946 - "POST /node/68d4611da5c2c05277bcaf41 HTTP/1.0" 202 Accepted 2025-09-26 18:21:32,201 - lava_callback - INFO - Starting async job submit for node 68d4611da5c2c05277bcaf41 2025-09-26 18:21:32,222 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:21:32,235 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4611da5c2c05277bcaf41 HTTP/1.1" 200 None 2025-09-26 18:21:32,235 - lava_callback - DEBUG - Processing node 68d4611da5c2c05277bcaf41: baseline-arm64 2025-09-26 18:21:32,563 - lava_callback - ERROR - Error processing callback for node 68d4611da5c2c05277bcaf41: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None 2025-09-26 18:21:33,434 - lava_callback - INFO - Received callback for node 68d43274a5c2c05277bc8658 INFO: 172.21.0.1:41960 - "POST /node/68d43274a5c2c05277bc8658 HTTP/1.0" 202 Accepted 2025-09-26 18:21:33,442 - lava_callback - INFO - Starting async job submit for node 68d43274a5c2c05277bc8658 2025-09-26 18:21:33,546 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:21:33,581 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d43274a5c2c05277bc8658 HTTP/1.1" 200 None 2025-09-26 18:21:33,581 - lava_callback - DEBUG - Processing node 68d43274a5c2c05277bc8658: ltp-cve 2025-09-26 18:21:34,303 - lava_callback - ERROR - Error processing callback for node 68d43274a5c2c05277bc8658: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:41964 - "GET / HTTP/1.0" 200 OK 2025-09-26 18:21:41,846 - lava_callback - INFO - Received callback for node 68d46121a5c2c05277bcaf44 INFO: 172.21.0.1:36422 - "POST /node/68d46121a5c2c05277bcaf44 HTTP/1.0" 202 Accepted 2025-09-26 18:21:41,853 - lava_callback - INFO - Starting async job submit for node 68d46121a5c2c05277bcaf44 2025-09-26 18:21:41,873 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:21:41,885 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d46121a5c2c05277bcaf44 HTTP/1.1" 200 None 2025-09-26 18:21:41,886 - lava_callback - DEBUG - Processing node 68d46121a5c2c05277bcaf44: baseline-arm64 2025-09-26 18:21:42,195 - lava_callback - ERROR - Error processing callback for node 68d46121a5c2c05277bcaf44: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:36438 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41798 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50556 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35316 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35318 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40522 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40526 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50310 - "GET / HTTP/1.0" 200 OK 2025-09-26 18:22:30,606 - lava_callback - INFO - Received callback for node 68d46124a5c2c05277bcaf47 INFO: 172.21.0.1:50322 - "POST /node/68d46124a5c2c05277bcaf47 HTTP/1.0" 202 Accepted 2025-09-26 18:22:30,612 - lava_callback - INFO - Starting async job submit for node 68d46124a5c2c05277bcaf47 2025-09-26 18:22:30,641 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:22:30,673 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d46124a5c2c05277bcaf47 HTTP/1.1" 200 None 2025-09-26 18:22:30,674 - lava_callback - DEBUG - Processing node 68d46124a5c2c05277bcaf47: baseline-arm64 2025-09-26 18:22:31,008 - lava_callback - ERROR - Error processing callback for node 68d46124a5c2c05277bcaf47: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:50338 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43330 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42102 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39636 - "GET / HTTP/1.0" 200 OK 2025-09-26 18:23:09,862 - lava_callback - INFO - Received callback for node 68d46128a5c2c05277bcaf4d INFO: 172.21.0.1:50846 - "POST /node/68d46128a5c2c05277bcaf4d HTTP/1.0" 202 Accepted 2025-09-26 18:23:09,868 - lava_callback - INFO - Starting async job submit for node 68d46128a5c2c05277bcaf4d 2025-09-26 18:23:09,889 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:23:09,906 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d46128a5c2c05277bcaf4d HTTP/1.1" 200 None 2025-09-26 18:23:09,907 - lava_callback - DEBUG - Processing node 68d46128a5c2c05277bcaf4d: baseline-arm64 2025-09-26 18:23:10,208 - lava_callback - ERROR - Error processing callback for node 68d46128a5c2c05277bcaf4d: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None 2025-09-26 18:23:10,474 - lava_callback - INFO - Received callback for node 68d4c2d9a5c2c05277bcfcb8 INFO: 172.21.0.1:50858 - "POST /node/68d4c2d9a5c2c05277bcfcb8 HTTP/1.0" 202 Accepted 2025-09-26 18:23:10,480 - lava_callback - INFO - Starting async job submit for node 68d4c2d9a5c2c05277bcfcb8 2025-09-26 18:23:10,501 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:23:10,514 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4c2d9a5c2c05277bcfcb8 HTTP/1.1" 200 None 2025-09-26 18:23:10,514 - lava_callback - DEBUG - Processing node 68d4c2d9a5c2c05277bcfcb8: baseline-arm 2025-09-26 18:23:10,747 - lava_callback - ERROR - Error processing callback for node 68d4c2d9a5c2c05277bcfcb8: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None 2025-09-26 18:23:11,097 - lava_callback - INFO - Received callback for node 68d46125a5c2c05277bcaf4a INFO: 172.21.0.1:50866 - "POST /node/68d46125a5c2c05277bcaf4a HTTP/1.0" 202 Accepted 2025-09-26 18:23:11,104 - lava_callback - INFO - Starting async job submit for node 68d46125a5c2c05277bcaf4a 2025-09-26 18:23:11,124 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:23:11,138 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d46125a5c2c05277bcaf4a HTTP/1.1" 200 None 2025-09-26 18:23:11,138 - lava_callback - DEBUG - Processing node 68d46125a5c2c05277bcaf4a: baseline-arm64 2025-09-26 18:23:11,467 - lava_callback - ERROR - Error processing callback for node 68d46125a5c2c05277bcaf4a: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:50868 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50872 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47030 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47032 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38638 - "GET / HTTP/1.0" 200 OK 2025-09-26 18:23:30,717 - lava_callback - INFO - Received callback for node 68d4612aa5c2c05277bcaf50 INFO: 172.21.0.1:38652 - "POST /node/68d4612aa5c2c05277bcaf50 HTTP/1.0" 202 Accepted 2025-09-26 18:23:30,723 - lava_callback - INFO - Starting async job submit for node 68d4612aa5c2c05277bcaf50 2025-09-26 18:23:30,744 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:23:30,756 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4612aa5c2c05277bcaf50 HTTP/1.1" 200 None 2025-09-26 18:23:30,757 - lava_callback - DEBUG - Processing node 68d4612aa5c2c05277bcaf50: baseline-arm64 2025-09-26 18:23:31,050 - lava_callback - ERROR - Error processing callback for node 68d4612aa5c2c05277bcaf50: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:38662 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47340 - "GET / HTTP/1.0" 200 OK 2025-09-26 18:23:54,572 - lava_callback - INFO - Received callback for node 68d437f4a5c2c05277bc9079 INFO: 172.21.0.1:44630 - "POST /node/68d437f4a5c2c05277bc9079 HTTP/1.0" 202 Accepted 2025-09-26 18:23:54,578 - lava_callback - INFO - Starting async job submit for node 68d437f4a5c2c05277bc9079 2025-09-26 18:23:54,608 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:23:54,620 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d437f4a5c2c05277bc9079 HTTP/1.1" 200 None 2025-09-26 18:23:54,621 - lava_callback - DEBUG - Processing node 68d437f4a5c2c05277bc9079: baseline-arm 2025-09-26 18:23:54,895 - lava_callback - ERROR - Error processing callback for node 68d437f4a5c2c05277bc9079: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:44634 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57594 - "GET / HTTP/1.0" 200 OK 2025-09-26 18:24:13,490 - lava_callback - INFO - Received callback for node 68d4380ca5c2c05277bc90c4 INFO: 172.21.0.1:35560 - "POST /node/68d4380ca5c2c05277bc90c4 HTTP/1.0" 202 Accepted 2025-09-26 18:24:13,497 - lava_callback - INFO - Starting async job submit for node 68d4380ca5c2c05277bc90c4 2025-09-26 18:24:13,619 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:24:13,636 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4380ca5c2c05277bc90c4 HTTP/1.1" 200 None 2025-09-26 18:24:13,636 - lava_callback - DEBUG - Processing node 68d4380ca5c2c05277bc90c4: ltp-containers 2025-09-26 18:24:14,439 - lava_callback - ERROR - Error processing callback for node 68d4380ca5c2c05277bc90c4: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:35576 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35578 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51916 - "GET / HTTP/1.0" 200 OK 2025-09-26 18:24:26,003 - lava_callback - INFO - Received callback for node 68d4612fa5c2c05277bcaf53 INFO: 172.21.0.1:51926 - "POST /node/68d4612fa5c2c05277bcaf53 HTTP/1.0" 202 Accepted 2025-09-26 18:24:26,009 - lava_callback - INFO - Starting async job submit for node 68d4612fa5c2c05277bcaf53 2025-09-26 18:24:26,030 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:24:26,042 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4612fa5c2c05277bcaf53 HTTP/1.1" 200 None 2025-09-26 18:24:26,043 - lava_callback - DEBUG - Processing node 68d4612fa5c2c05277bcaf53: baseline-arm64 2025-09-26 18:24:26,345 - lava_callback - ERROR - Error processing callback for node 68d4612fa5c2c05277bcaf53: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:51942 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53198 - "GET / HTTP/1.0" 200 OK 2025-09-26 18:24:30,808 - lava_callback - INFO - Received callback for node 68d46130a5c2c05277bcaf56 INFO: 172.21.0.1:53214 - "POST /node/68d46130a5c2c05277bcaf56 HTTP/1.0" 202 Accepted 2025-09-26 18:24:30,815 - lava_callback - INFO - Starting async job submit for node 68d46130a5c2c05277bcaf56 2025-09-26 18:24:30,837 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:24:30,850 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d46130a5c2c05277bcaf56 HTTP/1.1" 200 None 2025-09-26 18:24:30,851 - lava_callback - DEBUG - Processing node 68d46130a5c2c05277bcaf56: baseline-arm64 2025-09-26 18:24:31,138 - lava_callback - ERROR - Error processing callback for node 68d46130a5c2c05277bcaf56: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None 2025-09-26 18:24:36,371 - lava_callback - INFO - Received callback for node 68d46132a5c2c05277bcaf59 INFO: 172.21.0.1:53228 - "POST /node/68d46132a5c2c05277bcaf59 HTTP/1.0" 202 Accepted 2025-09-26 18:24:36,377 - lava_callback - INFO - Starting async job submit for node 68d46132a5c2c05277bcaf59 2025-09-26 18:24:36,398 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:24:36,412 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d46132a5c2c05277bcaf59 HTTP/1.1" 200 None 2025-09-26 18:24:36,412 - lava_callback - DEBUG - Processing node 68d46132a5c2c05277bcaf59: baseline-arm64 INFO: 172.21.0.1:53244 - "GET / HTTP/1.0" 200 OK 2025-09-26 18:24:36,747 - lava_callback - ERROR - Error processing callback for node 68d46132a5c2c05277bcaf59: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:48682 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41634 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37714 - "GET / HTTP/1.0" 200 OK 2025-09-26 18:25:14,773 - lava_callback - INFO - Received callback for node 68d4c2d9a5c2c05277bcfcbb INFO: 172.21.0.1:42124 - "POST /node/68d4c2d9a5c2c05277bcfcbb HTTP/1.0" 202 Accepted 2025-09-26 18:25:14,779 - lava_callback - INFO - Starting async job submit for node 68d4c2d9a5c2c05277bcfcbb 2025-09-26 18:25:14,799 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:25:14,812 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4c2d9a5c2c05277bcfcbb HTTP/1.1" 200 None 2025-09-26 18:25:14,812 - lava_callback - DEBUG - Processing node 68d4c2d9a5c2c05277bcfcbb: baseline-arm 2025-09-26 18:25:15,036 - lava_callback - ERROR - Error processing callback for node 68d4c2d9a5c2c05277bcfcbb: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:42140 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42148 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51056 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51058 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42988 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42996 - "GET / HTTP/1.0" 200 OK 2025-09-26 18:25:37,072 - lava_callback - INFO - Received callback for node 68d6c95f3c94e462c6881977 INFO: 172.21.0.1:43002 - "POST /node/68d6c95f3c94e462c6881977 HTTP/1.0" 202 Accepted 2025-09-26 18:25:37,079 - lava_callback - INFO - Starting async job submit for node 68d6c95f3c94e462c6881977 2025-09-26 18:25:37,112 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:25:37,127 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6c95f3c94e462c6881977 HTTP/1.1" 200 None 2025-09-26 18:25:37,128 - lava_callback - DEBUG - Processing node 68d6c95f3c94e462c6881977: baseline-x86-kcidebug-intel 2025-09-26 18:25:37,377 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 18:25:37,524 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 18:25:37,525 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-kcidebug-intel-68d6c95f3c94e462c6881977/log.txt.gz 2025-09-26 18:25:37,531 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 18:25:37,680 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 18:25:37,681 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-kcidebug-intel-68d6c95f3c94e462c6881977/lava_callback.json.gz DEBUG: Setup failed for 68d6c95f3c94e462c6881977. Transitting job node result: incomplete -> fail 2025-09-26 18:25:37,741 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:25:37,753 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6c95f3c94e462c6881977 HTTP/1.1" 200 None 2025-09-26 18:25:37,754 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:25:37,766 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6b9f52f83206171a92b71 HTTP/1.1" 200 None 2025-09-26 18:25:37,767 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:25:37,805 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6c95f3c94e462c6881977 HTTP/1.1" 200 None 2025-09-26 18:25:37,805 - lava_callback - INFO - Completed processing callback for node 68d6c95f3c94e462c6881977 2025-09-26 18:25:45,973 - lava_callback - INFO - Received callback for node 68d4614fa5c2c05277bcaf89 INFO: 172.21.0.1:33116 - "POST /node/68d4614fa5c2c05277bcaf89 HTTP/1.0" 202 Accepted 2025-09-26 18:25:45,979 - lava_callback - INFO - Starting async job submit for node 68d4614fa5c2c05277bcaf89 2025-09-26 18:25:45,999 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:25:46,016 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4614fa5c2c05277bcaf89 HTTP/1.1" 200 None 2025-09-26 18:25:46,016 - lava_callback - DEBUG - Processing node 68d4614fa5c2c05277bcaf89: baseline-arm64 2025-09-26 18:25:46,346 - lava_callback - ERROR - Error processing callback for node 68d4614fa5c2c05277bcaf89: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:33124 - "GET / HTTP/1.0" 200 OK 2025-09-26 18:25:47,537 - lava_callback - INFO - Received callback for node 68d46135a5c2c05277bcaf5c INFO: 172.21.0.1:33134 - "POST /node/68d46135a5c2c05277bcaf5c HTTP/1.0" 202 Accepted 2025-09-26 18:25:47,543 - lava_callback - INFO - Starting async job submit for node 68d46135a5c2c05277bcaf5c 2025-09-26 18:25:47,610 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:25:47,621 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d46135a5c2c05277bcaf5c HTTP/1.1" 200 None 2025-09-26 18:25:47,622 - lava_callback - DEBUG - Processing node 68d46135a5c2c05277bcaf5c: baseline-arm64 2025-09-26 18:25:48,086 - lava_callback - ERROR - Error processing callback for node 68d46135a5c2c05277bcaf5c: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None 2025-09-26 18:25:55,541 - lava_callback - INFO - Received callback for node 68d46152a5c2c05277bcaf8c INFO: 172.21.0.1:40292 - "POST /node/68d46152a5c2c05277bcaf8c HTTP/1.0" 202 Accepted 2025-09-26 18:25:55,550 - lava_callback - INFO - Starting async job submit for node 68d46152a5c2c05277bcaf8c 2025-09-26 18:25:55,571 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:25:55,583 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d46152a5c2c05277bcaf8c HTTP/1.1" 200 None 2025-09-26 18:25:55,584 - lava_callback - DEBUG - Processing node 68d46152a5c2c05277bcaf8c: baseline-arm64 2025-09-26 18:25:55,903 - lava_callback - ERROR - Error processing callback for node 68d46152a5c2c05277bcaf8c: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:40296 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44728 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50834 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50846 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59382 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59384 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46170 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46176 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37126 - "GET / HTTP/1.0" 200 OK 2025-09-26 18:26:49,795 - lava_callback - INFO - Received callback for node 68d6d6433c94e462c6881f13 INFO: 172.21.0.1:45748 - "POST /node/68d6d6433c94e462c6881f13 HTTP/1.0" 202 Accepted 2025-09-26 18:26:49,802 - lava_callback - INFO - Starting async job submit for node 68d6d6433c94e462c6881f13 2025-09-26 18:26:49,851 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:26:49,888 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6d6433c94e462c6881f13 HTTP/1.1" 200 None 2025-09-26 18:26:49,889 - lava_callback - DEBUG - Processing node 68d6d6433c94e462c6881f13: baseline-x86 2025-09-26 18:26:50,562 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 18:26:50,705 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 18:26:50,706 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-68d6d6433c94e462c6881f13/log.txt.gz 2025-09-26 18:26:50,722 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 18:26:50,883 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 18:26:50,884 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-68d6d6433c94e462c6881f13/lava_callback.json.gz DEBUG: Setup failed for 68d6d6433c94e462c6881f13. Transitting job node result: incomplete -> fail 2025-09-26 18:26:50,970 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:26:50,988 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6d6433c94e462c6881f13 HTTP/1.1" 200 None 2025-09-26 18:26:50,989 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:26:51,025 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6bda33c94e462c6881504 HTTP/1.1" 200 None 2025-09-26 18:26:51,027 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:26:51,056 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6d6433c94e462c6881f13 HTTP/1.1" 200 None 2025-09-26 18:26:51,057 - lava_callback - INFO - Completed processing callback for node 68d6d6433c94e462c6881f13 INFO: 172.21.0.1:45760 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38016 - "GET / HTTP/1.0" 200 OK 2025-09-26 18:27:07,269 - lava_callback - INFO - Received callback for node 68d4615aa5c2c05277bcaf92 INFO: 172.21.0.1:38020 - "POST /node/68d4615aa5c2c05277bcaf92 HTTP/1.0" 202 Accepted 2025-09-26 18:27:07,275 - lava_callback - INFO - Starting async job submit for node 68d4615aa5c2c05277bcaf92 2025-09-26 18:27:07,296 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:27:07,310 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4615aa5c2c05277bcaf92 HTTP/1.1" 200 None 2025-09-26 18:27:07,311 - lava_callback - DEBUG - Processing node 68d4615aa5c2c05277bcaf92: baseline-arm64 2025-09-26 18:27:07,603 - lava_callback - ERROR - Error processing callback for node 68d4615aa5c2c05277bcaf92: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None 2025-09-26 18:27:09,627 - lava_callback - INFO - Received callback for node 68d46157a5c2c05277bcaf8f INFO: 172.21.0.1:42124 - "POST /node/68d46157a5c2c05277bcaf8f HTTP/1.0" 202 Accepted 2025-09-26 18:27:09,633 - lava_callback - INFO - Starting async job submit for node 68d46157a5c2c05277bcaf8f 2025-09-26 18:27:09,653 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:27:09,665 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d46157a5c2c05277bcaf8f HTTP/1.1" 200 None 2025-09-26 18:27:09,666 - lava_callback - DEBUG - Processing node 68d46157a5c2c05277bcaf8f: baseline-arm64 2025-09-26 18:27:09,962 - lava_callback - ERROR - Error processing callback for node 68d46157a5c2c05277bcaf8f: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None 2025-09-26 18:27:13,803 - lava_callback - INFO - Received callback for node 68d437ffa5c2c05277bc90a0 INFO: 172.21.0.1:42138 - "POST /node/68d437ffa5c2c05277bc90a0 HTTP/1.0" 202 Accepted 2025-09-26 18:27:13,809 - lava_callback - INFO - Starting async job submit for node 68d437ffa5c2c05277bc90a0 2025-09-26 18:27:13,809 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:27:13,823 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d437ffa5c2c05277bc90a0 HTTP/1.1" 200 None 2025-09-26 18:27:13,823 - lava_callback - DEBUG - Processing node 68d437ffa5c2c05277bc90a0: baseline-arm-broonie 2025-09-26 18:27:13,823 - lava_callback - ERROR - Error processing callback for node 68d437ffa5c2c05277bc90a0: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:42144 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42158 - "GET / HTTP/1.0" 200 OK 2025-09-26 18:27:18,698 - lava_callback - INFO - Received callback for node 68d4615fa5c2c05277bcaf95 INFO: 172.21.0.1:42174 - "POST /node/68d4615fa5c2c05277bcaf95 HTTP/1.0" 202 Accepted 2025-09-26 18:27:18,704 - lava_callback - INFO - Starting async job submit for node 68d4615fa5c2c05277bcaf95 2025-09-26 18:27:18,725 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:27:18,737 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4615fa5c2c05277bcaf95 HTTP/1.1" 200 None 2025-09-26 18:27:18,738 - lava_callback - DEBUG - Processing node 68d4615fa5c2c05277bcaf95: baseline-arm64 2025-09-26 18:27:19,056 - lava_callback - ERROR - Error processing callback for node 68d4615fa5c2c05277bcaf95: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:37242 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37250 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53594 - "GET / HTTP/1.0" 200 OK 2025-09-26 18:27:31,047 - lava_callback - INFO - Received callback for node 68d4c2dba5c2c05277bcfcbe INFO: 172.21.0.1:53596 - "POST /node/68d4c2dba5c2c05277bcfcbe HTTP/1.0" 202 Accepted 2025-09-26 18:27:31,053 - lava_callback - INFO - Starting async job submit for node 68d4c2dba5c2c05277bcfcbe 2025-09-26 18:27:31,073 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:27:31,086 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4c2dba5c2c05277bcfcbe HTTP/1.1" 200 None 2025-09-26 18:27:31,087 - lava_callback - DEBUG - Processing node 68d4c2dba5c2c05277bcfcbe: baseline-arm 2025-09-26 18:27:31,312 - lava_callback - ERROR - Error processing callback for node 68d4c2dba5c2c05277bcfcbe: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None 2025-09-26 18:27:33,833 - lava_callback - INFO - Received callback for node 68d43810a5c2c05277bc90d3 INFO: 172.21.0.1:53600 - "POST /node/68d43810a5c2c05277bc90d3 HTTP/1.0" 202 Accepted 2025-09-26 18:27:33,839 - lava_callback - INFO - Starting async job submit for node 68d43810a5c2c05277bc90d3 2025-09-26 18:27:33,839 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:27:33,895 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d43810a5c2c05277bc90d3 HTTP/1.1" 200 None 2025-09-26 18:27:33,896 - lava_callback - DEBUG - Processing node 68d43810a5c2c05277bc90d3: ltp-fs-bind 2025-09-26 18:27:33,896 - lava_callback - ERROR - Error processing callback for node 68d43810a5c2c05277bc90d3: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 18:27:34,613 - lava_callback - INFO - Received callback for node 68d43811a5c2c05277bc90d6 INFO: 172.21.0.1:53610 - "POST /node/68d43811a5c2c05277bc90d6 HTTP/1.0" 202 Accepted 2025-09-26 18:27:34,619 - lava_callback - INFO - Starting async job submit for node 68d43811a5c2c05277bc90d6 2025-09-26 18:27:34,620 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:27:34,632 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d43811a5c2c05277bc90d6 HTTP/1.1" 200 None 2025-09-26 18:27:34,632 - lava_callback - DEBUG - Processing node 68d43811a5c2c05277bc90d6: ltp-fsx 2025-09-26 18:27:34,632 - lava_callback - ERROR - Error processing callback for node 68d43811a5c2c05277bc90d6: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 18:27:35,444 - lava_callback - INFO - Received callback for node 68d43811a5c2c05277bc90d9 INFO: 172.21.0.1:53614 - "POST /node/68d43811a5c2c05277bc90d9 HTTP/1.0" 202 Accepted 2025-09-26 18:27:35,450 - lava_callback - INFO - Starting async job submit for node 68d43811a5c2c05277bc90d9 2025-09-26 18:27:35,451 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:27:35,474 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d43811a5c2c05277bc90d9 HTTP/1.1" 200 None 2025-09-26 18:27:35,474 - lava_callback - DEBUG - Processing node 68d43811a5c2c05277bc90d9: ltp-input 2025-09-26 18:27:35,474 - lava_callback - ERROR - Error processing callback for node 68d43811a5c2c05277bc90d9: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 18:27:36,213 - lava_callback - INFO - Received callback for node 68d43815a5c2c05277bc90f1 INFO: 172.21.0.1:53630 - "POST /node/68d43815a5c2c05277bc90f1 HTTP/1.0" 202 Accepted 2025-09-26 18:27:36,219 - lava_callback - INFO - Starting async job submit for node 68d43815a5c2c05277bc90f1 2025-09-26 18:27:36,220 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:27:36,234 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d43815a5c2c05277bc90f1 HTTP/1.1" 200 None 2025-09-26 18:27:36,235 - lava_callback - DEBUG - Processing node 68d43815a5c2c05277bc90f1: ltp-ipc 2025-09-26 18:27:36,235 - lava_callback - ERROR - Error processing callback for node 68d43815a5c2c05277bc90f1: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:53644 - "GET / HTTP/1.0" 200 OK 2025-09-26 18:27:36,973 - lava_callback - INFO - Received callback for node 68d43816a5c2c05277bc90f6 INFO: 172.21.0.1:53654 - "POST /node/68d43816a5c2c05277bc90f6 HTTP/1.0" 202 Accepted 2025-09-26 18:27:36,979 - lava_callback - INFO - Starting async job submit for node 68d43816a5c2c05277bc90f6 2025-09-26 18:27:36,980 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:27:36,991 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d43816a5c2c05277bc90f6 HTTP/1.1" 200 None 2025-09-26 18:27:36,991 - lava_callback - DEBUG - Processing node 68d43816a5c2c05277bc90f6: ltp-pty 2025-09-26 18:27:36,991 - lava_callback - ERROR - Error processing callback for node 68d43816a5c2c05277bc90f6: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 18:27:37,730 - lava_callback - INFO - Received callback for node 68d43816a5c2c05277bc90fa INFO: 172.21.0.1:53664 - "POST /node/68d43816a5c2c05277bc90fa HTTP/1.0" 202 Accepted 2025-09-26 18:27:37,736 - lava_callback - INFO - Starting async job submit for node 68d43816a5c2c05277bc90fa 2025-09-26 18:27:37,737 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:27:37,751 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d43816a5c2c05277bc90fa HTTP/1.1" 200 None 2025-09-26 18:27:37,752 - lava_callback - DEBUG - Processing node 68d43816a5c2c05277bc90fa: ltp-sched 2025-09-26 18:27:37,752 - lava_callback - ERROR - Error processing callback for node 68d43816a5c2c05277bc90fa: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 18:27:38,479 - lava_callback - INFO - Received callback for node 68d43817a5c2c05277bc90fd INFO: 172.21.0.1:53680 - "POST /node/68d43817a5c2c05277bc90fd HTTP/1.0" 202 Accepted 2025-09-26 18:27:38,485 - lava_callback - INFO - Starting async job submit for node 68d43817a5c2c05277bc90fd 2025-09-26 18:27:38,486 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:27:38,505 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d43817a5c2c05277bc90fd HTTP/1.1" 200 None 2025-09-26 18:27:38,505 - lava_callback - DEBUG - Processing node 68d43817a5c2c05277bc90fd: ltp-smoketest 2025-09-26 18:27:38,505 - lava_callback - ERROR - Error processing callback for node 68d43817a5c2c05277bc90fd: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 18:27:39,226 - lava_callback - INFO - Received callback for node 68d43818a5c2c05277bc9100 INFO: 172.21.0.1:53694 - "POST /node/68d43818a5c2c05277bc9100 HTTP/1.0" 202 Accepted 2025-09-26 18:27:39,232 - lava_callback - INFO - Starting async job submit for node 68d43818a5c2c05277bc9100 2025-09-26 18:27:39,233 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:27:39,250 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d43818a5c2c05277bc9100 HTTP/1.1" 200 None 2025-09-26 18:27:39,250 - lava_callback - DEBUG - Processing node 68d43818a5c2c05277bc9100: ltp-syscalls 2025-09-26 18:27:39,250 - lava_callback - ERROR - Error processing callback for node 68d43818a5c2c05277bc9100: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 18:27:39,997 - lava_callback - INFO - Received callback for node 68d43819a5c2c05277bc9103 INFO: 172.21.0.1:57418 - "POST /node/68d43819a5c2c05277bc9103 HTTP/1.0" 202 Accepted 2025-09-26 18:27:40,003 - lava_callback - INFO - Starting async job submit for node 68d43819a5c2c05277bc9103 2025-09-26 18:27:40,004 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:27:40,017 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d43819a5c2c05277bc9103 HTTP/1.1" 200 None 2025-09-26 18:27:40,017 - lava_callback - DEBUG - Processing node 68d43819a5c2c05277bc9103: ltp-syscalls-ipc 2025-09-26 18:27:40,017 - lava_callback - ERROR - Error processing callback for node 68d43819a5c2c05277bc9103: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 18:27:40,751 - lava_callback - INFO - Received callback for node 68d43819a5c2c05277bc9106 INFO: 172.21.0.1:57434 - "POST /node/68d43819a5c2c05277bc9106 HTTP/1.0" 202 Accepted 2025-09-26 18:27:40,757 - lava_callback - INFO - Starting async job submit for node 68d43819a5c2c05277bc9106 2025-09-26 18:27:40,758 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:27:40,770 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d43819a5c2c05277bc9106 HTTP/1.1" 200 None 2025-09-26 18:27:40,771 - lava_callback - DEBUG - Processing node 68d43819a5c2c05277bc9106: ltp-timers 2025-09-26 18:27:40,771 - lava_callback - ERROR - Error processing callback for node 68d43819a5c2c05277bc9106: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 18:27:41,610 - lava_callback - INFO - Received callback for node 68d4381aa5c2c05277bc9109 INFO: 172.21.0.1:57440 - "POST /node/68d4381aa5c2c05277bc9109 HTTP/1.0" 202 Accepted 2025-09-26 18:27:41,616 - lava_callback - INFO - Starting async job submit for node 68d4381aa5c2c05277bc9109 2025-09-26 18:27:41,616 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:27:41,781 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4381aa5c2c05277bc9109 HTTP/1.1" 200 None 2025-09-26 18:27:41,782 - lava_callback - DEBUG - Processing node 68d4381aa5c2c05277bc9109: ltp-watchqueue 2025-09-26 18:27:41,782 - lava_callback - ERROR - Error processing callback for node 68d4381aa5c2c05277bc9109: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:57444 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35352 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54326 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41454 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41460 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33314 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33324 - "GET / HTTP/1.0" 200 OK 2025-09-26 18:28:28,603 - lava_callback - INFO - Received callback for node 68d46162a5c2c05277bcaf9b INFO: 172.21.0.1:33328 - "POST /node/68d46162a5c2c05277bcaf9b HTTP/1.0" 202 Accepted 2025-09-26 18:28:28,610 - lava_callback - INFO - Starting async job submit for node 68d46162a5c2c05277bcaf9b 2025-09-26 18:28:28,645 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:28:28,664 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d46162a5c2c05277bcaf9b HTTP/1.1" 200 None 2025-09-26 18:28:28,664 - lava_callback - DEBUG - Processing node 68d46162a5c2c05277bcaf9b: baseline-arm64 2025-09-26 18:28:29,051 - lava_callback - ERROR - Error processing callback for node 68d46162a5c2c05277bcaf9b: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:43898 - "GET / HTTP/1.0" 200 OK 2025-09-26 18:28:30,496 - lava_callback - INFO - Received callback for node 68d46160a5c2c05277bcaf98 INFO: 172.21.0.1:43914 - "POST /node/68d46160a5c2c05277bcaf98 HTTP/1.0" 202 Accepted 2025-09-26 18:28:30,502 - lava_callback - INFO - Starting async job submit for node 68d46160a5c2c05277bcaf98 2025-09-26 18:28:30,522 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:28:30,544 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d46160a5c2c05277bcaf98 HTTP/1.1" 200 None 2025-09-26 18:28:30,545 - lava_callback - DEBUG - Processing node 68d46160a5c2c05277bcaf98: baseline-arm64 2025-09-26 18:28:30,842 - lava_callback - ERROR - Error processing callback for node 68d46160a5c2c05277bcaf98: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None 2025-09-26 18:28:35,493 - lava_callback - INFO - Received callback for node 68d46165a5c2c05277bcaf9e INFO: 172.21.0.1:43926 - "POST /node/68d46165a5c2c05277bcaf9e HTTP/1.0" 202 Accepted 2025-09-26 18:28:35,499 - lava_callback - INFO - Starting async job submit for node 68d46165a5c2c05277bcaf9e 2025-09-26 18:28:35,519 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:28:35,555 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d46165a5c2c05277bcaf9e HTTP/1.1" 200 None 2025-09-26 18:28:35,555 - lava_callback - DEBUG - Processing node 68d46165a5c2c05277bcaf9e: baseline-arm64 2025-09-26 18:28:35,849 - lava_callback - ERROR - Error processing callback for node 68d46165a5c2c05277bcaf9e: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:43940 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37188 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47300 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44938 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36266 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36276 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50870 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50884 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53786 - "GET / HTTP/1.0" 200 OK 2025-09-26 18:29:29,919 - lava_callback - INFO - Received callback for node 68d4c2dba5c2c05277bcfcc1 INFO: 172.21.0.1:53794 - "POST /node/68d4c2dba5c2c05277bcfcc1 HTTP/1.0" 202 Accepted 2025-09-26 18:29:29,926 - lava_callback - INFO - Starting async job submit for node 68d4c2dba5c2c05277bcfcc1 2025-09-26 18:29:29,946 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:29:29,959 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4c2dba5c2c05277bcfcc1 HTTP/1.1" 200 None 2025-09-26 18:29:29,959 - lava_callback - DEBUG - Processing node 68d4c2dba5c2c05277bcfcc1: baseline-arm 2025-09-26 18:29:30,215 - lava_callback - ERROR - Error processing callback for node 68d4c2dba5c2c05277bcfcc1: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:53798 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33320 - "GET / HTTP/1.0" 200 OK 2025-09-26 18:29:50,723 - lava_callback - INFO - Received callback for node 68d463aca5c2c05277bcb278 INFO: 172.21.0.1:49174 - "POST /node/68d463aca5c2c05277bcb278 HTTP/1.0" 202 Accepted 2025-09-26 18:29:50,729 - lava_callback - INFO - Starting async job submit for node 68d463aca5c2c05277bcb278 2025-09-26 18:29:50,750 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:29:50,762 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d463aca5c2c05277bcb278 HTTP/1.1" 200 None 2025-09-26 18:29:50,762 - lava_callback - DEBUG - Processing node 68d463aca5c2c05277bcb278: baseline-arm64 2025-09-26 18:29:51,053 - lava_callback - ERROR - Error processing callback for node 68d463aca5c2c05277bcb278: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:49182 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47452 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59986 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59992 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33104 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33106 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56626 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56632 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55438 - "GET / HTTP/1.0" 200 OK 2025-09-26 18:30:53,641 - lava_callback - INFO - Received callback for node 68d463afa5c2c05277bcb27b INFO: 172.21.0.1:59362 - "POST /node/68d463afa5c2c05277bcb27b HTTP/1.0" 202 Accepted 2025-09-26 18:30:53,647 - lava_callback - INFO - Starting async job submit for node 68d463afa5c2c05277bcb27b 2025-09-26 18:30:53,668 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:30:53,680 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d463afa5c2c05277bcb27b HTTP/1.1" 200 None 2025-09-26 18:30:53,680 - lava_callback - DEBUG - Processing node 68d463afa5c2c05277bcb27b: baseline-arm64 2025-09-26 18:30:53,977 - lava_callback - ERROR - Error processing callback for node 68d463afa5c2c05277bcb27b: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:59364 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37802 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33080 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33094 - "GET / HTTP/1.0" 200 OK 2025-09-26 18:31:22,025 - lava_callback - INFO - Received callback for node 68d463b6a5c2c05277bcb28f INFO: 172.21.0.1:54476 - "POST /node/68d463b6a5c2c05277bcb28f HTTP/1.0" 202 Accepted 2025-09-26 18:31:22,031 - lava_callback - INFO - Starting async job submit for node 68d463b6a5c2c05277bcb28f 2025-09-26 18:31:22,051 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:31:22,065 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d463b6a5c2c05277bcb28f HTTP/1.1" 200 None 2025-09-26 18:31:22,065 - lava_callback - DEBUG - Processing node 68d463b6a5c2c05277bcb28f: baseline-arm64 2025-09-26 18:31:22,382 - lava_callback - ERROR - Error processing callback for node 68d463b6a5c2c05277bcb28f: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:54484 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54494 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37540 - "GET / HTTP/1.0" 200 OK 2025-09-26 18:31:30,207 - lava_callback - INFO - Received callback for node 68d4c393a5c2c05277bcfdb6 INFO: 172.21.0.1:37556 - "POST /node/68d4c393a5c2c05277bcfdb6 HTTP/1.0" 202 Accepted 2025-09-26 18:31:30,213 - lava_callback - INFO - Starting async job submit for node 68d4c393a5c2c05277bcfdb6 2025-09-26 18:31:30,234 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:31:30,247 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4c393a5c2c05277bcfdb6 HTTP/1.1" 200 None 2025-09-26 18:31:30,247 - lava_callback - DEBUG - Processing node 68d4c393a5c2c05277bcfdb6: baseline-arm 2025-09-26 18:31:30,478 - lava_callback - ERROR - Error processing callback for node 68d4c393a5c2c05277bcfdb6: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:37558 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39428 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39194 - "GET / HTTP/1.0" 200 OK 2025-09-26 18:32:02,621 - lava_callback - INFO - Received callback for node 68d463b8a5c2c05277bcb292 INFO: 172.21.0.1:51952 - "POST /node/68d463b8a5c2c05277bcb292 HTTP/1.0" 202 Accepted 2025-09-26 18:32:02,627 - lava_callback - INFO - Starting async job submit for node 68d463b8a5c2c05277bcb292 2025-09-26 18:32:02,648 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:32:02,665 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d463b8a5c2c05277bcb292 HTTP/1.1" 200 None 2025-09-26 18:32:02,666 - lava_callback - DEBUG - Processing node 68d463b8a5c2c05277bcb292: baseline-arm64 2025-09-26 18:32:02,958 - lava_callback - ERROR - Error processing callback for node 68d463b8a5c2c05277bcb292: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:51958 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46586 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46596 - "GET / HTTP/1.0" 200 OK 2025-09-26 18:32:17,553 - lava_callback - INFO - Received callback for node 68d463bda5c2c05277bcb295 INFO: 172.21.0.1:46610 - "POST /node/68d463bda5c2c05277bcb295 HTTP/1.0" 202 Accepted 2025-09-26 18:32:17,562 - lava_callback - INFO - Starting async job submit for node 68d463bda5c2c05277bcb295 2025-09-26 18:32:17,583 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:32:17,596 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d463bda5c2c05277bcb295 HTTP/1.1" 200 None 2025-09-26 18:32:17,596 - lava_callback - DEBUG - Processing node 68d463bda5c2c05277bcb295: baseline-arm64 2025-09-26 18:32:17,913 - lava_callback - ERROR - Error processing callback for node 68d463bda5c2c05277bcb295: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:45344 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45358 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45086 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45100 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46146 - "GET / HTTP/1.0" 200 OK 2025-09-26 18:32:49,047 - lava_callback - INFO - Received callback for node 68d463c0a5c2c05277bcb298 INFO: 172.21.0.1:46162 - "POST /node/68d463c0a5c2c05277bcb298 HTTP/1.0" 202 Accepted 2025-09-26 18:32:49,053 - lava_callback - INFO - Starting async job submit for node 68d463c0a5c2c05277bcb298 2025-09-26 18:32:49,074 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:32:49,086 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d463c0a5c2c05277bcb298 HTTP/1.1" 200 None 2025-09-26 18:32:49,087 - lava_callback - DEBUG - Processing node 68d463c0a5c2c05277bcb298: baseline-arm64 2025-09-26 18:32:49,381 - lava_callback - ERROR - Error processing callback for node 68d463c0a5c2c05277bcb298: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:50250 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38700 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47122 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47124 - "GET / HTTP/1.0" 200 OK 2025-09-26 18:33:21,963 - lava_callback - INFO - Received callback for node 68d463c4a5c2c05277bcb29b INFO: 172.21.0.1:43688 - "POST /node/68d463c4a5c2c05277bcb29b HTTP/1.0" 202 Accepted 2025-09-26 18:33:21,970 - lava_callback - INFO - Starting async job submit for node 68d463c4a5c2c05277bcb29b 2025-09-26 18:33:21,991 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:33:22,030 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d463c4a5c2c05277bcb29b HTTP/1.1" 200 None 2025-09-26 18:33:22,030 - lava_callback - DEBUG - Processing node 68d463c4a5c2c05277bcb29b: baseline-arm64 2025-09-26 18:33:22,369 - lava_callback - ERROR - Error processing callback for node 68d463c4a5c2c05277bcb29b: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:43698 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43710 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53788 - "GET / HTTP/1.0" 200 OK 2025-09-26 18:33:30,140 - lava_callback - INFO - Received callback for node 68d4c393a5c2c05277bcfdb9 INFO: 172.21.0.1:53790 - "POST /node/68d4c393a5c2c05277bcfdb9 HTTP/1.0" 202 Accepted 2025-09-26 18:33:30,146 - lava_callback - INFO - Starting async job submit for node 68d4c393a5c2c05277bcfdb9 2025-09-26 18:33:30,167 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:33:30,210 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4c393a5c2c05277bcfdb9 HTTP/1.1" 200 None 2025-09-26 18:33:30,210 - lava_callback - DEBUG - Processing node 68d4c393a5c2c05277bcfdb9: baseline-arm 2025-09-26 18:33:30,441 - lava_callback - ERROR - Error processing callback for node 68d4c393a5c2c05277bcfdb9: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:53794 - "GET / HTTP/1.0" 200 OK 2025-09-26 18:33:46,185 - lava_callback - INFO - Received callback for node 68d43979a5c2c05277bc92cf INFO: 172.21.0.1:58680 - "POST /node/68d43979a5c2c05277bc92cf HTTP/1.0" 202 Accepted 2025-09-26 18:33:46,198 - lava_callback - INFO - Starting async job submit for node 68d43979a5c2c05277bc92cf 2025-09-26 18:33:46,200 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:33:46,249 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d43979a5c2c05277bc92cf HTTP/1.1" 200 None 2025-09-26 18:33:46,250 - lava_callback - DEBUG - Processing node 68d43979a5c2c05277bc92cf: baseline-arm64-baylibre 2025-09-26 18:33:46,250 - lava_callback - ERROR - Error processing callback for node 68d43979a5c2c05277bc92cf: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:58684 - "GET / HTTP/1.0" 200 OK 2025-09-26 18:33:56,514 - lava_callback - INFO - Received callback for node 68d463c7a5c2c05277bcb29e INFO: 172.21.0.1:54096 - "POST /node/68d463c7a5c2c05277bcb29e HTTP/1.0" 202 Accepted 2025-09-26 18:33:56,522 - lava_callback - INFO - Starting async job submit for node 68d463c7a5c2c05277bcb29e INFO: 172.21.0.1:54108 - "GET / HTTP/1.0" 200 OK 2025-09-26 18:33:56,550 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:33:56,562 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d463c7a5c2c05277bcb29e HTTP/1.1" 200 None 2025-09-26 18:33:56,563 - lava_callback - DEBUG - Processing node 68d463c7a5c2c05277bcb29e: baseline-arm64 2025-09-26 18:33:56,945 - lava_callback - ERROR - Error processing callback for node 68d463c7a5c2c05277bcb29e: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None 2025-09-26 18:34:05,344 - lava_callback - INFO - Received callback for node 68d4641ca5c2c05277bcb2fa INFO: 172.21.0.1:37706 - "POST /node/68d4641ca5c2c05277bcb2fa HTTP/1.0" 202 Accepted 2025-09-26 18:34:05,351 - lava_callback - INFO - Starting async job submit for node 68d4641ca5c2c05277bcb2fa 2025-09-26 18:34:05,375 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:34:05,388 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4641ca5c2c05277bcb2fa HTTP/1.1" 200 None 2025-09-26 18:34:05,388 - lava_callback - DEBUG - Processing node 68d4641ca5c2c05277bcb2fa: baseline-arm64 2025-09-26 18:34:05,788 - lava_callback - ERROR - Error processing callback for node 68d4641ca5c2c05277bcb2fa: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None 2025-09-26 18:34:06,175 - lava_callback - INFO - Received callback for node 68d4398ea5c2c05277bc92f0 INFO: 172.21.0.1:37722 - "POST /node/68d4398ea5c2c05277bc92f0 HTTP/1.0" 202 Accepted 2025-09-26 18:34:06,184 - lava_callback - INFO - Starting async job submit for node 68d4398ea5c2c05277bc92f0 2025-09-26 18:34:06,186 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:34:06,203 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4398ea5c2c05277bc92f0 HTTP/1.1" 200 None 2025-09-26 18:34:06,204 - lava_callback - DEBUG - Processing node 68d4398ea5c2c05277bc92f0: baseline-arm64-baylibre 2025-09-26 18:34:06,204 - lava_callback - ERROR - Error processing callback for node 68d4398ea5c2c05277bc92f0: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:37730 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59640 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59656 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34228 - "GET / HTTP/1.0" 200 OK 2025-09-26 18:34:26,173 - lava_callback - INFO - Received callback for node 68d4399ea5c2c05277bc92ff INFO: 172.21.0.1:34230 - "POST /node/68d4399ea5c2c05277bc92ff HTTP/1.0" 202 Accepted 2025-09-26 18:34:26,179 - lava_callback - INFO - Starting async job submit for node 68d4399ea5c2c05277bc92ff 2025-09-26 18:34:26,179 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:34:26,191 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4399ea5c2c05277bc92ff HTTP/1.1" 200 None 2025-09-26 18:34:26,192 - lava_callback - DEBUG - Processing node 68d4399ea5c2c05277bc92ff: baseline-arm64-baylibre 2025-09-26 18:34:26,192 - lava_callback - ERROR - Error processing callback for node 68d4399ea5c2c05277bc92ff: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:34246 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33996 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33998 - "GET / HTTP/1.0" 200 OK 2025-09-26 18:34:43,039 - lava_callback - INFO - Received callback for node 68d4641da5c2c05277bcb2fd INFO: 172.21.0.1:36490 - "POST /node/68d4641da5c2c05277bcb2fd HTTP/1.0" 202 Accepted 2025-09-26 18:34:43,045 - lava_callback - INFO - Starting async job submit for node 68d4641da5c2c05277bcb2fd 2025-09-26 18:34:43,065 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:34:43,082 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4641da5c2c05277bcb2fd HTTP/1.1" 200 None 2025-09-26 18:34:43,082 - lava_callback - DEBUG - Processing node 68d4641da5c2c05277bcb2fd: baseline-arm64 2025-09-26 18:34:43,376 - lava_callback - ERROR - Error processing callback for node 68d4641da5c2c05277bcb2fd: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:36496 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49800 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59502 - "GET / HTTP/1.0" 200 OK 2025-09-26 18:35:11,177 - lava_callback - INFO - Received callback for node 68d46422a5c2c05277bcb303 INFO: 172.21.0.1:49412 - "POST /node/68d46422a5c2c05277bcb303 HTTP/1.0" 202 Accepted 2025-09-26 18:35:11,183 - lava_callback - INFO - Starting async job submit for node 68d46422a5c2c05277bcb303 2025-09-26 18:35:11,210 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:35:11,223 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d46422a5c2c05277bcb303 HTTP/1.1" 200 None 2025-09-26 18:35:11,223 - lava_callback - DEBUG - Processing node 68d46422a5c2c05277bcb303: baseline-arm64 2025-09-26 18:35:11,582 - lava_callback - ERROR - Error processing callback for node 68d46422a5c2c05277bcb303: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:49414 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49424 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41348 - "GET / HTTP/1.0" 200 OK 2025-09-26 18:35:25,260 - lava_callback - INFO - Received callback for node 68d46422a5c2c05277bcb300 INFO: 172.21.0.1:41364 - "POST /node/68d46422a5c2c05277bcb300 HTTP/1.0" 202 Accepted 2025-09-26 18:35:25,267 - lava_callback - INFO - Starting async job submit for node 68d46422a5c2c05277bcb300 2025-09-26 18:35:25,291 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:35:25,305 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d46422a5c2c05277bcb300 HTTP/1.1" 200 None 2025-09-26 18:35:25,305 - lava_callback - DEBUG - Processing node 68d46422a5c2c05277bcb300: baseline-arm64 2025-09-26 18:35:25,702 - lava_callback - ERROR - Error processing callback for node 68d46422a5c2c05277bcb300: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:41374 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60596 - "GET / HTTP/1.0" 200 OK 2025-09-26 18:35:30,932 - lava_callback - INFO - Received callback for node 68d4c395a5c2c05277bcfdbc INFO: 172.21.0.1:60608 - "POST /node/68d4c395a5c2c05277bcfdbc HTTP/1.0" 202 Accepted 2025-09-26 18:35:30,938 - lava_callback - INFO - Starting async job submit for node 68d4c395a5c2c05277bcfdbc 2025-09-26 18:35:30,959 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:35:30,972 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4c395a5c2c05277bcfdbc HTTP/1.1" 200 None 2025-09-26 18:35:30,972 - lava_callback - DEBUG - Processing node 68d4c395a5c2c05277bcfdbc: baseline-arm 2025-09-26 18:35:31,254 - lava_callback - ERROR - Error processing callback for node 68d4c395a5c2c05277bcfdbc: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:60620 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36402 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46828 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45464 - "GET / HTTP/1.0" 200 OK 2025-09-26 18:36:10,729 - lava_callback - INFO - Received callback for node 68d46435a5c2c05277bcb323 INFO: 172.21.0.1:40692 - "POST /node/68d46435a5c2c05277bcb323 HTTP/1.0" 202 Accepted 2025-09-26 18:36:10,735 - lava_callback - INFO - Starting async job submit for node 68d46435a5c2c05277bcb323 2025-09-26 18:36:10,761 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:36:10,773 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d46435a5c2c05277bcb323 HTTP/1.1" 200 None 2025-09-26 18:36:10,774 - lava_callback - DEBUG - Processing node 68d46435a5c2c05277bcb323: baseline-arm64 2025-09-26 18:36:11,126 - lava_callback - ERROR - Error processing callback for node 68d46435a5c2c05277bcb323: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:40708 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40710 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59964 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59968 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41874 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41880 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49742 - "GET / HTTP/1.0" 200 OK 2025-09-26 18:36:48,330 - lava_callback - INFO - Received callback for node 68d4643aa5c2c05277bcb326 INFO: 172.21.0.1:49758 - "POST /node/68d4643aa5c2c05277bcb326 HTTP/1.0" 202 Accepted 2025-09-26 18:36:48,336 - lava_callback - INFO - Starting async job submit for node 68d4643aa5c2c05277bcb326 2025-09-26 18:36:48,377 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:36:48,390 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4643aa5c2c05277bcb326 HTTP/1.1" 200 None 2025-09-26 18:36:48,391 - lava_callback - DEBUG - Processing node 68d4643aa5c2c05277bcb326: baseline-arm64 2025-09-26 18:36:48,691 - lava_callback - ERROR - Error processing callback for node 68d4643aa5c2c05277bcb326: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:60924 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38836 - "GET / HTTP/1.0" 200 OK 2025-09-26 18:37:10,958 - lava_callback - INFO - Received callback for node 68d4643da5c2c05277bcb329 INFO: 172.21.0.1:37874 - "POST /node/68d4643da5c2c05277bcb329 HTTP/1.0" 202 Accepted 2025-09-26 18:37:10,964 - lava_callback - INFO - Starting async job submit for node 68d4643da5c2c05277bcb329 2025-09-26 18:37:10,984 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:37:10,996 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4643da5c2c05277bcb329 HTTP/1.1" 200 None 2025-09-26 18:37:10,997 - lava_callback - DEBUG - Processing node 68d4643da5c2c05277bcb329: baseline-arm64 2025-09-26 18:37:11,295 - lava_callback - ERROR - Error processing callback for node 68d4643da5c2c05277bcb329: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:37878 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37892 - "GET / HTTP/1.0" 200 OK 2025-09-26 18:37:17,990 - lava_callback - INFO - Received callback for node 68d4c395a5c2c05277bcfdbf INFO: 172.21.0.1:37908 - "POST /node/68d4c395a5c2c05277bcfdbf HTTP/1.0" 202 Accepted 2025-09-26 18:37:17,996 - lava_callback - INFO - Starting async job submit for node 68d4c395a5c2c05277bcfdbf 2025-09-26 18:37:18,017 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:37:18,030 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4c395a5c2c05277bcfdbf HTTP/1.1" 200 None 2025-09-26 18:37:18,030 - lava_callback - DEBUG - Processing node 68d4c395a5c2c05277bcfdbf: baseline-arm 2025-09-26 18:37:18,287 - lava_callback - ERROR - Error processing callback for node 68d4c395a5c2c05277bcfdbf: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:50464 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50474 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41828 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41842 - "GET / HTTP/1.0" 200 OK 2025-09-26 18:37:45,811 - lava_callback - INFO - Received callback for node 68d46432a5c2c05277bcb320 INFO: 172.21.0.1:43214 - "POST /node/68d46432a5c2c05277bcb320 HTTP/1.0" 202 Accepted 2025-09-26 18:37:45,839 - lava_callback - INFO - Starting async job submit for node 68d46432a5c2c05277bcb320 2025-09-26 18:37:45,859 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:37:45,872 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d46432a5c2c05277bcb320 HTTP/1.1" 200 None 2025-09-26 18:37:45,873 - lava_callback - DEBUG - Processing node 68d46432a5c2c05277bcb320: baseline-arm64 INFO: 172.21.0.1:43216 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46998 - "GET / HTTP/1.0" 200 OK 2025-09-26 18:37:58,833 - lava_callback - ERROR - Error processing callback for node 68d46432a5c2c05277bcb320: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None 2025-09-26 18:38:03,764 - lava_callback - INFO - Received callback for node 68d4649ba5c2c05277bcb392 INFO: 172.21.0.1:55986 - "POST /node/68d4649ba5c2c05277bcb392 HTTP/1.0" 202 Accepted 2025-09-26 18:38:03,772 - lava_callback - INFO - Starting async job submit for node 68d4649ba5c2c05277bcb392 2025-09-26 18:38:03,796 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:38:03,809 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4649ba5c2c05277bcb392 HTTP/1.1" 200 None 2025-09-26 18:38:03,809 - lava_callback - DEBUG - Processing node 68d4649ba5c2c05277bcb392: baseline-arm64 2025-09-26 18:38:04,172 - lava_callback - ERROR - Error processing callback for node 68d4649ba5c2c05277bcb392: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:55992 - "GET / HTTP/1.0" 200 OK 2025-09-26 18:38:13,970 - lava_callback - INFO - Received callback for node 68d4649da5c2c05277bcb395 INFO: 172.21.0.1:43238 - "POST /node/68d4649da5c2c05277bcb395 HTTP/1.0" 202 Accepted 2025-09-26 18:38:13,981 - lava_callback - INFO - Starting async job submit for node 68d4649da5c2c05277bcb395 2025-09-26 18:38:14,079 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:38:14,096 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4649da5c2c05277bcb395 HTTP/1.1" 200 None 2025-09-26 18:38:14,097 - lava_callback - DEBUG - Processing node 68d4649da5c2c05277bcb395: baseline-arm64 2025-09-26 18:38:14,527 - lava_callback - INFO - Received callback for node 68d4380ea5c2c05277bc90ca INFO: 172.21.0.1:43248 - "POST /node/68d4380ea5c2c05277bc90ca HTTP/1.0" 202 Accepted 2025-09-26 18:38:14,628 - lava_callback - INFO - Starting async job submit for node 68d4380ea5c2c05277bc90ca INFO: 172.21.0.1:43262 - "GET /api/metrics HTTP/1.0" 200 OK 2025-09-26 18:38:15,504 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:38:15,597 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4380ea5c2c05277bc90ca HTTP/1.1" 200 None 2025-09-26 18:38:15,614 - lava_callback - ERROR - Error processing callback for node 68d4649da5c2c05277bcb395: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None 2025-09-26 18:38:15,614 - lava_callback - DEBUG - Processing node 68d4380ea5c2c05277bc90ca: ltp-controllers INFO: 172.21.0.1:43274 - "GET / HTTP/1.0" 200 OK 2025-09-26 18:38:19,096 - lava_callback - ERROR - Error processing callback for node 68d4380ea5c2c05277bc90ca: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:53620 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53630 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41150 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41152 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37012 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53894 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39702 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54238 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54242 - "GET / HTTP/1.0" 200 OK 2025-09-26 18:39:24,560 - lava_callback - INFO - Received callback for node 68d464a5a5c2c05277bcb39b INFO: 172.21.0.1:60160 - "POST /node/68d464a5a5c2c05277bcb39b HTTP/1.0" 202 Accepted 2025-09-26 18:39:24,566 - lava_callback - INFO - Starting async job submit for node 68d464a5a5c2c05277bcb39b 2025-09-26 18:39:24,587 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:39:24,600 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d464a5a5c2c05277bcb39b HTTP/1.1" 200 None 2025-09-26 18:39:24,600 - lava_callback - DEBUG - Processing node 68d464a5a5c2c05277bcb39b: baseline-arm64 2025-09-26 18:39:24,906 - lava_callback - ERROR - Error processing callback for node 68d464a5a5c2c05277bcb39b: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:60162 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60164 - "GET / HTTP/1.0" 200 OK 2025-09-26 18:39:30,007 - lava_callback - INFO - Received callback for node 68d464bfa5c2c05277bcb3c7 INFO: 172.21.0.1:51346 - "POST /node/68d464bfa5c2c05277bcb3c7 HTTP/1.0" 202 Accepted 2025-09-26 18:39:30,014 - lava_callback - INFO - Starting async job submit for node 68d464bfa5c2c05277bcb3c7 2025-09-26 18:39:30,034 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.21.0.1:51362 - "GET / HTTP/1.0" 200 OK 2025-09-26 18:39:30,047 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d464bfa5c2c05277bcb3c7 HTTP/1.1" 200 None 2025-09-26 18:39:30,047 - lava_callback - DEBUG - Processing node 68d464bfa5c2c05277bcb3c7: baseline-arm64 2025-09-26 18:39:30,374 - lava_callback - ERROR - Error processing callback for node 68d464bfa5c2c05277bcb3c7: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None 2025-09-26 18:39:31,231 - lava_callback - INFO - Received callback for node 68d4c397a5c2c05277bcfdc2 INFO: 172.21.0.1:51378 - "POST /node/68d4c397a5c2c05277bcfdc2 HTTP/1.0" 202 Accepted 2025-09-26 18:39:31,237 - lava_callback - INFO - Starting async job submit for node 68d4c397a5c2c05277bcfdc2 2025-09-26 18:39:31,258 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:39:31,270 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4c397a5c2c05277bcfdc2 HTTP/1.1" 200 None 2025-09-26 18:39:31,271 - lava_callback - DEBUG - Processing node 68d4c397a5c2c05277bcfdc2: baseline-arm 2025-09-26 18:39:31,491 - lava_callback - ERROR - Error processing callback for node 68d4c397a5c2c05277bcfdc2: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None 2025-09-26 18:39:34,757 - lava_callback - INFO - Received callback for node 68d464a2a5c2c05277bcb398 INFO: 172.21.0.1:51388 - "POST /node/68d464a2a5c2c05277bcb398 HTTP/1.0" 202 Accepted 2025-09-26 18:39:34,764 - lava_callback - INFO - Starting async job submit for node 68d464a2a5c2c05277bcb398 2025-09-26 18:39:34,805 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:39:34,817 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d464a2a5c2c05277bcb398 HTTP/1.1" 200 None 2025-09-26 18:39:34,817 - lava_callback - DEBUG - Processing node 68d464a2a5c2c05277bcb398: baseline-arm64 2025-09-26 18:39:35,262 - lava_callback - ERROR - Error processing callback for node 68d464a2a5c2c05277bcb398: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:51390 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41932 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33494 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50328 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52720 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52722 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41718 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41724 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41080 - "GET / HTTP/1.0" 200 OK 2025-09-26 18:40:30,072 - lava_callback - INFO - Received callback for node 68d464c7a5c2c05277bcb3cd INFO: 172.21.0.1:41094 - "POST /node/68d464c7a5c2c05277bcb3cd HTTP/1.0" 202 Accepted 2025-09-26 18:40:30,079 - lava_callback - INFO - Starting async job submit for node 68d464c7a5c2c05277bcb3cd 2025-09-26 18:40:30,099 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:40:30,115 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d464c7a5c2c05277bcb3cd HTTP/1.1" 200 None 2025-09-26 18:40:30,116 - lava_callback - DEBUG - Processing node 68d464c7a5c2c05277bcb3cd: baseline-arm64 2025-09-26 18:40:30,431 - lava_callback - ERROR - Error processing callback for node 68d464c7a5c2c05277bcb3cd: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:41104 - "GET / HTTP/1.0" 200 OK 2025-09-26 18:40:43,301 - lava_callback - INFO - Received callback for node 68d464c2a5c2c05277bcb3ca INFO: 172.21.0.1:48078 - "POST /node/68d464c2a5c2c05277bcb3ca HTTP/1.0" 202 Accepted 2025-09-26 18:40:43,307 - lava_callback - INFO - Starting async job submit for node 68d464c2a5c2c05277bcb3ca 2025-09-26 18:40:43,328 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:40:43,340 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d464c2a5c2c05277bcb3ca HTTP/1.1" 200 None 2025-09-26 18:40:43,340 - lava_callback - DEBUG - Processing node 68d464c2a5c2c05277bcb3ca: baseline-arm64 2025-09-26 18:40:43,637 - lava_callback - ERROR - Error processing callback for node 68d464c2a5c2c05277bcb3ca: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:48086 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33754 - "GET / HTTP/1.0" 200 OK 2025-09-26 18:41:06,056 - lava_callback - INFO - Received callback for node 68d464caa5c2c05277bcb3d0 INFO: 172.21.0.1:41252 - "POST /node/68d464caa5c2c05277bcb3d0 HTTP/1.0" 202 Accepted 2025-09-26 18:41:06,063 - lava_callback - INFO - Starting async job submit for node 68d464caa5c2c05277bcb3d0 2025-09-26 18:41:06,088 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:41:06,104 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d464caa5c2c05277bcb3d0 HTTP/1.1" 200 None 2025-09-26 18:41:06,105 - lava_callback - DEBUG - Processing node 68d464caa5c2c05277bcb3d0: baseline-arm64 2025-09-26 18:41:06,459 - lava_callback - ERROR - Error processing callback for node 68d464caa5c2c05277bcb3d0: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:41266 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59864 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59880 - "GET / HTTP/1.0" 200 OK 2025-09-26 18:41:25,769 - lava_callback - INFO - Received callback for node 68d4c398a5c2c05277bcfdc5 INFO: 172.21.0.1:45418 - "POST /node/68d4c398a5c2c05277bcfdc5 HTTP/1.0" 202 Accepted 2025-09-26 18:41:25,775 - lava_callback - INFO - Starting async job submit for node 68d4c398a5c2c05277bcfdc5 2025-09-26 18:41:25,795 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:41:25,808 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4c398a5c2c05277bcfdc5 HTTP/1.1" 200 None 2025-09-26 18:41:25,808 - lava_callback - DEBUG - Processing node 68d4c398a5c2c05277bcfdc5: baseline-arm 2025-09-26 18:41:26,043 - lava_callback - ERROR - Error processing callback for node 68d4c398a5c2c05277bcfdc5: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:45422 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45426 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41268 - "GET / HTTP/1.0" 200 OK 2025-09-26 18:41:30,076 - lava_callback - INFO - Received callback for node 68d4656ca5c2c05277bcb497 INFO: 172.21.0.1:41272 - "POST /node/68d4656ca5c2c05277bcb497 HTTP/1.0" 202 Accepted 2025-09-26 18:41:30,082 - lava_callback - INFO - Starting async job submit for node 68d4656ca5c2c05277bcb497 2025-09-26 18:41:30,103 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:41:30,115 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4656ca5c2c05277bcb497 HTTP/1.1" 200 None 2025-09-26 18:41:30,116 - lava_callback - DEBUG - Processing node 68d4656ca5c2c05277bcb497: baseline-arm64 2025-09-26 18:41:30,415 - lava_callback - ERROR - Error processing callback for node 68d4656ca5c2c05277bcb497: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:41276 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48610 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33602 - "GET / HTTP/1.0" 200 OK 2025-09-26 18:42:04,013 - lava_callback - INFO - Received callback for node 68d4656fa5c2c05277bcb49a INFO: 172.21.0.1:47052 - "POST /node/68d4656fa5c2c05277bcb49a HTTP/1.0" 202 Accepted 2025-09-26 18:42:04,019 - lava_callback - INFO - Starting async job submit for node 68d4656fa5c2c05277bcb49a 2025-09-26 18:42:04,040 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:42:04,052 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4656fa5c2c05277bcb49a HTTP/1.1" 200 None 2025-09-26 18:42:04,053 - lava_callback - DEBUG - Processing node 68d4656fa5c2c05277bcb49a: baseline-arm64 2025-09-26 18:42:04,376 - lava_callback - ERROR - Error processing callback for node 68d4656fa5c2c05277bcb49a: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:47066 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39862 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39872 - "GET / HTTP/1.0" 200 OK 2025-09-26 18:42:25,622 - lava_callback - INFO - Received callback for node 68d46574a5c2c05277bcb49d INFO: 172.21.0.1:39798 - "POST /node/68d46574a5c2c05277bcb49d HTTP/1.0" 202 Accepted 2025-09-26 18:42:25,628 - lava_callback - INFO - Starting async job submit for node 68d46574a5c2c05277bcb49d 2025-09-26 18:42:25,649 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:42:25,661 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d46574a5c2c05277bcb49d HTTP/1.1" 200 None 2025-09-26 18:42:25,662 - lava_callback - DEBUG - Processing node 68d46574a5c2c05277bcb49d: baseline-arm64 2025-09-26 18:42:25,956 - lava_callback - ERROR - Error processing callback for node 68d46574a5c2c05277bcb49d: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:39806 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39810 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35396 - "GET / HTTP/1.0" 200 OK 2025-09-26 18:42:31,117 - lava_callback - INFO - Received callback for node 68d46576a5c2c05277bcb4a4 INFO: 172.21.0.1:35408 - "POST /node/68d46576a5c2c05277bcb4a4 HTTP/1.0" 202 Accepted 2025-09-26 18:42:31,123 - lava_callback - INFO - Starting async job submit for node 68d46576a5c2c05277bcb4a4 2025-09-26 18:42:31,144 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:42:31,161 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d46576a5c2c05277bcb4a4 HTTP/1.1" 200 None 2025-09-26 18:42:31,162 - lava_callback - DEBUG - Processing node 68d46576a5c2c05277bcb4a4: baseline-arm64 2025-09-26 18:42:31,459 - lava_callback - ERROR - Error processing callback for node 68d46576a5c2c05277bcb4a4: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:35422 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44812 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53302 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57266 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52770 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52784 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38164 - "GET / HTTP/1.0" 200 OK 2025-09-26 18:43:26,825 - lava_callback - INFO - Received callback for node 68d46586a5c2c05277bcb4b3 INFO: 172.21.0.1:38180 - "POST /node/68d46586a5c2c05277bcb4b3 HTTP/1.0" 202 Accepted 2025-09-26 18:43:26,831 - lava_callback - INFO - Starting async job submit for node 68d46586a5c2c05277bcb4b3 2025-09-26 18:43:26,851 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:43:26,864 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d46586a5c2c05277bcb4b3 HTTP/1.1" 200 None 2025-09-26 18:43:26,864 - lava_callback - DEBUG - Processing node 68d46586a5c2c05277bcb4b3: baseline-arm64 INFO: 172.21.0.1:38192 - "GET / HTTP/1.0" 200 OK 2025-09-26 18:43:27,186 - lava_callback - ERROR - Error processing callback for node 68d46586a5c2c05277bcb4b3: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:51210 - "GET / HTTP/1.0" 200 OK 2025-09-26 18:43:34,248 - lava_callback - INFO - Received callback for node 68d437fba5c2c05277bc908b INFO: 172.21.0.1:51214 - "POST /node/68d437fba5c2c05277bc908b HTTP/1.0" 202 Accepted 2025-09-26 18:43:34,254 - lava_callback - INFO - Starting async job submit for node 68d437fba5c2c05277bc908b 2025-09-26 18:43:34,284 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:43:34,302 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d437fba5c2c05277bc908b HTTP/1.1" 200 None 2025-09-26 18:43:34,302 - lava_callback - DEBUG - Processing node 68d437fba5c2c05277bc908b: baseline-arm 2025-09-26 18:43:34,569 - lava_callback - ERROR - Error processing callback for node 68d437fba5c2c05277bc908b: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:51224 - "GET / HTTP/1.0" 200 OK 2025-09-26 18:43:36,730 - lava_callback - INFO - Received callback for node 68d4658ba5c2c05277bcb4c7 INFO: 172.21.0.1:51234 - "POST /node/68d4658ba5c2c05277bcb4c7 HTTP/1.0" 202 Accepted 2025-09-26 18:43:36,736 - lava_callback - INFO - Starting async job submit for node 68d4658ba5c2c05277bcb4c7 2025-09-26 18:43:36,758 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:43:36,774 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4658ba5c2c05277bcb4c7 HTTP/1.1" 200 None 2025-09-26 18:43:36,774 - lava_callback - DEBUG - Processing node 68d4658ba5c2c05277bcb4c7: baseline-arm64 2025-09-26 18:43:37,135 - lava_callback - ERROR - Error processing callback for node 68d4658ba5c2c05277bcb4c7: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None 2025-09-26 18:43:43,218 - lava_callback - INFO - Received callback for node 68d46586a5c2c05277bcb4b6 INFO: 172.21.0.1:53650 - "POST /node/68d46586a5c2c05277bcb4b6 HTTP/1.0" 202 Accepted 2025-09-26 18:43:43,224 - lava_callback - INFO - Starting async job submit for node 68d46586a5c2c05277bcb4b6 2025-09-26 18:43:43,245 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:43:43,266 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d46586a5c2c05277bcb4b6 HTTP/1.1" 200 None 2025-09-26 18:43:43,267 - lava_callback - DEBUG - Processing node 68d46586a5c2c05277bcb4b6: baseline-arm64 2025-09-26 18:43:43,572 - lava_callback - ERROR - Error processing callback for node 68d46586a5c2c05277bcb4b6: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:53662 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33404 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41586 - "GET / HTTP/1.0" 200 OK 2025-09-26 18:44:06,687 - lava_callback - INFO - Received callback for node 68d43be9a5c2c05277bc95b6 INFO: 172.21.0.1:41594 - "POST /node/68d43be9a5c2c05277bc95b6 HTTP/1.0" 202 Accepted 2025-09-26 18:44:06,692 - lava_callback - INFO - Starting async job submit for node 68d43be9a5c2c05277bc95b6 2025-09-26 18:44:06,693 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:44:06,706 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d43be9a5c2c05277bc95b6 HTTP/1.1" 200 None 2025-09-26 18:44:06,706 - lava_callback - DEBUG - Processing node 68d43be9a5c2c05277bc95b6: baseline-arm64-baylibre 2025-09-26 18:44:06,706 - lava_callback - ERROR - Error processing callback for node 68d43be9a5c2c05277bc95b6: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:50562 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50568 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56142 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56146 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44014 - "GET / HTTP/1.0" 200 OK 2025-09-26 18:44:32,064 - lava_callback - INFO - Received callback for node 68d4c399a5c2c05277bcfdc8 INFO: 172.21.0.1:44026 - "POST /node/68d4c399a5c2c05277bcfdc8 HTTP/1.0" 202 Accepted 2025-09-26 18:44:32,070 - lava_callback - INFO - Starting async job submit for node 68d4c399a5c2c05277bcfdc8 2025-09-26 18:44:32,091 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:44:32,103 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4c399a5c2c05277bcfdc8 HTTP/1.1" 200 None 2025-09-26 18:44:32,104 - lava_callback - DEBUG - Processing node 68d4c399a5c2c05277bcfdc8: baseline-arm 2025-09-26 18:44:32,328 - lava_callback - ERROR - Error processing callback for node 68d4c399a5c2c05277bcfdc8: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None 2025-09-26 18:44:33,870 - lava_callback - INFO - Received callback for node 68d45538a5c2c05277bca94d INFO: 172.21.0.1:44028 - "POST /node/68d45538a5c2c05277bca94d HTTP/1.0" 202 Accepted 2025-09-26 18:44:33,876 - lava_callback - INFO - Starting async job submit for node 68d45538a5c2c05277bca94d 2025-09-26 18:44:33,897 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:44:33,912 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d45538a5c2c05277bca94d HTTP/1.1" 200 None 2025-09-26 18:44:33,913 - lava_callback - DEBUG - Processing node 68d45538a5c2c05277bca94d: baseline-arm-broonie 2025-09-26 18:44:34,151 - lava_callback - ERROR - Error processing callback for node 68d45538a5c2c05277bca94d: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:44040 - "GET / HTTP/1.0" 200 OK 2025-09-26 18:44:46,153 - lava_callback - INFO - Received callback for node 68d4658ca5c2c05277bcb4ca INFO: 172.21.0.1:58658 - "POST /node/68d4658ca5c2c05277bcb4ca HTTP/1.0" 202 Accepted 2025-09-26 18:44:46,159 - lava_callback - INFO - Starting async job submit for node 68d4658ca5c2c05277bcb4ca 2025-09-26 18:44:46,180 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:44:46,194 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4658ca5c2c05277bcb4ca HTTP/1.1" 200 None 2025-09-26 18:44:46,194 - lava_callback - DEBUG - Processing node 68d4658ca5c2c05277bcb4ca: baseline-arm64 2025-09-26 18:44:46,502 - lava_callback - ERROR - Error processing callback for node 68d4658ca5c2c05277bcb4ca: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:58666 - "GET / HTTP/1.0" 200 OK 2025-09-26 18:44:46,703 - lava_callback - INFO - Received callback for node 68d43c16a5c2c05277bc95f0 INFO: 172.21.0.1:58672 - "POST /node/68d43c16a5c2c05277bc95f0 HTTP/1.0" 202 Accepted 2025-09-26 18:44:46,709 - lava_callback - INFO - Starting async job submit for node 68d43c16a5c2c05277bc95f0 2025-09-26 18:44:46,710 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:44:46,722 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d43c16a5c2c05277bc95f0 HTTP/1.1" 200 None 2025-09-26 18:44:46,723 - lava_callback - DEBUG - Processing node 68d43c16a5c2c05277bc95f0: baseline-arm64-baylibre 2025-09-26 18:44:46,723 - lava_callback - ERROR - Error processing callback for node 68d43c16a5c2c05277bc95f0: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 18:44:51,291 - lava_callback - INFO - Received callback for node 68d465c5a5c2c05277bcb51c INFO: 172.21.0.1:48458 - "POST /node/68d465c5a5c2c05277bcb51c HTTP/1.0" 202 Accepted 2025-09-26 18:44:51,298 - lava_callback - INFO - Starting async job submit for node 68d465c5a5c2c05277bcb51c 2025-09-26 18:44:51,318 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:44:51,335 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d465c5a5c2c05277bcb51c HTTP/1.1" 200 None 2025-09-26 18:44:51,336 - lava_callback - DEBUG - Processing node 68d465c5a5c2c05277bcb51c: baseline-arm64 2025-09-26 18:44:51,636 - lava_callback - ERROR - Error processing callback for node 68d465c5a5c2c05277bcb51c: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:48464 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46746 - "GET / HTTP/1.0" 200 OK 2025-09-26 18:45:07,899 - lava_callback - INFO - Received callback for node 68d465c8a5c2c05277bcb51f INFO: 172.21.0.1:46750 - "POST /node/68d465c8a5c2c05277bcb51f HTTP/1.0" 202 Accepted 2025-09-26 18:45:07,905 - lava_callback - INFO - Starting async job submit for node 68d465c8a5c2c05277bcb51f 2025-09-26 18:45:07,926 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:45:07,960 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d465c8a5c2c05277bcb51f HTTP/1.1" 200 None 2025-09-26 18:45:07,960 - lava_callback - DEBUG - Processing node 68d465c8a5c2c05277bcb51f: baseline-arm64 2025-09-26 18:45:08,278 - lava_callback - ERROR - Error processing callback for node 68d465c8a5c2c05277bcb51f: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:34940 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34946 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35460 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35462 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49782 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49790 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33904 - "GET / HTTP/1.0" 200 OK 2025-09-26 18:45:46,836 - lava_callback - INFO - Received callback for node 68d43c56a5c2c05277bc9640 INFO: 172.21.0.1:33910 - "POST /node/68d43c56a5c2c05277bc9640 HTTP/1.0" 202 Accepted 2025-09-26 18:45:46,842 - lava_callback - INFO - Starting async job submit for node 68d43c56a5c2c05277bc9640 2025-09-26 18:45:46,843 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:45:46,879 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d43c56a5c2c05277bc9640 HTTP/1.1" 200 None 2025-09-26 18:45:46,880 - lava_callback - DEBUG - Processing node 68d43c56a5c2c05277bc9640: baseline-arm64-baylibre 2025-09-26 18:45:46,880 - lava_callback - ERROR - Error processing callback for node 68d43c56a5c2c05277bc9640: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 18:45:51,475 - lava_callback - INFO - Received callback for node 68d465cda5c2c05277bcb522 INFO: 172.21.0.1:55184 - "POST /node/68d465cda5c2c05277bcb522 HTTP/1.0" 202 Accepted 2025-09-26 18:45:51,481 - lava_callback - INFO - Starting async job submit for node 68d465cda5c2c05277bcb522 2025-09-26 18:45:51,502 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:45:51,514 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d465cda5c2c05277bcb522 HTTP/1.1" 200 None 2025-09-26 18:45:51,514 - lava_callback - DEBUG - Processing node 68d465cda5c2c05277bcb522: baseline-arm64 2025-09-26 18:45:51,807 - lava_callback - ERROR - Error processing callback for node 68d465cda5c2c05277bcb522: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:55200 - "GET / HTTP/1.0" 200 OK 2025-09-26 18:46:03,750 - lava_callback - INFO - Received callback for node 68d465d0a5c2c05277bcb525 INFO: 172.21.0.1:40294 - "POST /node/68d465d0a5c2c05277bcb525 HTTP/1.0" 202 Accepted 2025-09-26 18:46:03,756 - lava_callback - INFO - Starting async job submit for node 68d465d0a5c2c05277bcb525 2025-09-26 18:46:03,776 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:46:03,811 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d465d0a5c2c05277bcb525 HTTP/1.1" 200 None 2025-09-26 18:46:03,811 - lava_callback - DEBUG - Processing node 68d465d0a5c2c05277bcb525: baseline-arm64 2025-09-26 18:46:04,111 - lava_callback - ERROR - Error processing callback for node 68d465d0a5c2c05277bcb525: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:40296 - "GET / HTTP/1.0" 200 OK 2025-09-26 18:46:14,059 - lava_callback - INFO - Received callback for node 68d6daa23c94e462c6881f2d INFO: 172.21.0.1:49690 - "POST /node/68d6daa23c94e462c6881f2d HTTP/1.0" 202 Accepted 2025-09-26 18:46:14,065 - lava_callback - INFO - Starting async job submit for node 68d6daa23c94e462c6881f2d 2025-09-26 18:46:14,094 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:46:14,128 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6daa23c94e462c6881f2d HTTP/1.1" 200 None 2025-09-26 18:46:14,128 - lava_callback - DEBUG - Processing node 68d6daa23c94e462c6881f2d: baseline-x86-kcidebug-intel 2025-09-26 18:46:14,387 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 18:46:14,524 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 18:46:14,525 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-kcidebug-intel-68d6daa23c94e462c6881f2d/log.txt.gz 2025-09-26 18:46:14,531 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 18:46:14,669 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 18:46:14,669 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-kcidebug-intel-68d6daa23c94e462c6881f2d/lava_callback.json.gz DEBUG: Setup failed for 68d6daa23c94e462c6881f2d. Transitting job node result: incomplete -> fail 2025-09-26 18:46:14,728 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:46:14,747 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6daa23c94e462c6881f2d HTTP/1.1" 200 None 2025-09-26 18:46:14,748 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:46:14,759 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6b9f52f83206171a92b71 HTTP/1.1" 200 None 2025-09-26 18:46:14,760 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:46:14,786 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6daa23c94e462c6881f2d HTTP/1.1" 200 None 2025-09-26 18:46:14,786 - lava_callback - INFO - Completed processing callback for node 68d6daa23c94e462c6881f2d INFO: 172.21.0.1:49702 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49716 - "GET / HTTP/1.0" 200 OK 2025-09-26 18:46:23,353 - lava_callback - INFO - Received callback for node 68d465e2a5c2c05277bcb546 INFO: 172.21.0.1:46260 - "POST /node/68d465e2a5c2c05277bcb546 HTTP/1.0" 202 Accepted 2025-09-26 18:46:23,359 - lava_callback - INFO - Starting async job submit for node 68d465e2a5c2c05277bcb546 2025-09-26 18:46:23,379 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:46:23,392 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d465e2a5c2c05277bcb546 HTTP/1.1" 200 None 2025-09-26 18:46:23,392 - lava_callback - DEBUG - Processing node 68d465e2a5c2c05277bcb546: baseline-arm64 2025-09-26 18:46:23,692 - lava_callback - ERROR - Error processing callback for node 68d465e2a5c2c05277bcb546: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:46274 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46278 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39162 - "GET / HTTP/1.0" 200 OK 2025-09-26 18:46:30,874 - lava_callback - INFO - Received callback for node 68d4c39aa5c2c05277bcfdcb INFO: 172.21.0.1:39172 - "POST /node/68d4c39aa5c2c05277bcfdcb HTTP/1.0" 202 Accepted 2025-09-26 18:46:30,879 - lava_callback - INFO - Starting async job submit for node 68d4c39aa5c2c05277bcfdcb 2025-09-26 18:46:30,900 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:46:30,921 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4c39aa5c2c05277bcfdcb HTTP/1.1" 200 None 2025-09-26 18:46:30,921 - lava_callback - DEBUG - Processing node 68d4c39aa5c2c05277bcfdcb: baseline-arm 2025-09-26 18:46:31,148 - lava_callback - ERROR - Error processing callback for node 68d4c39aa5c2c05277bcfdcb: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None 2025-09-26 18:46:34,372 - lava_callback - INFO - Received callback for node 68d4380fa5c2c05277bc90cd INFO: 172.21.0.1:39180 - "POST /node/68d4380fa5c2c05277bc90cd HTTP/1.0" 202 Accepted 2025-09-26 18:46:34,382 - lava_callback - INFO - Starting async job submit for node 68d4380fa5c2c05277bc90cd 2025-09-26 18:46:34,507 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:46:34,519 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4380fa5c2c05277bc90cd HTTP/1.1" 200 None 2025-09-26 18:46:34,519 - lava_callback - DEBUG - Processing node 68d4380fa5c2c05277bc90cd: ltp-cve 2025-09-26 18:46:35,278 - lava_callback - ERROR - Error processing callback for node 68d4380fa5c2c05277bc90cd: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:39182 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42330 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41678 - "GET / HTTP/1.0" 200 OK 2025-09-26 18:46:56,788 - lava_callback - INFO - Received callback for node 68d465e3a5c2c05277bcb549 INFO: 172.21.0.1:41682 - "POST /node/68d465e3a5c2c05277bcb549 HTTP/1.0" 202 Accepted 2025-09-26 18:46:56,794 - lava_callback - INFO - Starting async job submit for node 68d465e3a5c2c05277bcb549 2025-09-26 18:46:56,815 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:46:56,836 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d465e3a5c2c05277bcb549 HTTP/1.1" 200 None 2025-09-26 18:46:56,836 - lava_callback - DEBUG - Processing node 68d465e3a5c2c05277bcb549: baseline-arm64 2025-09-26 18:46:57,138 - lava_callback - ERROR - Error processing callback for node 68d465e3a5c2c05277bcb549: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:49162 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50002 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50004 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60068 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60072 - "GET / HTTP/1.0" 200 OK 2025-09-26 18:47:29,126 - lava_callback - INFO - Received callback for node 68d465e7a5c2c05277bcb54c INFO: 172.21.0.1:60082 - "POST /node/68d465e7a5c2c05277bcb54c HTTP/1.0" 202 Accepted 2025-09-26 18:47:29,132 - lava_callback - INFO - Starting async job submit for node 68d465e7a5c2c05277bcb54c 2025-09-26 18:47:29,152 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:47:29,187 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d465e7a5c2c05277bcb54c HTTP/1.1" 200 None 2025-09-26 18:47:29,187 - lava_callback - DEBUG - Processing node 68d465e7a5c2c05277bcb54c: baseline-arm64 2025-09-26 18:47:29,488 - lava_callback - ERROR - Error processing callback for node 68d465e7a5c2c05277bcb54c: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:43278 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43290 - "GET / HTTP/1.0" 200 OK 2025-09-26 18:47:46,218 - lava_callback - INFO - Received callback for node 68d465eaa5c2c05277bcb54f INFO: 172.21.0.1:34660 - "POST /node/68d465eaa5c2c05277bcb54f HTTP/1.0" 202 Accepted 2025-09-26 18:47:46,224 - lava_callback - INFO - Starting async job submit for node 68d465eaa5c2c05277bcb54f 2025-09-26 18:47:46,245 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:47:46,257 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d465eaa5c2c05277bcb54f HTTP/1.1" 200 None 2025-09-26 18:47:46,258 - lava_callback - DEBUG - Processing node 68d465eaa5c2c05277bcb54f: baseline-arm64 INFO: 172.21.0.1:34668 - "GET / HTTP/1.0" 200 OK 2025-09-26 18:47:46,574 - lava_callback - ERROR - Error processing callback for node 68d465eaa5c2c05277bcb54f: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:33178 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34704 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58612 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58614 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57460 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57470 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58180 - "GET / HTTP/1.0" 200 OK 2025-09-26 18:48:31,237 - lava_callback - INFO - Received callback for node 68d4c3b7a5c2c05277bcfe16 INFO: 172.21.0.1:58186 - "POST /node/68d4c3b7a5c2c05277bcfe16 HTTP/1.0" 202 Accepted 2025-09-26 18:48:31,243 - lava_callback - INFO - Starting async job submit for node 68d4c3b7a5c2c05277bcfe16 2025-09-26 18:48:31,263 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:48:31,305 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4c3b7a5c2c05277bcfe16 HTTP/1.1" 200 None 2025-09-26 18:48:31,305 - lava_callback - DEBUG - Processing node 68d4c3b7a5c2c05277bcfe16: baseline-arm 2025-09-26 18:48:31,535 - lava_callback - ERROR - Error processing callback for node 68d4c3b7a5c2c05277bcfe16: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:58202 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48736 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60636 - "GET / HTTP/1.0" 200 OK 2025-09-26 18:48:59,492 - lava_callback - INFO - Received callback for node 68d42998a5c2c05277bc77a5 INFO: 172.21.0.1:57900 - "POST /node/68d42998a5c2c05277bc77a5 HTTP/1.0" 202 Accepted 2025-09-26 18:48:59,499 - lava_callback - INFO - Starting async job submit for node 68d42998a5c2c05277bc77a5 2025-09-26 18:48:59,524 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:48:59,560 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d42998a5c2c05277bc77a5 HTTP/1.1" 200 None 2025-09-26 18:48:59,560 - lava_callback - DEBUG - Processing node 68d42998a5c2c05277bc77a5: ltp-cap-bounds 2025-09-26 18:48:59,925 - lava_callback - ERROR - Error processing callback for node 68d42998a5c2c05277bc77a5: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:57904 - "GET / HTTP/1.0" 200 OK 2025-09-26 18:49:08,041 - lava_callback - INFO - Received callback for node 68d6c5373c94e462c688182a INFO: 172.21.0.1:57908 - "POST /node/68d6c5373c94e462c688182a HTTP/1.0" 202 Accepted 2025-09-26 18:49:08,047 - lava_callback - INFO - Starting async job submit for node 68d6c5373c94e462c688182a 2025-09-26 18:49:08,112 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:49:08,124 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6c5373c94e462c688182a HTTP/1.1" 200 None 2025-09-26 18:49:08,124 - lava_callback - DEBUG - Processing node 68d6c5373c94e462c688182a: ltp-smoketest 2025-09-26 18:49:08,254 - lava_callback - WARNING - Failed to upload log 2025-09-26 18:49:08,258 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 18:49:08,399 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 18:49:08,399 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoketest-68d6c5373c94e462c688182a/lava_callback.json.gz 2025-09-26 18:49:08,443 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:49:08,473 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6c5373c94e462c688182a HTTP/1.1" 200 None 2025-09-26 18:49:08,474 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:49:08,486 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d651d5cca86fb17472aa18 HTTP/1.1" 200 None 2025-09-26 18:49:08,487 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:49:08,537 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6c5373c94e462c688182a HTTP/1.1" 200 None 2025-09-26 18:49:08,541 - lava_callback - INFO - Completed processing callback for node 68d6c5373c94e462c688182a INFO: 172.21.0.1:41726 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41732 - "GET / HTTP/1.0" 200 OK 2025-09-26 18:49:19,029 - lava_callback - INFO - Received callback for node 68d6c78d3c94e462c68818b3 INFO: 172.21.0.1:41740 - "POST /node/68d6c78d3c94e462c68818b3 HTTP/1.0" 202 Accepted 2025-09-26 18:49:19,034 - lava_callback - INFO - Starting async job submit for node 68d6c78d3c94e462c68818b3 2025-09-26 18:49:19,069 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:49:19,081 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6c78d3c94e462c68818b3 HTTP/1.1" 200 None 2025-09-26 18:49:19,082 - lava_callback - DEBUG - Processing node 68d6c78d3c94e462c68818b3: baseline-arm64-broonie 2025-09-26 18:49:19,212 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 18:49:19,345 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 18:49:19,346 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-68d6c78d3c94e462c68818b3/log.txt.gz 2025-09-26 18:49:19,350 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 18:49:19,484 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 18:49:19,485 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-68d6c78d3c94e462c68818b3/lava_callback.json.gz 2025-09-26 18:49:19,520 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:49:19,532 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6c78d3c94e462c68818b3 HTTP/1.1" 200 None 2025-09-26 18:49:19,533 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:49:19,546 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6b9dd2f83206171a92b48 HTTP/1.1" 200 None 2025-09-26 18:49:19,547 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:49:19,575 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6c78d3c94e462c68818b3 HTTP/1.1" 200 None 2025-09-26 18:49:19,576 - lava_callback - INFO - Completed processing callback for node 68d6c78d3c94e462c68818b3 INFO: 172.21.0.1:44414 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44418 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51174 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51184 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46546 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41066 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57102 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44570 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44584 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48292 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48304 - "GET / HTTP/1.0" 200 OK 2025-09-26 18:50:30,020 - lava_callback - INFO - Received callback for node 68d4c3b8a5c2c05277bcfe19 INFO: 172.21.0.1:59966 - "POST /node/68d4c3b8a5c2c05277bcfe19 HTTP/1.0" 202 Accepted 2025-09-26 18:50:30,029 - lava_callback - INFO - Starting async job submit for node 68d4c3b8a5c2c05277bcfe19 2025-09-26 18:50:30,050 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:50:30,069 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4c3b8a5c2c05277bcfe19 HTTP/1.1" 200 None 2025-09-26 18:50:30,069 - lava_callback - DEBUG - Processing node 68d4c3b8a5c2c05277bcfe19: baseline-arm INFO: 172.21.0.1:59976 - "GET / HTTP/1.0" 200 OK 2025-09-26 18:50:30,303 - lava_callback - ERROR - Error processing callback for node 68d4c3b8a5c2c05277bcfe19: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:59992 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51836 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51734 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37550 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60744 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60758 - "GET / HTTP/1.0" 200 OK 2025-09-26 18:51:24,172 - lava_callback - INFO - Received callback for node 68d6c78f3c94e462c68818bf INFO: 172.21.0.1:40626 - "POST /node/68d6c78f3c94e462c68818bf HTTP/1.0" 202 Accepted 2025-09-26 18:51:24,178 - lava_callback - INFO - Starting async job submit for node 68d6c78f3c94e462c68818bf 2025-09-26 18:51:24,199 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:51:24,220 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6c78f3c94e462c68818bf HTTP/1.1" 200 None 2025-09-26 18:51:24,220 - lava_callback - DEBUG - Processing node 68d6c78f3c94e462c68818bf: baseline-arm64-broonie 2025-09-26 18:51:24,482 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 18:51:24,629 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 18:51:24,630 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-68d6c78f3c94e462c68818bf/log.txt.gz 2025-09-26 18:51:24,636 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 18:51:24,760 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 18:51:24,761 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-68d6c78f3c94e462c68818bf/lava_callback.json.gz 2025-09-26 18:51:24,782 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:51:24,794 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6c78f3c94e462c68818bf HTTP/1.1" 200 None 2025-09-26 18:51:24,795 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:51:24,807 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6b9dd2f83206171a92b48 HTTP/1.1" 200 None 2025-09-26 18:51:24,807 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:51:24,839 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6c78f3c94e462c68818bf HTTP/1.1" 200 None 2025-09-26 18:51:24,840 - lava_callback - INFO - Completed processing callback for node 68d6c78f3c94e462c68818bf INFO: 172.21.0.1:40638 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40654 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40930 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40936 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54344 - "GET / HTTP/1.0" 200 OK 2025-09-26 18:51:47,205 - lava_callback - INFO - Received callback for node 68d43daba5c2c05277bc97eb INFO: 172.21.0.1:54358 - "POST /node/68d43daba5c2c05277bc97eb HTTP/1.0" 202 Accepted 2025-09-26 18:51:47,212 - lava_callback - INFO - Starting async job submit for node 68d43daba5c2c05277bc97eb 2025-09-26 18:51:47,212 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:51:47,225 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d43daba5c2c05277bc97eb HTTP/1.1" 200 None 2025-09-26 18:51:47,225 - lava_callback - DEBUG - Processing node 68d43daba5c2c05277bc97eb: baseline-arm64-baylibre 2025-09-26 18:51:47,225 - lava_callback - ERROR - Error processing callback for node 68d43daba5c2c05277bc97eb: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 18:51:47,905 - lava_callback - INFO - Received callback for node 68d43db6a5c2c05277bc97f1 INFO: 172.21.0.1:54360 - "POST /node/68d43db6a5c2c05277bc97f1 HTTP/1.0" 202 Accepted 2025-09-26 18:51:47,911 - lava_callback - INFO - Starting async job submit for node 68d43db6a5c2c05277bc97f1 2025-09-26 18:51:47,912 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:51:47,934 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d43db6a5c2c05277bc97f1 HTTP/1.1" 200 None 2025-09-26 18:51:47,934 - lava_callback - DEBUG - Processing node 68d43db6a5c2c05277bc97f1: baseline-arm64-baylibre 2025-09-26 18:51:47,934 - lava_callback - ERROR - Error processing callback for node 68d43db6a5c2c05277bc97f1: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 18:51:48,582 - lava_callback - INFO - Received callback for node 68d43dbda5c2c05277bc97f7 INFO: 172.21.0.1:54368 - "POST /node/68d43dbda5c2c05277bc97f7 HTTP/1.0" 202 Accepted 2025-09-26 18:51:48,587 - lava_callback - INFO - Starting async job submit for node 68d43dbda5c2c05277bc97f7 2025-09-26 18:51:48,588 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:51:48,601 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d43dbda5c2c05277bc97f7 HTTP/1.1" 200 None 2025-09-26 18:51:48,601 - lava_callback - DEBUG - Processing node 68d43dbda5c2c05277bc97f7: baseline-arm64-baylibre 2025-09-26 18:51:48,601 - lava_callback - ERROR - Error processing callback for node 68d43dbda5c2c05277bc97f7: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:46466 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59140 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34618 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34624 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54900 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47270 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47276 - "GET / HTTP/1.0" 200 OK 2025-09-26 18:52:31,216 - lava_callback - INFO - Received callback for node 68d4c3b9a5c2c05277bcfe1c INFO: 172.21.0.1:47292 - "POST /node/68d4c3b9a5c2c05277bcfe1c HTTP/1.0" 202 Accepted 2025-09-26 18:52:31,222 - lava_callback - INFO - Starting async job submit for node 68d4c3b9a5c2c05277bcfe1c 2025-09-26 18:52:31,243 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:52:31,277 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4c3b9a5c2c05277bcfe1c HTTP/1.1" 200 None 2025-09-26 18:52:31,277 - lava_callback - DEBUG - Processing node 68d4c3b9a5c2c05277bcfe1c: baseline-arm 2025-09-26 18:52:31,501 - lava_callback - ERROR - Error processing callback for node 68d4c3b9a5c2c05277bcfe1c: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:47306 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47644 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42392 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40150 - "GET / HTTP/1.0" 200 OK 2025-09-26 18:53:09,318 - lava_callback - INFO - Received callback for node 68d6c5c13c94e462c6881839 INFO: 172.21.0.1:41938 - "POST /node/68d6c5c13c94e462c6881839 HTTP/1.0" 202 Accepted 2025-09-26 18:53:09,324 - lava_callback - INFO - Starting async job submit for node 68d6c5c13c94e462c6881839 2025-09-26 18:53:09,345 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:53:09,358 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6c5c13c94e462c6881839 HTTP/1.1" 200 None 2025-09-26 18:53:09,359 - lava_callback - DEBUG - Processing node 68d6c5c13c94e462c6881839: baseline-arm 2025-09-26 18:53:09,541 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 18:53:09,685 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 18:53:09,686 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-68d6c5c13c94e462c6881839/log.txt.gz 2025-09-26 18:53:09,691 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 18:53:09,833 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 18:53:09,833 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-68d6c5c13c94e462c6881839/lava_callback.json.gz 2025-09-26 18:53:09,855 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:53:09,867 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6c5c13c94e462c6881839 HTTP/1.1" 200 None 2025-09-26 18:53:09,868 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:53:09,889 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6bd813c94e462c68814e8 HTTP/1.1" 200 None 2025-09-26 18:53:09,890 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:53:09,919 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6c5c13c94e462c6881839 HTTP/1.1" 200 None 2025-09-26 18:53:09,919 - lava_callback - INFO - Completed processing callback for node 68d6c5c13c94e462c6881839 INFO: 172.21.0.1:41950 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41952 - "GET / HTTP/1.0" 200 OK 2025-09-26 18:53:24,025 - lava_callback - INFO - Received callback for node 68d6d48d3c94e462c6881e3c INFO: 172.21.0.1:36146 - "POST /node/68d6d48d3c94e462c6881e3c HTTP/1.0" 202 Accepted 2025-09-26 18:53:24,031 - lava_callback - INFO - Starting async job submit for node 68d6d48d3c94e462c6881e3c 2025-09-26 18:53:24,052 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:53:24,066 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6d48d3c94e462c6881e3c HTTP/1.1" 200 None 2025-09-26 18:53:24,066 - lava_callback - DEBUG - Processing node 68d6d48d3c94e462c6881e3c: baseline-arm64-broonie 2025-09-26 18:53:24,305 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 18:53:24,432 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 18:53:24,433 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-68d6d48d3c94e462c6881e3c/log.txt.gz 2025-09-26 18:53:24,439 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 18:53:24,575 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 18:53:24,576 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-68d6d48d3c94e462c6881e3c/lava_callback.json.gz 2025-09-26 18:53:24,598 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:53:24,610 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6d48d3c94e462c6881e3c HTTP/1.1" 200 None 2025-09-26 18:53:24,611 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:53:24,622 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6bd8e3c94e462c68814f1 HTTP/1.1" 200 None 2025-09-26 18:53:24,623 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:53:24,654 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6d48d3c94e462c6881e3c HTTP/1.1" 200 None 2025-09-26 18:53:24,655 - lava_callback - INFO - Completed processing callback for node 68d6d48d3c94e462c6881e3c INFO: 172.21.0.1:36154 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43130 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43134 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43146 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42124 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44572 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43104 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54882 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54898 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51770 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57800 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57814 - "GET / HTTP/1.0" 200 OK 2025-09-26 18:54:35,208 - lava_callback - INFO - Received callback for node 68d4c3baa5c2c05277bcfe1f INFO: 172.21.0.1:57820 - "POST /node/68d4c3baa5c2c05277bcfe1f HTTP/1.0" 202 Accepted 2025-09-26 18:54:35,214 - lava_callback - INFO - Starting async job submit for node 68d4c3baa5c2c05277bcfe1f 2025-09-26 18:54:35,235 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:54:35,247 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4c3baa5c2c05277bcfe1f HTTP/1.1" 200 None 2025-09-26 18:54:35,247 - lava_callback - DEBUG - Processing node 68d4c3baa5c2c05277bcfe1f: baseline-arm 2025-09-26 18:54:35,490 - lava_callback - ERROR - Error processing callback for node 68d4c3baa5c2c05277bcfe1f: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:57832 - "GET / HTTP/1.0" 200 OK 2025-09-26 18:54:45,469 - lava_callback - INFO - Received callback for node 68d493baa5c2c05277bceeba INFO: 172.21.0.1:41366 - "POST /node/68d493baa5c2c05277bceeba HTTP/1.0" 202 Accepted 2025-09-26 18:54:45,475 - lava_callback - INFO - Starting async job submit for node 68d493baa5c2c05277bceeba 2025-09-26 18:54:45,493 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:54:45,508 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d493baa5c2c05277bceeba HTTP/1.1" 200 None 2025-09-26 18:54:45,509 - lava_callback - DEBUG - Processing node 68d493baa5c2c05277bceeba: baseline-arm64-broonie 2025-09-26 18:54:45,807 - lava_callback - ERROR - Error processing callback for node 68d493baa5c2c05277bceeba: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:41376 - "GET / HTTP/1.0" 200 OK 2025-09-26 18:54:47,379 - lava_callback - INFO - Received callback for node 68d43e64a5c2c05277bc98a7 INFO: 172.21.0.1:41378 - "POST /node/68d43e64a5c2c05277bc98a7 HTTP/1.0" 202 Accepted 2025-09-26 18:54:47,385 - lava_callback - INFO - Starting async job submit for node 68d43e64a5c2c05277bc98a7 2025-09-26 18:54:47,385 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:54:47,397 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d43e64a5c2c05277bc98a7 HTTP/1.1" 200 None 2025-09-26 18:54:47,398 - lava_callback - DEBUG - Processing node 68d43e64a5c2c05277bc98a7: baseline-arm64-baylibre 2025-09-26 18:54:47,398 - lava_callback - ERROR - Error processing callback for node 68d43e64a5c2c05277bc98a7: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:36616 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34590 - "GET / HTTP/1.0" 200 OK 2025-09-26 18:55:07,367 - lava_callback - INFO - Received callback for node 68d43e7ba5c2c05277bc98b6 INFO: 172.21.0.1:34598 - "POST /node/68d43e7ba5c2c05277bc98b6 HTTP/1.0" 202 Accepted 2025-09-26 18:55:07,372 - lava_callback - INFO - Starting async job submit for node 68d43e7ba5c2c05277bc98b6 2025-09-26 18:55:07,373 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:55:07,385 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d43e7ba5c2c05277bc98b6 HTTP/1.1" 200 None 2025-09-26 18:55:07,386 - lava_callback - DEBUG - Processing node 68d43e7ba5c2c05277bc98b6: baseline-arm64-baylibre 2025-09-26 18:55:07,386 - lava_callback - ERROR - Error processing callback for node 68d43e7ba5c2c05277bc98b6: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:43032 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43034 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38116 - "GET / HTTP/1.0" 200 OK 2025-09-26 18:55:27,376 - lava_callback - INFO - Received callback for node 68d43e8ba5c2c05277bc98c2 INFO: 172.21.0.1:38118 - "POST /node/68d43e8ba5c2c05277bc98c2 HTTP/1.0" 202 Accepted 2025-09-26 18:55:27,381 - lava_callback - INFO - Starting async job submit for node 68d43e8ba5c2c05277bc98c2 2025-09-26 18:55:27,382 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:55:27,403 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d43e8ba5c2c05277bc98c2 HTTP/1.1" 200 None 2025-09-26 18:55:27,403 - lava_callback - DEBUG - Processing node 68d43e8ba5c2c05277bc98c2: baseline-arm64-baylibre 2025-09-26 18:55:27,403 - lava_callback - ERROR - Error processing callback for node 68d43e8ba5c2c05277bc98c2: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:55578 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55594 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55606 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42380 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34990 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56734 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54994 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54998 - "GET / HTTP/1.0" 200 OK 2025-09-26 18:56:24,578 - lava_callback - INFO - Received callback for node 68d44acda5c2c05277bca046 INFO: 172.21.0.1:53670 - "POST /node/68d44acda5c2c05277bca046 HTTP/1.0" 202 Accepted 2025-09-26 18:56:24,585 - lava_callback - INFO - Starting async job submit for node 68d44acda5c2c05277bca046 2025-09-26 18:56:24,618 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:56:24,640 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d44acda5c2c05277bca046 HTTP/1.1" 200 None 2025-09-26 18:56:24,640 - lava_callback - DEBUG - Processing node 68d44acda5c2c05277bca046: ltp-sched 2025-09-26 18:56:25,142 - lava_callback - ERROR - Error processing callback for node 68d44acda5c2c05277bca046: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:53674 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52002 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52004 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52010 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40320 - "GET / HTTP/1.0" 200 OK 2025-09-26 18:56:50,240 - lava_callback - INFO - Received callback for node 68d4c3bba5c2c05277bcfe22 INFO: 172.21.0.1:53182 - "POST /node/68d4c3bba5c2c05277bcfe22 HTTP/1.0" 202 Accepted 2025-09-26 18:56:50,246 - lava_callback - INFO - Starting async job submit for node 68d4c3bba5c2c05277bcfe22 2025-09-26 18:56:50,267 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:56:50,279 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4c3bba5c2c05277bcfe22 HTTP/1.1" 200 None 2025-09-26 18:56:50,280 - lava_callback - DEBUG - Processing node 68d4c3bba5c2c05277bcfe22: baseline-arm 2025-09-26 18:56:50,509 - lava_callback - ERROR - Error processing callback for node 68d4c3bba5c2c05277bcfe22: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:53190 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52222 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57310 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57326 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58694 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58404 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58412 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58414 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55052 - "GET / HTTP/1.0" 200 OK 2025-09-26 18:57:54,638 - lava_callback - INFO - Received callback for node 68d44965a5c2c05277bc9fcb INFO: 172.21.0.1:59938 - "POST /node/68d44965a5c2c05277bc9fcb HTTP/1.0" 202 Accepted 2025-09-26 18:57:54,644 - lava_callback - INFO - Starting async job submit for node 68d44965a5c2c05277bc9fcb 2025-09-26 18:57:54,672 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:57:54,699 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d44965a5c2c05277bc9fcb HTTP/1.1" 200 None 2025-09-26 18:57:54,700 - lava_callback - DEBUG - Processing node 68d44965a5c2c05277bc9fcb: ltp-sched 2025-09-26 18:57:55,057 - lava_callback - ERROR - Error processing callback for node 68d44965a5c2c05277bc9fcb: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:59950 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56784 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39416 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39424 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47616 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35380 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35390 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35394 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38060 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39010 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39556 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52914 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52916 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38494 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52248 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52250 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52260 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58044 - "GET / HTTP/1.0" 200 OK 2025-09-26 18:59:51,175 - lava_callback - INFO - Received callback for node 68d4c3bca5c2c05277bcfe25 INFO: 172.21.0.1:44462 - "POST /node/68d4c3bca5c2c05277bcfe25 HTTP/1.0" 202 Accepted 2025-09-26 18:59:51,181 - lava_callback - INFO - Starting async job submit for node 68d4c3bca5c2c05277bcfe25 2025-09-26 18:59:51,203 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 18:59:51,243 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4c3bca5c2c05277bcfe25 HTTP/1.1" 200 None 2025-09-26 18:59:51,244 - lava_callback - DEBUG - Processing node 68d4c3bca5c2c05277bcfe25: baseline-arm 2025-09-26 18:59:51,478 - lava_callback - ERROR - Error processing callback for node 68d4c3bca5c2c05277bcfe25: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:44478 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51430 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45338 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45340 - "GET / HTTP/1.0" 200 OK 2025-09-26 19:00:22,929 - lava_callback - INFO - Received callback for node 68d46623a5c2c05277bcb594 INFO: 172.21.0.1:50842 - "POST /node/68d46623a5c2c05277bcb594 HTTP/1.0" 202 Accepted 2025-09-26 19:00:22,936 - lava_callback - INFO - Starting async job submit for node 68d46623a5c2c05277bcb594 2025-09-26 19:00:22,956 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:00:22,982 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d46623a5c2c05277bcb594 HTTP/1.1" 200 None 2025-09-26 19:00:22,983 - lava_callback - DEBUG - Processing node 68d46623a5c2c05277bcb594: baseline-arm64 2025-09-26 19:00:23,280 - lava_callback - ERROR - Error processing callback for node 68d46623a5c2c05277bcb594: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:50850 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53912 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53928 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53932 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51500 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42856 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58056 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39526 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39540 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41416 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44564 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44570 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44576 - "GET / HTTP/1.0" 200 OK 2025-09-26 19:01:45,879 - lava_callback - INFO - Received callback for node 68d46626a5c2c05277bcb597 INFO: 172.21.0.1:54432 - "POST /node/68d46626a5c2c05277bcb597 HTTP/1.0" 202 Accepted 2025-09-26 19:01:45,886 - lava_callback - INFO - Starting async job submit for node 68d46626a5c2c05277bcb597 2025-09-26 19:01:45,905 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:01:45,918 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d46626a5c2c05277bcb597 HTTP/1.1" 200 None 2025-09-26 19:01:45,918 - lava_callback - DEBUG - Processing node 68d46626a5c2c05277bcb597: baseline-arm64 INFO: 172.21.0.1:54442 - "GET / HTTP/1.0" 200 OK 2025-09-26 19:01:46,801 - lava_callback - ERROR - Error processing callback for node 68d46626a5c2c05277bcb597: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None 2025-09-26 19:01:54,656 - lava_callback - INFO - Received callback for node 68d4662ba5c2c05277bcb59a INFO: 172.21.0.1:60116 - "POST /node/68d4662ba5c2c05277bcb59a HTTP/1.0" 202 Accepted 2025-09-26 19:01:54,662 - lava_callback - INFO - Starting async job submit for node 68d4662ba5c2c05277bcb59a 2025-09-26 19:01:54,682 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:01:54,699 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4662ba5c2c05277bcb59a HTTP/1.1" 200 None 2025-09-26 19:01:54,699 - lava_callback - DEBUG - Processing node 68d4662ba5c2c05277bcb59a: baseline-arm64 2025-09-26 19:01:55,018 - lava_callback - ERROR - Error processing callback for node 68d4662ba5c2c05277bcb59a: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:60132 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41202 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35592 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35608 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44308 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54360 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54368 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54370 - "GET / HTTP/1.0" 200 OK 2025-09-26 19:02:45,823 - lava_callback - INFO - Received callback for node 68d4c3bea5c2c05277bcfe28 INFO: 172.21.0.1:59268 - "POST /node/68d4c3bea5c2c05277bcfe28 HTTP/1.0" 202 Accepted 2025-09-26 19:02:45,829 - lava_callback - INFO - Starting async job submit for node 68d4c3bea5c2c05277bcfe28 2025-09-26 19:02:45,850 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:02:45,865 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4c3bea5c2c05277bcfe28 HTTP/1.1" 200 None 2025-09-26 19:02:45,865 - lava_callback - DEBUG - Processing node 68d4c3bea5c2c05277bcfe28: baseline-arm 2025-09-26 19:02:46,091 - lava_callback - ERROR - Error processing callback for node 68d4c3bea5c2c05277bcfe28: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:59272 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55002 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47776 - "GET / HTTP/1.0" 200 OK 2025-09-26 19:03:11,011 - lava_callback - INFO - Received callback for node 68d4662ca5c2c05277bcb59d INFO: 172.21.0.1:34432 - "POST /node/68d4662ca5c2c05277bcb59d HTTP/1.0" 202 Accepted 2025-09-26 19:03:11,017 - lava_callback - INFO - Starting async job submit for node 68d4662ca5c2c05277bcb59d 2025-09-26 19:03:11,038 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:03:11,053 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4662ca5c2c05277bcb59d HTTP/1.1" 200 None 2025-09-26 19:03:11,053 - lava_callback - DEBUG - Processing node 68d4662ca5c2c05277bcb59d: baseline-arm64 2025-09-26 19:03:11,346 - lava_callback - ERROR - Error processing callback for node 68d4662ca5c2c05277bcb59d: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None 2025-09-26 19:03:13,883 - lava_callback - INFO - Received callback for node 68d44405a5c2c05277bc9c2e INFO: 172.21.0.1:34438 - "POST /node/68d44405a5c2c05277bc9c2e HTTP/1.0" 202 Accepted 2025-09-26 19:03:13,889 - lava_callback - INFO - Starting async job submit for node 68d44405a5c2c05277bc9c2e 2025-09-26 19:03:13,919 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:03:13,932 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d44405a5c2c05277bc9c2e HTTP/1.1" 200 None 2025-09-26 19:03:13,932 - lava_callback - DEBUG - Processing node 68d44405a5c2c05277bc9c2e: baseline-arm 2025-09-26 19:03:14,215 - lava_callback - ERROR - Error processing callback for node 68d44405a5c2c05277bc9c2e: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:34442 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34448 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42998 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34292 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34300 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34304 - "GET / HTTP/1.0" 200 OK 2025-09-26 19:03:44,519 - lava_callback - INFO - Received callback for node 68d4b2e1a5c2c05277bcf844 INFO: 172.21.0.1:38452 - "POST /node/68d4b2e1a5c2c05277bcf844 HTTP/1.0" 202 Accepted 2025-09-26 19:03:44,525 - lava_callback - INFO - Starting async job submit for node 68d4b2e1a5c2c05277bcf844 2025-09-26 19:03:44,543 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:03:44,555 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4b2e1a5c2c05277bcf844 HTTP/1.1" 200 None 2025-09-26 19:03:44,555 - lava_callback - DEBUG - Processing node 68d4b2e1a5c2c05277bcf844: baseline-arm64-broonie 2025-09-26 19:03:44,854 - lava_callback - ERROR - Error processing callback for node 68d4b2e1a5c2c05277bcf844: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None 2025-09-26 19:03:45,961 - lava_callback - INFO - Received callback for node 68d6c78c3c94e462c68818b0 INFO: 172.21.0.1:38468 - "POST /node/68d6c78c3c94e462c68818b0 HTTP/1.0" 202 Accepted 2025-09-26 19:03:45,967 - lava_callback - INFO - Starting async job submit for node 68d6c78c3c94e462c68818b0 2025-09-26 19:03:45,988 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:03:46,022 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6c78c3c94e462c68818b0 HTTP/1.1" 200 None 2025-09-26 19:03:46,022 - lava_callback - DEBUG - Processing node 68d6c78c3c94e462c68818b0: baseline-arm64-broonie 2025-09-26 19:03:46,283 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 19:03:46,440 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 19:03:46,441 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-68d6c78c3c94e462c68818b0/log.txt.gz 2025-09-26 19:03:46,448 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.21.0.1:38476 - "GET / HTTP/1.0" 200 OK 2025-09-26 19:03:46,586 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 19:03:46,587 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-68d6c78c3c94e462c68818b0/lava_callback.json.gz 2025-09-26 19:03:46,628 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:03:46,666 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6c78c3c94e462c68818b0 HTTP/1.1" 200 None 2025-09-26 19:03:46,667 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:03:46,678 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6b9dd2f83206171a92b48 HTTP/1.1" 200 None 2025-09-26 19:03:46,679 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:03:46,705 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6c78c3c94e462c68818b0 HTTP/1.1" 200 None 2025-09-26 19:03:46,706 - lava_callback - INFO - Completed processing callback for node 68d6c78c3c94e462c68818b0 2025-09-26 19:03:47,243 - lava_callback - INFO - Received callback for node 68d4662fa5c2c05277bcb5a0 INFO: 172.21.0.1:38482 - "POST /node/68d4662fa5c2c05277bcb5a0 HTTP/1.0" 202 Accepted 2025-09-26 19:03:47,252 - lava_callback - INFO - Starting async job submit for node 68d4662fa5c2c05277bcb5a0 2025-09-26 19:03:47,273 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:03:47,285 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4662fa5c2c05277bcb5a0 HTTP/1.1" 200 None 2025-09-26 19:03:47,286 - lava_callback - DEBUG - Processing node 68d4662fa5c2c05277bcb5a0: baseline-arm64 2025-09-26 19:03:47,582 - lava_callback - ERROR - Error processing callback for node 68d4662fa5c2c05277bcb5a0: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:43774 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37310 - "GET / HTTP/1.0" 200 OK 2025-09-26 19:04:13,799 - lava_callback - INFO - Received callback for node 68d46631a5c2c05277bcb5a3 INFO: 172.21.0.1:57852 - "POST /node/68d46631a5c2c05277bcb5a3 HTTP/1.0" 202 Accepted 2025-09-26 19:04:13,805 - lava_callback - INFO - Starting async job submit for node 68d46631a5c2c05277bcb5a3 2025-09-26 19:04:13,825 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:04:13,837 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d46631a5c2c05277bcb5a3 HTTP/1.1" 200 None 2025-09-26 19:04:13,838 - lava_callback - DEBUG - Processing node 68d46631a5c2c05277bcb5a3: baseline-arm64 2025-09-26 19:04:14,137 - lava_callback - ERROR - Error processing callback for node 68d46631a5c2c05277bcb5a3: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:57854 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57866 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34410 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57834 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57836 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57850 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46850 - "GET / HTTP/1.0" 200 OK 2025-09-26 19:04:51,877 - lava_callback - INFO - Received callback for node 68d4c3bea5c2c05277bcfe2b INFO: 172.21.0.1:56148 - "POST /node/68d4c3bea5c2c05277bcfe2b HTTP/1.0" 202 Accepted 2025-09-26 19:04:51,883 - lava_callback - INFO - Starting async job submit for node 68d4c3bea5c2c05277bcfe2b 2025-09-26 19:04:51,903 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:04:51,937 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4c3bea5c2c05277bcfe2b HTTP/1.1" 200 None 2025-09-26 19:04:51,937 - lava_callback - DEBUG - Processing node 68d4c3bea5c2c05277bcfe2b: baseline-arm 2025-09-26 19:04:52,181 - lava_callback - ERROR - Error processing callback for node 68d4c3bea5c2c05277bcfe2b: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:56158 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42092 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48050 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48066 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44374 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35110 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35122 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35138 - "GET / HTTP/1.0" 200 OK 2025-09-26 19:05:36,941 - lava_callback - INFO - Received callback for node 68d46636a5c2c05277bcb5a6 INFO: 172.21.0.1:35144 - "POST /node/68d46636a5c2c05277bcb5a6 HTTP/1.0" 202 Accepted 2025-09-26 19:05:36,947 - lava_callback - INFO - Starting async job submit for node 68d46636a5c2c05277bcb5a6 2025-09-26 19:05:36,968 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:05:36,983 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d46636a5c2c05277bcb5a6 HTTP/1.1" 200 None 2025-09-26 19:05:36,984 - lava_callback - DEBUG - Processing node 68d46636a5c2c05277bcb5a6: baseline-arm64 2025-09-26 19:05:37,283 - lava_callback - ERROR - Error processing callback for node 68d46636a5c2c05277bcb5a6: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:51630 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44912 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48420 - "GET / HTTP/1.0" 200 OK 2025-09-26 19:06:15,152 - lava_callback - INFO - Received callback for node 68d44977a5c2c05277bc9fd2 INFO: 172.21.0.1:46644 - "POST /node/68d44977a5c2c05277bc9fd2 HTTP/1.0" 202 Accepted 2025-09-26 19:06:15,160 - lava_callback - INFO - Starting async job submit for node 68d44977a5c2c05277bc9fd2 INFO: 172.21.0.1:46654 - "GET /api/metrics HTTP/1.0" 200 OK 2025-09-26 19:06:15,188 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:06:15,200 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d44977a5c2c05277bc9fd2 HTTP/1.1" 200 None 2025-09-26 19:06:15,201 - lava_callback - DEBUG - Processing node 68d44977a5c2c05277bc9fd2: ltp-syscalls-ipc 2025-09-26 19:06:16,078 - lava_callback - ERROR - Error processing callback for node 68d44977a5c2c05277bc9fd2: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:46666 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41128 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41622 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41628 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41640 - "GET / HTTP/1.0" 200 OK 2025-09-26 19:06:45,427 - lava_callback - INFO - Received callback for node 68d4be87a5c2c05277bcfb4d INFO: 172.21.0.1:45744 - "POST /node/68d4be87a5c2c05277bcfb4d HTTP/1.0" 202 Accepted 2025-09-26 19:06:45,433 - lava_callback - INFO - Starting async job submit for node 68d4be87a5c2c05277bcfb4d 2025-09-26 19:06:45,450 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:06:45,481 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4be87a5c2c05277bcfb4d HTTP/1.1" 200 None 2025-09-26 19:06:45,481 - lava_callback - DEBUG - Processing node 68d4be87a5c2c05277bcfb4d: baseline-arm64-broonie 2025-09-26 19:06:45,802 - lava_callback - ERROR - Error processing callback for node 68d4be87a5c2c05277bcfb4d: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:45760 - "GET / HTTP/1.0" 200 OK 2025-09-26 19:06:53,570 - lava_callback - INFO - Received callback for node 68d4c450a5c2c05277bcff2b INFO: 172.21.0.1:53340 - "POST /node/68d4c450a5c2c05277bcff2b HTTP/1.0" 202 Accepted 2025-09-26 19:06:53,576 - lava_callback - INFO - Starting async job submit for node 68d4c450a5c2c05277bcff2b 2025-09-26 19:06:53,597 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:06:53,621 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4c450a5c2c05277bcff2b HTTP/1.1" 200 None 2025-09-26 19:06:53,622 - lava_callback - DEBUG - Processing node 68d4c450a5c2c05277bcff2b: baseline-arm 2025-09-26 19:06:53,844 - lava_callback - ERROR - Error processing callback for node 68d4c450a5c2c05277bcff2b: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None 2025-09-26 19:06:54,267 - lava_callback - INFO - Received callback for node 68d46638a5c2c05277bcb5a9 INFO: 172.21.0.1:53354 - "POST /node/68d46638a5c2c05277bcb5a9 HTTP/1.0" 202 Accepted 2025-09-26 19:06:54,274 - lava_callback - INFO - Starting async job submit for node 68d46638a5c2c05277bcb5a9 2025-09-26 19:06:54,294 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:06:54,307 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d46638a5c2c05277bcb5a9 HTTP/1.1" 200 None 2025-09-26 19:06:54,307 - lava_callback - DEBUG - Processing node 68d46638a5c2c05277bcb5a9: baseline-arm64 2025-09-26 19:06:54,605 - lava_callback - ERROR - Error processing callback for node 68d46638a5c2c05277bcb5a9: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None 2025-09-26 19:06:55,037 - lava_callback - INFO - Received callback for node 68d6df773c94e462c68821a8 INFO: 172.21.0.1:53370 - "POST /node/68d6df773c94e462c68821a8 HTTP/1.0" 202 Accepted 2025-09-26 19:06:55,043 - lava_callback - INFO - Starting async job submit for node 68d6df773c94e462c68821a8 2025-09-26 19:06:55,073 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:06:55,107 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6df773c94e462c68821a8 HTTP/1.1" 200 None 2025-09-26 19:06:55,108 - lava_callback - DEBUG - Processing node 68d6df773c94e462c68821a8: baseline-x86-kcidebug-intel 2025-09-26 19:06:55,372 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 19:06:55,532 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 19:06:55,533 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-kcidebug-intel-68d6df773c94e462c68821a8/log.txt.gz 2025-09-26 19:06:55,539 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 19:06:55,668 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 19:06:55,669 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-kcidebug-intel-68d6df773c94e462c68821a8/lava_callback.json.gz DEBUG: Setup failed for 68d6df773c94e462c68821a8. Transitting job node result: incomplete -> fail 2025-09-26 19:06:55,728 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:06:55,740 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6df773c94e462c68821a8 HTTP/1.1" 200 None 2025-09-26 19:06:55,741 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:06:55,752 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6b9f52f83206171a92b71 HTTP/1.1" 200 None 2025-09-26 19:06:55,753 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:06:55,777 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6df773c94e462c68821a8 HTTP/1.1" 200 None 2025-09-26 19:06:55,778 - lava_callback - INFO - Completed processing callback for node 68d6df773c94e462c68821a8 INFO: 172.21.0.1:53372 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35124 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44516 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44518 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38796 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47772 - "GET / HTTP/1.0" 200 OK 2025-09-26 19:07:32,892 - lava_callback - INFO - Received callback for node 68d6ca9c3c94e462c6881ca7 INFO: 172.21.0.1:47788 - "POST /node/68d6ca9c3c94e462c6881ca7 HTTP/1.0" 202 Accepted 2025-09-26 19:07:32,898 - lava_callback - INFO - Starting async job submit for node 68d6ca9c3c94e462c6881ca7 2025-09-26 19:07:32,919 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:07:32,931 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6ca9c3c94e462c6881ca7 HTTP/1.1" 200 None 2025-09-26 19:07:32,932 - lava_callback - DEBUG - Processing node 68d6ca9c3c94e462c6881ca7: baseline-arm64 INFO: 172.21.0.1:47794 - "GET / HTTP/1.0" 200 OK 2025-09-26 19:07:33,340 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 19:07:33,467 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 19:07:33,468 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6ca9c3c94e462c6881ca7/log.txt.gz 2025-09-26 19:07:33,477 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 19:07:33,711 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 19:07:33,712 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6ca9c3c94e462c6881ca7/lava_callback.json.gz 2025-09-26 19:07:33,733 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:07:33,745 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6ca9c3c94e462c6881ca7 HTTP/1.1" 200 None 2025-09-26 19:07:33,746 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:07:33,757 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d648ed5d233c363a762351 HTTP/1.1" 200 None 2025-09-26 19:07:33,759 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:07:33,790 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6ca9c3c94e462c6881ca7 HTTP/1.1" 200 None 2025-09-26 19:07:33,791 - lava_callback - INFO - Completed processing callback for node 68d6ca9c3c94e462c6881ca7 INFO: 172.21.0.1:47810 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49924 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60788 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45738 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53388 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53404 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48964 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45402 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45404 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45418 - "GET / HTTP/1.0" 200 OK 2025-09-26 19:08:44,472 - lava_callback - INFO - Received callback for node 68d6c78d3c94e462c68818b6 INFO: 172.21.0.1:33616 - "POST /node/68d6c78d3c94e462c68818b6 HTTP/1.0" 202 Accepted 2025-09-26 19:08:44,479 - lava_callback - INFO - Starting async job submit for node 68d6c78d3c94e462c68818b6 2025-09-26 19:08:44,515 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:08:44,527 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6c78d3c94e462c68818b6 HTTP/1.1" 200 None 2025-09-26 19:08:44,528 - lava_callback - DEBUG - Processing node 68d6c78d3c94e462c68818b6: baseline-arm64-broonie 2025-09-26 19:08:45,125 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 19:08:45,262 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 19:08:45,263 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-68d6c78d3c94e462c68818b6/log.txt.gz 2025-09-26 19:08:45,274 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 19:08:45,391 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 19:08:45,391 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-68d6c78d3c94e462c68818b6/lava_callback.json.gz 2025-09-26 19:08:45,426 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:08:45,438 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6c78d3c94e462c68818b6 HTTP/1.1" 200 None 2025-09-26 19:08:45,439 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:08:45,457 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6b9dd2f83206171a92b48 HTTP/1.1" 200 None 2025-09-26 19:08:45,458 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:08:45,482 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6c78d3c94e462c68818b6 HTTP/1.1" 200 None 2025-09-26 19:08:45,482 - lava_callback - INFO - Completed processing callback for node 68d6c78d3c94e462c68818b6 INFO: 172.21.0.1:33630 - "GET / HTTP/1.0" 200 OK 2025-09-26 19:08:51,247 - lava_callback - INFO - Received callback for node 68d6ca9d3c94e462c6881caa INFO: 172.21.0.1:47832 - "POST /node/68d6ca9d3c94e462c6881caa HTTP/1.0" 202 Accepted 2025-09-26 19:08:51,253 - lava_callback - INFO - Starting async job submit for node 68d6ca9d3c94e462c6881caa 2025-09-26 19:08:51,274 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:08:51,289 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6ca9d3c94e462c6881caa HTTP/1.1" 200 None 2025-09-26 19:08:51,290 - lava_callback - DEBUG - Processing node 68d6ca9d3c94e462c6881caa: baseline-arm64 2025-09-26 19:08:51,720 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 19:08:51,834 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 19:08:51,835 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6ca9d3c94e462c6881caa/log.txt.gz 2025-09-26 19:08:51,844 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 19:08:51,949 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 19:08:51,950 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6ca9d3c94e462c6881caa/lava_callback.json.gz 2025-09-26 19:08:51,971 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:08:51,983 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6ca9d3c94e462c6881caa HTTP/1.1" 200 None 2025-09-26 19:08:51,984 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:08:51,995 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d648ed5d233c363a762351 HTTP/1.1" 200 None 2025-09-26 19:08:51,995 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:08:52,020 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6ca9d3c94e462c6881caa HTTP/1.1" 200 None 2025-09-26 19:08:52,021 - lava_callback - INFO - Completed processing callback for node 68d6ca9d3c94e462c6881caa INFO: 172.21.0.1:47842 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33196 - "GET / HTTP/1.0" 200 OK 2025-09-26 19:09:09,825 - lava_callback - INFO - Received callback for node 68d6c5c23c94e462c688183c INFO: 172.21.0.1:59606 - "POST /node/68d6c5c23c94e462c688183c HTTP/1.0" 202 Accepted 2025-09-26 19:09:09,832 - lava_callback - INFO - Starting async job submit for node 68d6c5c23c94e462c688183c 2025-09-26 19:09:09,883 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:09:09,895 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6c5c23c94e462c688183c HTTP/1.1" 200 None 2025-09-26 19:09:09,895 - lava_callback - DEBUG - Processing node 68d6c5c23c94e462c688183c: ltp-smoketest 2025-09-26 19:09:10,262 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 19:09:10,393 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 19:09:10,394 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoketest-68d6c5c23c94e462c688183c/log.txt.gz 2025-09-26 19:09:10,403 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 19:09:10,541 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 19:09:10,542 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoketest-68d6c5c23c94e462c688183c/lava_callback.json.gz 2025-09-26 19:09:10,650 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:09:10,684 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6c5c23c94e462c688183c HTTP/1.1" 200 None 2025-09-26 19:09:10,685 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:09:10,699 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6bd813c94e462c68814e8 HTTP/1.1" 200 None 2025-09-26 19:09:10,700 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:09:10,759 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6c5c23c94e462c688183c HTTP/1.1" 200 None 2025-09-26 19:09:10,760 - lava_callback - INFO - Completed processing callback for node 68d6c5c23c94e462c688183c 2025-09-26 19:09:11,970 - lava_callback - INFO - Received callback for node 68d4c450a5c2c05277bcff2e INFO: 172.21.0.1:59608 - "POST /node/68d4c450a5c2c05277bcff2e HTTP/1.0" 202 Accepted 2025-09-26 19:09:11,976 - lava_callback - INFO - Starting async job submit for node 68d4c450a5c2c05277bcff2e 2025-09-26 19:09:11,997 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:09:12,009 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4c450a5c2c05277bcff2e HTTP/1.1" 200 None 2025-09-26 19:09:12,009 - lava_callback - DEBUG - Processing node 68d4c450a5c2c05277bcff2e: baseline-arm 2025-09-26 19:09:12,262 - lava_callback - ERROR - Error processing callback for node 68d4c450a5c2c05277bcff2e: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:59612 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59614 - "GET / HTTP/1.0" 200 OK 2025-09-26 19:09:24,689 - lava_callback - INFO - Received callback for node 68d6d48b3c94e462c6881e2d INFO: 172.21.0.1:47518 - "POST /node/68d6d48b3c94e462c6881e2d HTTP/1.0" 202 Accepted 2025-09-26 19:09:24,695 - lava_callback - INFO - Starting async job submit for node 68d6d48b3c94e462c6881e2d 2025-09-26 19:09:24,717 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:09:24,732 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6d48b3c94e462c6881e2d HTTP/1.1" 200 None 2025-09-26 19:09:24,733 - lava_callback - DEBUG - Processing node 68d6d48b3c94e462c6881e2d: baseline-arm64-broonie 2025-09-26 19:09:24,957 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 19:09:25,091 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 19:09:25,092 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-68d6d48b3c94e462c6881e2d/log.txt.gz 2025-09-26 19:09:25,098 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 19:09:25,216 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 19:09:25,217 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-68d6d48b3c94e462c6881e2d/lava_callback.json.gz 2025-09-26 19:09:25,239 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:09:25,253 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6d48b3c94e462c6881e2d HTTP/1.1" 200 None 2025-09-26 19:09:25,254 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:09:25,267 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6bd8e3c94e462c68814f1 HTTP/1.1" 200 None 2025-09-26 19:09:25,269 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:09:25,302 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6d48b3c94e462c6881e2d HTTP/1.1" 200 None 2025-09-26 19:09:25,303 - lava_callback - INFO - Completed processing callback for node 68d6d48b3c94e462c6881e2d INFO: 172.21.0.1:47520 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38566 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38580 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38594 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60280 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38152 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50194 - "GET / HTTP/1.0" 200 OK 2025-09-26 19:10:13,028 - lava_callback - INFO - Received callback for node 68d6cac73c94e462c6881cb9 INFO: 172.21.0.1:39938 - "POST /node/68d6cac73c94e462c6881cb9 HTTP/1.0" 202 Accepted 2025-09-26 19:10:13,039 - lava_callback - INFO - Starting async job submit for node 68d6cac73c94e462c6881cb9 2025-09-26 19:10:13,063 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:10:13,078 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6cac73c94e462c6881cb9 HTTP/1.1" 200 None 2025-09-26 19:10:13,078 - lava_callback - DEBUG - Processing node 68d6cac73c94e462c6881cb9: baseline-arm64 2025-09-26 19:10:13,544 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 19:10:13,676 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 19:10:13,676 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6cac73c94e462c6881cb9/log.txt.gz 2025-09-26 19:10:13,687 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 19:10:13,829 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 19:10:13,829 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6cac73c94e462c6881cb9/lava_callback.json.gz 2025-09-26 19:10:13,853 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:10:13,865 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6cac73c94e462c6881cb9 HTTP/1.1" 200 None 2025-09-26 19:10:13,866 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:10:13,891 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d648ed5d233c363a762351 HTTP/1.1" 200 None 2025-09-26 19:10:13,892 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:10:13,921 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6cac73c94e462c6881cb9 HTTP/1.1" 200 None 2025-09-26 19:10:13,922 - lava_callback - INFO - Completed processing callback for node 68d6cac73c94e462c6881cb9 INFO: 172.21.0.1:39950 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39954 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50188 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45076 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45086 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45098 - "GET / HTTP/1.0" 200 OK 2025-09-26 19:10:44,425 - lava_callback - INFO - Received callback for node 68d6e0303c94e462c68821ad INFO: 172.21.0.1:45334 - "POST /node/68d6e0303c94e462c68821ad HTTP/1.0" 202 Accepted 2025-09-26 19:10:44,431 - lava_callback - INFO - Starting async job submit for node 68d6e0303c94e462c68821ad 2025-09-26 19:10:44,452 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:10:44,464 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6e0303c94e462c68821ad HTTP/1.1" 200 None 2025-09-26 19:10:44,464 - lava_callback - DEBUG - Processing node 68d6e0303c94e462c68821ad: baseline-arm64-broonie 2025-09-26 19:10:44,708 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 19:10:44,815 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 19:10:44,816 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-68d6e0303c94e462c68821ad/log.txt.gz 2025-09-26 19:10:44,822 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 19:10:45,002 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 19:10:45,002 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-68d6e0303c94e462c68821ad/lava_callback.json.gz 2025-09-26 19:10:45,024 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:10:45,035 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6e0303c94e462c68821ad HTTP/1.1" 200 None 2025-09-26 19:10:45,036 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:10:45,047 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6b9dd2f83206171a92b48 HTTP/1.1" 200 None 2025-09-26 19:10:45,048 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:10:45,072 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6e0303c94e462c68821ad HTTP/1.1" 200 None 2025-09-26 19:10:45,073 - lava_callback - INFO - Completed processing callback for node 68d6e0303c94e462c68821ad INFO: 172.21.0.1:45340 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38322 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54896 - "GET / HTTP/1.0" 200 OK 2025-09-26 19:11:12,817 - lava_callback - INFO - Received callback for node 68d4c452a5c2c05277bcff31 INFO: 172.21.0.1:33616 - "POST /node/68d4c452a5c2c05277bcff31 HTTP/1.0" 202 Accepted 2025-09-26 19:11:12,823 - lava_callback - INFO - Starting async job submit for node 68d4c452a5c2c05277bcff31 2025-09-26 19:11:12,844 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:11:12,858 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4c452a5c2c05277bcff31 HTTP/1.1" 200 None 2025-09-26 19:11:12,859 - lava_callback - DEBUG - Processing node 68d4c452a5c2c05277bcff31: baseline-arm 2025-09-26 19:11:13,093 - lava_callback - ERROR - Error processing callback for node 68d4c452a5c2c05277bcff31: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:33622 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33630 - "GET / HTTP/1.0" 200 OK 2025-09-26 19:11:24,580 - lava_callback - INFO - Received callback for node 68d6d48a3c94e462c6881e2a INFO: 172.21.0.1:41618 - "POST /node/68d6d48a3c94e462c6881e2a HTTP/1.0" 202 Accepted 2025-09-26 19:11:24,586 - lava_callback - INFO - Starting async job submit for node 68d6d48a3c94e462c6881e2a 2025-09-26 19:11:24,607 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:11:24,646 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6d48a3c94e462c6881e2a HTTP/1.1" 200 None 2025-09-26 19:11:24,647 - lava_callback - DEBUG - Processing node 68d6d48a3c94e462c6881e2a: baseline-arm64-broonie 2025-09-26 19:11:24,913 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 19:11:25,051 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 19:11:25,052 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-68d6d48a3c94e462c6881e2a/log.txt.gz 2025-09-26 19:11:25,058 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 19:11:25,197 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 19:11:25,198 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-68d6d48a3c94e462c6881e2a/lava_callback.json.gz 2025-09-26 19:11:25,220 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:11:25,232 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6d48a3c94e462c6881e2a HTTP/1.1" 200 None 2025-09-26 19:11:25,233 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:11:25,263 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6bd8e3c94e462c68814f1 HTTP/1.1" 200 None 2025-09-26 19:11:25,264 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:11:25,307 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6d48a3c94e462c6881e2a HTTP/1.1" 200 None 2025-09-26 19:11:25,308 - lava_callback - INFO - Completed processing callback for node 68d6d48a3c94e462c6881e2a INFO: 172.21.0.1:41626 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34464 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34474 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34476 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42254 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60472 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55340 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60900 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60914 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36758 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47760 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47770 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47778 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41832 - "GET / HTTP/1.0" 200 OK 2025-09-26 19:12:55,099 - lava_callback - INFO - Received callback for node 68d6cacc3c94e462c6881cc6 INFO: 172.21.0.1:37772 - "POST /node/68d6cacc3c94e462c6881cc6 HTTP/1.0" 202 Accepted 2025-09-26 19:12:55,106 - lava_callback - INFO - Starting async job submit for node 68d6cacc3c94e462c6881cc6 2025-09-26 19:12:55,127 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:12:55,140 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6cacc3c94e462c6881cc6 HTTP/1.1" 200 None 2025-09-26 19:12:55,140 - lava_callback - DEBUG - Processing node 68d6cacc3c94e462c6881cc6: baseline-arm64 2025-09-26 19:12:55,572 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 19:12:55,712 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 19:12:55,713 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6cacc3c94e462c6881cc6/log.txt.gz 2025-09-26 19:12:55,722 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 19:12:55,852 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 19:12:55,853 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6cacc3c94e462c6881cc6/lava_callback.json.gz 2025-09-26 19:12:55,874 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:12:55,905 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6cacc3c94e462c6881cc6 HTTP/1.1" 200 None 2025-09-26 19:12:55,906 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:12:55,941 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d648ed5d233c363a762351 HTTP/1.1" 200 None 2025-09-26 19:12:55,943 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:12:55,971 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6cacc3c94e462c6881cc6 HTTP/1.1" 200 None 2025-09-26 19:12:55,972 - lava_callback - INFO - Completed processing callback for node 68d6cacc3c94e462c6881cc6 INFO: 172.21.0.1:37786 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50652 - "GET / HTTP/1.0" 200 OK 2025-09-26 19:13:12,337 - lava_callback - INFO - Received callback for node 68d4c452a5c2c05277bcff34 INFO: 172.21.0.1:54766 - "POST /node/68d4c452a5c2c05277bcff34 HTTP/1.0" 202 Accepted 2025-09-26 19:13:12,344 - lava_callback - INFO - Starting async job submit for node 68d4c452a5c2c05277bcff34 2025-09-26 19:13:12,365 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:13:12,377 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4c452a5c2c05277bcff34 HTTP/1.1" 200 None 2025-09-26 19:13:12,378 - lava_callback - DEBUG - Processing node 68d4c452a5c2c05277bcff34: baseline-arm 2025-09-26 19:13:12,613 - lava_callback - ERROR - Error processing callback for node 68d4c452a5c2c05277bcff34: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:54782 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54786 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37912 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48334 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48338 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48354 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52128 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33366 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43008 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34220 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34222 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51414 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53938 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53952 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53966 - "GET / HTTP/1.0" 200 OK 2025-09-26 19:14:46,470 - lava_callback - INFO - Received callback for node 68d6c78a3c94e462c68818a4 INFO: 172.21.0.1:42662 - "POST /node/68d6c78a3c94e462c68818a4 HTTP/1.0" 202 Accepted 2025-09-26 19:14:46,475 - lava_callback - INFO - Starting async job submit for node 68d6c78a3c94e462c68818a4 2025-09-26 19:14:46,497 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.21.0.1:42678 - "GET / HTTP/1.0" 200 OK 2025-09-26 19:14:46,516 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6c78a3c94e462c68818a4 HTTP/1.1" 200 None 2025-09-26 19:14:46,517 - lava_callback - DEBUG - Processing node 68d6c78a3c94e462c68818a4: baseline-arm64-broonie 2025-09-26 19:14:46,739 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 19:14:46,872 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 19:14:46,873 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-68d6c78a3c94e462c68818a4/log.txt.gz 2025-09-26 19:14:46,879 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 19:14:47,033 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 19:14:47,034 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-68d6c78a3c94e462c68818a4/lava_callback.json.gz 2025-09-26 19:14:47,056 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:14:47,069 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6c78a3c94e462c68818a4 HTTP/1.1" 200 None 2025-09-26 19:14:47,070 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:14:47,085 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6b9dd2f83206171a92b48 HTTP/1.1" 200 None 2025-09-26 19:14:47,086 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:14:47,115 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6c78a3c94e462c68818a4 HTTP/1.1" 200 None 2025-09-26 19:14:47,116 - lava_callback - INFO - Completed processing callback for node 68d6c78a3c94e462c68818a4 INFO: 172.21.0.1:38174 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43334 - "GET / HTTP/1.0" 200 OK 2025-09-26 19:15:12,241 - lava_callback - INFO - Received callback for node 68d4c462a5c2c05277bcff5b INFO: 172.21.0.1:43498 - "POST /node/68d4c462a5c2c05277bcff5b HTTP/1.0" 202 Accepted 2025-09-26 19:15:12,246 - lava_callback - INFO - Starting async job submit for node 68d4c462a5c2c05277bcff5b 2025-09-26 19:15:12,267 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:15:12,281 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4c462a5c2c05277bcff5b HTTP/1.1" 200 None 2025-09-26 19:15:12,282 - lava_callback - DEBUG - Processing node 68d4c462a5c2c05277bcff5b: baseline-arm 2025-09-26 19:15:12,526 - lava_callback - ERROR - Error processing callback for node 68d4c462a5c2c05277bcff5b: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:43504 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43508 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48054 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43684 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43688 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43696 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41364 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35800 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33670 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45166 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45168 - "GET / HTTP/1.0" 200 OK 2025-09-26 19:16:17,374 - lava_callback - INFO - Received callback for node 68d6d47e3c94e462c6881dd6 INFO: 172.21.0.1:45184 - "POST /node/68d6d47e3c94e462c6881dd6 HTTP/1.0" 202 Accepted 2025-09-26 19:16:17,380 - lava_callback - INFO - Starting async job submit for node 68d6d47e3c94e462c6881dd6 2025-09-26 19:16:17,400 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:16:17,412 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6d47e3c94e462c6881dd6 HTTP/1.1" 200 None 2025-09-26 19:16:17,413 - lava_callback - DEBUG - Processing node 68d6d47e3c94e462c6881dd6: baseline-arm64 2025-09-26 19:16:17,661 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 19:16:17,804 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 19:16:17,804 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6d47e3c94e462c6881dd6/log.txt.gz 2025-09-26 19:16:17,811 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 19:16:17,937 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 19:16:17,937 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6d47e3c94e462c6881dd6/lava_callback.json.gz 2025-09-26 19:16:17,959 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:16:17,982 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6d47e3c94e462c6881dd6 HTTP/1.1" 200 None 2025-09-26 19:16:17,983 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:16:17,994 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6bd8e3c94e462c68814f1 HTTP/1.1" 200 None 2025-09-26 19:16:17,995 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:16:18,049 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6d47e3c94e462c6881dd6 HTTP/1.1" 200 None 2025-09-26 19:16:18,050 - lava_callback - INFO - Completed processing callback for node 68d6d47e3c94e462c6881dd6 INFO: 172.21.0.1:50338 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50060 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50070 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50076 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45420 - "GET / HTTP/1.0" 200 OK 2025-09-26 19:16:47,409 - lava_callback - INFO - Received callback for node 68d6cac73c94e462c6881cbc INFO: 172.21.0.1:45430 - "POST /node/68d6cac73c94e462c6881cbc HTTP/1.0" 202 Accepted 2025-09-26 19:16:47,416 - lava_callback - INFO - Starting async job submit for node 68d6cac73c94e462c6881cbc 2025-09-26 19:16:47,445 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:16:47,459 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6cac73c94e462c6881cbc HTTP/1.1" 200 None 2025-09-26 19:16:47,459 - lava_callback - DEBUG - Processing node 68d6cac73c94e462c6881cbc: baseline-arm64 2025-09-26 19:16:48,245 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 19:16:48,389 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 19:16:48,389 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6cac73c94e462c6881cbc/log.txt.gz 2025-09-26 19:16:48,406 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 19:16:48,538 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 19:16:48,539 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6cac73c94e462c6881cbc/lava_callback.json.gz DEBUG: 68d6cac73c94e462c6881cbc Transitting job node result: pass -> incomplete 2025-09-26 19:16:48,575 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:16:48,588 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6cac73c94e462c6881cbc HTTP/1.1" 200 None 2025-09-26 19:16:48,589 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:16:48,606 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d648ed5d233c363a762351 HTTP/1.1" 200 None 2025-09-26 19:16:48,607 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:16:48,635 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6cac73c94e462c6881cbc HTTP/1.1" 200 None 2025-09-26 19:16:48,636 - lava_callback - INFO - Completed processing callback for node 68d6cac73c94e462c6881cbc INFO: 172.21.0.1:50038 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46166 - "GET / HTTP/1.0" 200 OK 2025-09-26 19:17:14,292 - lava_callback - INFO - Received callback for node 68d4c462a5c2c05277bcff5e INFO: 172.21.0.1:51216 - "POST /node/68d4c462a5c2c05277bcff5e HTTP/1.0" 202 Accepted 2025-09-26 19:17:14,298 - lava_callback - INFO - Starting async job submit for node 68d4c462a5c2c05277bcff5e 2025-09-26 19:17:14,317 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:17:14,349 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4c462a5c2c05277bcff5e HTTP/1.1" 200 None 2025-09-26 19:17:14,349 - lava_callback - DEBUG - Processing node 68d4c462a5c2c05277bcff5e: baseline-arm 2025-09-26 19:17:14,576 - lava_callback - ERROR - Error processing callback for node 68d4c462a5c2c05277bcff5e: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:51224 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51228 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33574 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43198 - "GET / HTTP/1.0" 200 OK 2025-09-26 19:17:32,072 - lava_callback - INFO - Received callback for node 68d6d4813c94e462c6881df1 INFO: 172.21.0.1:43212 - "POST /node/68d6d4813c94e462c6881df1 HTTP/1.0" 202 Accepted 2025-09-26 19:17:32,078 - lava_callback - INFO - Starting async job submit for node 68d6d4813c94e462c6881df1 2025-09-26 19:17:32,099 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:17:32,112 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6d4813c94e462c6881df1 HTTP/1.1" 200 None 2025-09-26 19:17:32,112 - lava_callback - DEBUG - Processing node 68d6d4813c94e462c6881df1: baseline-arm64 2025-09-26 19:17:32,374 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 19:17:32,512 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 19:17:32,513 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6d4813c94e462c6881df1/log.txt.gz 2025-09-26 19:17:32,519 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 19:17:32,669 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 19:17:32,669 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6d4813c94e462c6881df1/lava_callback.json.gz 2025-09-26 19:17:32,690 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:17:32,705 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6d4813c94e462c6881df1 HTTP/1.1" 200 None 2025-09-26 19:17:32,706 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:17:32,717 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6bd8e3c94e462c68814f1 HTTP/1.1" 200 None 2025-09-26 19:17:32,718 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:17:32,764 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6d4813c94e462c6881df1 HTTP/1.1" 200 None 2025-09-26 19:17:32,765 - lava_callback - INFO - Completed processing callback for node 68d6d4813c94e462c6881df1 INFO: 172.21.0.1:43220 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43226 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55138 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43442 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58804 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60940 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60956 - "GET / HTTP/1.0" 200 OK 2025-09-26 19:18:17,740 - lava_callback - INFO - Received callback for node 68d6cacb3c94e462c6881cc3 INFO: 172.21.0.1:60970 - "POST /node/68d6cacb3c94e462c6881cc3 HTTP/1.0" 202 Accepted 2025-09-26 19:18:17,748 - lava_callback - INFO - Starting async job submit for node 68d6cacb3c94e462c6881cc3 2025-09-26 19:18:17,777 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:18:17,790 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6cacb3c94e462c6881cc3 HTTP/1.1" 200 None 2025-09-26 19:18:17,790 - lava_callback - DEBUG - Processing node 68d6cacb3c94e462c6881cc3: baseline-arm64 2025-09-26 19:18:18,600 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 19:18:18,741 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 19:18:18,742 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6cacb3c94e462c6881cc3/log.txt.gz 2025-09-26 19:18:18,758 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 19:18:18,896 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 19:18:18,897 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6cacb3c94e462c6881cc3/lava_callback.json.gz DEBUG: 68d6cacb3c94e462c6881cc3 Transitting job node result: pass -> incomplete 2025-09-26 19:18:18,927 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:18:18,942 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6cacb3c94e462c6881cc3 HTTP/1.1" 200 None 2025-09-26 19:18:18,944 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:18:18,955 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d648ed5d233c363a762351 HTTP/1.1" 200 None 2025-09-26 19:18:18,956 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:18:18,989 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6cacb3c94e462c6881cc3 HTTP/1.1" 200 None 2025-09-26 19:18:18,990 - lava_callback - INFO - Completed processing callback for node 68d6cacb3c94e462c6881cc3 INFO: 172.21.0.1:36552 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35910 - "GET / HTTP/1.0" 200 OK 2025-09-26 19:18:30,790 - lava_callback - INFO - Received callback for node 68d6d48b3c94e462c6881e30 INFO: 172.21.0.1:35920 - "POST /node/68d6d48b3c94e462c6881e30 HTTP/1.0" 202 Accepted 2025-09-26 19:18:30,796 - lava_callback - INFO - Starting async job submit for node 68d6d48b3c94e462c6881e30 2025-09-26 19:18:30,818 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:18:30,830 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6d48b3c94e462c6881e30 HTTP/1.1" 200 None 2025-09-26 19:18:30,831 - lava_callback - DEBUG - Processing node 68d6d48b3c94e462c6881e30: baseline-arm64-broonie 2025-09-26 19:18:31,203 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 19:18:31,310 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 19:18:31,311 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-68d6d48b3c94e462c6881e30/log.txt.gz 2025-09-26 19:18:31,319 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 19:18:31,450 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 19:18:31,451 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-68d6d48b3c94e462c6881e30/lava_callback.json.gz 2025-09-26 19:18:31,473 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:18:31,485 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6d48b3c94e462c6881e30 HTTP/1.1" 200 None 2025-09-26 19:18:31,486 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:18:31,499 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6bd8e3c94e462c68814f1 HTTP/1.1" 200 None 2025-09-26 19:18:31,500 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:18:31,556 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6d48b3c94e462c6881e30 HTTP/1.1" 200 None 2025-09-26 19:18:31,557 - lava_callback - INFO - Completed processing callback for node 68d6d48b3c94e462c6881e30 INFO: 172.21.0.1:35924 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35932 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46842 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52448 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53248 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35352 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35364 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43968 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45608 - "GET / HTTP/1.0" 200 OK 2025-09-26 19:19:32,531 - lava_callback - INFO - Received callback for node 68d4c464a5c2c05277bcff61 INFO: 172.21.0.1:45620 - "POST /node/68d4c464a5c2c05277bcff61 HTTP/1.0" 202 Accepted 2025-09-26 19:19:32,537 - lava_callback - INFO - Starting async job submit for node 68d4c464a5c2c05277bcff61 2025-09-26 19:19:32,558 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:19:32,588 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4c464a5c2c05277bcff61 HTTP/1.1" 200 None 2025-09-26 19:19:32,589 - lava_callback - DEBUG - Processing node 68d4c464a5c2c05277bcff61: baseline-arm 2025-09-26 19:19:32,844 - lava_callback - ERROR - Error processing callback for node 68d4c464a5c2c05277bcff61: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:45636 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45648 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42144 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51710 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45110 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42562 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42576 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53798 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49128 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49130 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49140 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59002 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36998 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39792 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33568 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33584 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48340 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36820 - "GET / HTTP/1.0" 200 OK 2025-09-26 19:21:32,916 - lava_callback - INFO - Received callback for node 68d4c465a5c2c05277bcff64 INFO: 172.21.0.1:36832 - "POST /node/68d4c465a5c2c05277bcff64 HTTP/1.0" 202 Accepted 2025-09-26 19:21:32,922 - lava_callback - INFO - Starting async job submit for node 68d4c465a5c2c05277bcff64 2025-09-26 19:21:32,942 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:21:32,955 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4c465a5c2c05277bcff64 HTTP/1.1" 200 None 2025-09-26 19:21:32,955 - lava_callback - DEBUG - Processing node 68d4c465a5c2c05277bcff64: baseline-arm 2025-09-26 19:21:33,179 - lava_callback - ERROR - Error processing callback for node 68d4c465a5c2c05277bcff64: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:36846 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36856 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48612 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35194 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47694 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44664 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44674 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36630 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55792 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55804 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55816 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53928 - "GET / HTTP/1.0" 200 OK 2025-09-26 19:22:55,892 - lava_callback - INFO - Received callback for node 68d44408a5c2c05277bc9c31 INFO: 172.21.0.1:37628 - "POST /node/68d44408a5c2c05277bc9c31 HTTP/1.0" 202 Accepted 2025-09-26 19:22:55,898 - lava_callback - INFO - Starting async job submit for node 68d44408a5c2c05277bc9c31 2025-09-26 19:22:55,928 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:22:55,953 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d44408a5c2c05277bc9c31 HTTP/1.1" 200 None 2025-09-26 19:22:55,953 - lava_callback - DEBUG - Processing node 68d44408a5c2c05277bc9c31: baseline-arm 2025-09-26 19:22:56,216 - lava_callback - INFO - Received callback for node 68d44505a5c2c05277bc9e01 INFO: 172.21.0.1:37630 - "POST /node/68d44505a5c2c05277bc9e01 HTTP/1.0" 202 Accepted 2025-09-26 19:22:56,229 - lava_callback - INFO - Starting async job submit for node 68d44505a5c2c05277bc9e01 2025-09-26 19:22:56,231 - lava_callback - ERROR - Error processing callback for node 68d44408a5c2c05277bc9c31: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None 2025-09-26 19:22:56,231 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:22:56,244 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d44505a5c2c05277bc9e01 HTTP/1.1" 200 None 2025-09-26 19:22:56,245 - lava_callback - DEBUG - Processing node 68d44505a5c2c05277bc9e01: baseline-arm 2025-09-26 19:22:56,245 - lava_callback - ERROR - Error processing callback for node 68d44505a5c2c05277bc9e01: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:37644 - "GET / HTTP/1.0" 200 OK 2025-09-26 19:22:56,921 - lava_callback - INFO - Received callback for node 68d44507a5c2c05277bc9e07 INFO: 172.21.0.1:37656 - "POST /node/68d44507a5c2c05277bc9e07 HTTP/1.0" 202 Accepted 2025-09-26 19:22:56,927 - lava_callback - INFO - Starting async job submit for node 68d44507a5c2c05277bc9e07 2025-09-26 19:22:56,927 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:22:56,946 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d44507a5c2c05277bc9e07 HTTP/1.1" 200 None 2025-09-26 19:22:56,946 - lava_callback - DEBUG - Processing node 68d44507a5c2c05277bc9e07: baseline-arm 2025-09-26 19:22:56,946 - lava_callback - ERROR - Error processing callback for node 68d44507a5c2c05277bc9e07: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 19:22:57,622 - lava_callback - INFO - Received callback for node 68d44509a5c2c05277bc9e0a INFO: 172.21.0.1:37666 - "POST /node/68d44509a5c2c05277bc9e0a HTTP/1.0" 202 Accepted 2025-09-26 19:22:57,627 - lava_callback - INFO - Starting async job submit for node 68d44509a5c2c05277bc9e0a 2025-09-26 19:22:57,628 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:22:57,642 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d44509a5c2c05277bc9e0a HTTP/1.1" 200 None 2025-09-26 19:22:57,642 - lava_callback - DEBUG - Processing node 68d44509a5c2c05277bc9e0a: baseline-arm 2025-09-26 19:22:57,642 - lava_callback - ERROR - Error processing callback for node 68d44509a5c2c05277bc9e0a: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 19:22:58,336 - lava_callback - INFO - Received callback for node 68d44509a5c2c05277bc9e0d INFO: 172.21.0.1:37676 - "POST /node/68d44509a5c2c05277bc9e0d HTTP/1.0" 202 Accepted 2025-09-26 19:22:58,342 - lava_callback - INFO - Starting async job submit for node 68d44509a5c2c05277bc9e0d 2025-09-26 19:22:58,343 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:22:58,358 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d44509a5c2c05277bc9e0d HTTP/1.1" 200 None 2025-09-26 19:22:58,358 - lava_callback - DEBUG - Processing node 68d44509a5c2c05277bc9e0d: baseline-arm 2025-09-26 19:22:58,358 - lava_callback - ERROR - Error processing callback for node 68d44509a5c2c05277bc9e0d: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:37672 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44678 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44680 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39096 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60804 - "GET / HTTP/1.0" 200 OK 2025-09-26 19:23:32,902 - lava_callback - INFO - Received callback for node 68d4cc90a5c2c05277bd0650 INFO: 172.21.0.1:60818 - "POST /node/68d4cc90a5c2c05277bd0650 HTTP/1.0" 202 Accepted 2025-09-26 19:23:32,908 - lava_callback - INFO - Starting async job submit for node 68d4cc90a5c2c05277bd0650 2025-09-26 19:23:32,929 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:23:32,946 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4cc90a5c2c05277bd0650 HTTP/1.1" 200 None 2025-09-26 19:23:32,947 - lava_callback - DEBUG - Processing node 68d4cc90a5c2c05277bd0650: baseline-arm 2025-09-26 19:23:33,198 - lava_callback - ERROR - Error processing callback for node 68d4cc90a5c2c05277bd0650: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:60832 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60838 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58336 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40966 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45494 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57174 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57178 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58760 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50406 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50422 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50424 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59006 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52108 - "GET / HTTP/1.0" 200 OK 2025-09-26 19:25:06,414 - lava_callback - INFO - Received callback for node 68d6e4be3c94e462c6882208 INFO: 172.21.0.1:57562 - "POST /node/68d6e4be3c94e462c6882208 HTTP/1.0" 202 Accepted 2025-09-26 19:25:06,421 - lava_callback - INFO - Starting async job submit for node 68d6e4be3c94e462c6882208 2025-09-26 19:25:06,458 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:25:06,488 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6e4be3c94e462c6882208 HTTP/1.1" 200 None 2025-09-26 19:25:06,489 - lava_callback - DEBUG - Processing node 68d6e4be3c94e462c6882208: baseline-arm64-broonie INFO: 172.21.0.1:57570 - "GET / HTTP/1.0" 200 OK 2025-09-26 19:25:07,069 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 19:25:07,214 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 19:25:07,215 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-68d6e4be3c94e462c6882208/log.txt.gz 2025-09-26 19:25:07,227 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 19:25:07,364 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 19:25:07,364 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-68d6e4be3c94e462c6882208/lava_callback.json.gz 2025-09-26 19:25:07,399 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:25:07,412 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6e4be3c94e462c6882208 HTTP/1.1" 200 None 2025-09-26 19:25:07,413 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:25:07,424 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6b9dd2f83206171a92b48 HTTP/1.1" 200 None 2025-09-26 19:25:07,425 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:25:07,450 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6e4be3c94e462c6882208 HTTP/1.1" 200 None 2025-09-26 19:25:07,451 - lava_callback - INFO - Completed processing callback for node 68d6e4be3c94e462c6882208 INFO: 172.21.0.1:44358 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44370 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45498 - "GET / HTTP/1.0" 200 OK 2025-09-26 19:25:29,381 - lava_callback - INFO - Received callback for node 68d4cc90a5c2c05277bd0653 INFO: 172.21.0.1:58360 - "POST /node/68d4cc90a5c2c05277bd0653 HTTP/1.0" 202 Accepted 2025-09-26 19:25:29,387 - lava_callback - INFO - Starting async job submit for node 68d4cc90a5c2c05277bd0653 2025-09-26 19:25:29,425 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:25:29,445 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4cc90a5c2c05277bd0653 HTTP/1.1" 200 None 2025-09-26 19:25:29,445 - lava_callback - DEBUG - Processing node 68d4cc90a5c2c05277bd0653: baseline-arm 2025-09-26 19:25:29,670 - lava_callback - ERROR - Error processing callback for node 68d4cc90a5c2c05277bd0653: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:58362 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58368 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58380 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45810 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34448 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55472 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50708 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50712 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50388 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52670 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52672 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52676 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52400 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47338 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33096 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44696 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44708 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48346 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43846 - "GET / HTTP/1.0" 200 OK 2025-09-26 19:27:31,909 - lava_callback - INFO - Received callback for node 68d4cc92a5c2c05277bd0656 INFO: 172.21.0.1:43862 - "POST /node/68d4cc92a5c2c05277bd0656 HTTP/1.0" 202 Accepted 2025-09-26 19:27:31,915 - lava_callback - INFO - Starting async job submit for node 68d4cc92a5c2c05277bd0656 2025-09-26 19:27:31,939 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:27:31,951 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4cc92a5c2c05277bd0656 HTTP/1.1" 200 None 2025-09-26 19:27:31,951 - lava_callback - DEBUG - Processing node 68d4cc92a5c2c05277bd0656: baseline-arm 2025-09-26 19:27:32,176 - lava_callback - ERROR - Error processing callback for node 68d4cc92a5c2c05277bd0656: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None 2025-09-26 19:27:36,298 - lava_callback - INFO - Received callback for node 68d6e4503c94e462c68821c8 INFO: 172.21.0.1:43866 - "POST /node/68d6e4503c94e462c68821c8 HTTP/1.0" 202 Accepted 2025-09-26 19:27:36,303 - lava_callback - INFO - Starting async job submit for node 68d6e4503c94e462c68821c8 2025-09-26 19:27:36,333 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:27:36,346 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6e4503c94e462c68821c8 HTTP/1.1" 200 None 2025-09-26 19:27:36,347 - lava_callback - DEBUG - Processing node 68d6e4503c94e462c68821c8: baseline-x86-kcidebug-intel INFO: 172.21.0.1:43882 - "GET / HTTP/1.0" 200 OK 2025-09-26 19:27:36,589 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 19:27:36,739 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 19:27:36,739 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-kcidebug-intel-68d6e4503c94e462c68821c8/log.txt.gz 2025-09-26 19:27:36,746 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 19:27:36,870 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 19:27:36,870 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-kcidebug-intel-68d6e4503c94e462c68821c8/lava_callback.json.gz DEBUG: Setup failed for 68d6e4503c94e462c68821c8. Transitting job node result: incomplete -> fail 2025-09-26 19:27:36,929 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:27:36,942 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6e4503c94e462c68821c8 HTTP/1.1" 200 None 2025-09-26 19:27:36,943 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:27:36,974 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6b9f52f83206171a92b71 HTTP/1.1" 200 None 2025-09-26 19:27:36,975 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:27:37,003 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6e4503c94e462c68821c8 HTTP/1.1" 200 None 2025-09-26 19:27:37,004 - lava_callback - INFO - Completed processing callback for node 68d6e4503c94e462c68821c8 INFO: 172.21.0.1:43886 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49788 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52840 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54984 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50194 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50196 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49982 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47692 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47704 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47716 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38818 - "GET / HTTP/1.0" 200 OK 2025-09-26 19:28:52,880 - lava_callback - INFO - Received callback for node 68d469cba5c2c05277bcb9c7 INFO: 172.21.0.1:42026 - "POST /node/68d469cba5c2c05277bcb9c7 HTTP/1.0" 202 Accepted 2025-09-26 19:28:52,886 - lava_callback - INFO - Starting async job submit for node 68d469cba5c2c05277bcb9c7 2025-09-26 19:28:52,907 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:28:52,921 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d469cba5c2c05277bcb9c7 HTTP/1.1" 200 None 2025-09-26 19:28:52,921 - lava_callback - DEBUG - Processing node 68d469cba5c2c05277bcb9c7: baseline-arm64 2025-09-26 19:28:53,237 - lava_callback - ERROR - Error processing callback for node 68d469cba5c2c05277bcb9c7: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:42030 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48858 - "GET / HTTP/1.0" 200 OK 2025-09-26 19:29:08,076 - lava_callback - INFO - Received callback for node 68d6e4d63c94e462c688221f INFO: 172.21.0.1:48864 - "POST /node/68d6e4d63c94e462c688221f HTTP/1.0" 202 Accepted 2025-09-26 19:29:08,085 - lava_callback - INFO - Starting async job submit for node 68d6e4d63c94e462c688221f 2025-09-26 19:29:08,156 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:29:08,168 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6e4d63c94e462c688221f HTTP/1.1" 200 None 2025-09-26 19:29:08,169 - lava_callback - DEBUG - Processing node 68d6e4d63c94e462c688221f: ltp-smoketest 2025-09-26 19:29:08,799 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 19:29:08,933 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 19:29:08,934 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoketest-68d6e4d63c94e462c688221f/log.txt.gz 2025-09-26 19:29:08,946 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 19:29:09,103 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 19:29:09,104 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoketest-68d6e4d63c94e462c688221f/lava_callback.json.gz 2025-09-26 19:29:09,243 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:29:09,260 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6e4d63c94e462c688221f HTTP/1.1" 200 None 2025-09-26 19:29:09,261 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:29:09,279 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6bd813c94e462c68814e8 HTTP/1.1" 200 None 2025-09-26 19:29:09,280 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:29:09,306 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6e4d63c94e462c688221f HTTP/1.1" 200 None 2025-09-26 19:29:09,307 - lava_callback - INFO - Completed processing callback for node 68d6e4d63c94e462c688221f INFO: 172.21.0.1:44310 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44320 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59268 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48320 - "GET / HTTP/1.0" 200 OK 2025-09-26 19:29:32,697 - lava_callback - INFO - Received callback for node 68d4cc92a5c2c05277bd0659 INFO: 172.21.0.1:48328 - "POST /node/68d4cc92a5c2c05277bd0659 HTTP/1.0" 202 Accepted 2025-09-26 19:29:32,703 - lava_callback - INFO - Starting async job submit for node 68d4cc92a5c2c05277bd0659 2025-09-26 19:29:32,723 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:29:32,736 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4cc92a5c2c05277bd0659 HTTP/1.1" 200 None 2025-09-26 19:29:32,736 - lava_callback - DEBUG - Processing node 68d4cc92a5c2c05277bd0659: baseline-arm 2025-09-26 19:29:32,959 - lava_callback - ERROR - Error processing callback for node 68d4cc92a5c2c05277bd0659: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:48338 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48352 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56518 - "GET / HTTP/1.0" 200 OK 2025-09-26 19:29:52,879 - lava_callback - INFO - Received callback for node 68d469cca5c2c05277bcb9ca INFO: 172.21.0.1:54368 - "POST /node/68d469cca5c2c05277bcb9ca HTTP/1.0" 202 Accepted 2025-09-26 19:29:52,885 - lava_callback - INFO - Starting async job submit for node 68d469cca5c2c05277bcb9ca 2025-09-26 19:29:52,905 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:29:52,918 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d469cca5c2c05277bcb9ca HTTP/1.1" 200 None 2025-09-26 19:29:52,919 - lava_callback - DEBUG - Processing node 68d469cca5c2c05277bcb9ca: baseline-arm64 2025-09-26 19:29:53,215 - lava_callback - ERROR - Error processing callback for node 68d469cca5c2c05277bcb9ca: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:54376 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35128 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54284 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54288 - "GET / HTTP/1.0" 200 OK 2025-09-26 19:30:24,633 - lava_callback - INFO - Received callback for node 68d469cfa5c2c05277bcb9cd INFO: 172.21.0.1:40742 - "POST /node/68d469cfa5c2c05277bcb9cd HTTP/1.0" 202 Accepted 2025-09-26 19:30:24,639 - lava_callback - INFO - Starting async job submit for node 68d469cfa5c2c05277bcb9cd 2025-09-26 19:30:24,659 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:30:24,672 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d469cfa5c2c05277bcb9cd HTTP/1.1" 200 None 2025-09-26 19:30:24,673 - lava_callback - DEBUG - Processing node 68d469cfa5c2c05277bcb9cd: baseline-arm64 2025-09-26 19:30:25,040 - lava_callback - ERROR - Error processing callback for node 68d469cfa5c2c05277bcb9cd: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:40754 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56110 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56126 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56134 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48938 - "GET / HTTP/1.0" 200 OK 2025-09-26 19:30:55,664 - lava_callback - INFO - Received callback for node 68d469d1a5c2c05277bcb9d0 INFO: 172.21.0.1:53712 - "POST /node/68d469d1a5c2c05277bcb9d0 HTTP/1.0" 202 Accepted 2025-09-26 19:30:55,670 - lava_callback - INFO - Starting async job submit for node 68d469d1a5c2c05277bcb9d0 2025-09-26 19:30:55,691 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:30:55,703 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d469d1a5c2c05277bcb9d0 HTTP/1.1" 200 None 2025-09-26 19:30:55,703 - lava_callback - DEBUG - Processing node 68d469d1a5c2c05277bcb9d0: baseline-arm64 2025-09-26 19:30:55,996 - lava_callback - ERROR - Error processing callback for node 68d469d1a5c2c05277bcb9d0: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:53714 - "GET / HTTP/1.0" 200 OK 2025-09-26 19:31:05,355 - lava_callback - INFO - Received callback for node 68d4552fa5c2c05277bca944 INFO: 172.21.0.1:52310 - "POST /node/68d4552fa5c2c05277bca944 HTTP/1.0" 202 Accepted 2025-09-26 19:31:05,362 - lava_callback - INFO - Starting async job submit for node 68d4552fa5c2c05277bca944 2025-09-26 19:31:05,392 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:31:05,409 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4552fa5c2c05277bca944 HTTP/1.1" 200 None 2025-09-26 19:31:05,409 - lava_callback - DEBUG - Processing node 68d4552fa5c2c05277bca944: baseline-arm-broonie 2025-09-26 19:31:05,638 - lava_callback - ERROR - Error processing callback for node 68d4552fa5c2c05277bca944: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:52322 - "GET / HTTP/1.0" 200 OK 2025-09-26 19:31:08,074 - lava_callback - INFO - Received callback for node 68d469d6a5c2c05277bcb9d3 INFO: 172.21.0.1:52334 - "POST /node/68d469d6a5c2c05277bcb9d3 HTTP/1.0" 202 Accepted 2025-09-26 19:31:08,082 - lava_callback - INFO - Starting async job submit for node 68d469d6a5c2c05277bcb9d3 2025-09-26 19:31:08,107 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:31:08,119 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d469d6a5c2c05277bcb9d3 HTTP/1.1" 200 None 2025-09-26 19:31:08,119 - lava_callback - DEBUG - Processing node 68d469d6a5c2c05277bcb9d3: baseline-arm64 2025-09-26 19:31:08,501 - lava_callback - ERROR - Error processing callback for node 68d469d6a5c2c05277bcb9d3: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:45790 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45804 - "GET / HTTP/1.0" 200 OK 2025-09-26 19:31:26,196 - lava_callback - INFO - Received callback for node 68d68a1b2f83206171a9184e INFO: 172.21.0.1:58136 - "POST /node/68d68a1b2f83206171a9184e HTTP/1.0" 202 Accepted 2025-09-26 19:31:26,202 - lava_callback - INFO - Starting async job submit for node 68d68a1b2f83206171a9184e 2025-09-26 19:31:26,229 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:31:26,245 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d68a1b2f83206171a9184e HTTP/1.1" 200 None 2025-09-26 19:31:26,245 - lava_callback - DEBUG - Processing node 68d68a1b2f83206171a9184e: baseline-arm64-broonie INFO: 172.21.0.1:58142 - "GET / HTTP/1.0" 200 OK 2025-09-26 19:31:26,608 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 19:31:26,735 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 19:31:26,736 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-68d68a1b2f83206171a9184e/log.txt.gz 2025-09-26 19:31:26,746 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 19:31:26,886 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 19:31:26,887 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-68d68a1b2f83206171a9184e/lava_callback.json.gz 2025-09-26 19:31:26,912 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:31:26,925 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d68a1b2f83206171a9184e HTTP/1.1" 200 None 2025-09-26 19:31:26,926 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:31:26,938 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d62c44c1bc82a30b37a916 HTTP/1.1" 200 None 2025-09-26 19:31:26,939 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:31:26,969 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d68a1b2f83206171a9184e HTTP/1.1" 200 None 2025-09-26 19:31:26,970 - lava_callback - INFO - Completed processing callback for node 68d68a1b2f83206171a9184e INFO: 172.21.0.1:56824 - "GET / HTTP/1.0" 200 OK 2025-09-26 19:31:35,690 - lava_callback - INFO - Received callback for node 68d4cca2a5c2c05277bd0680 INFO: 172.21.0.1:56826 - "POST /node/68d4cca2a5c2c05277bd0680 HTTP/1.0" 202 Accepted 2025-09-26 19:31:35,695 - lava_callback - INFO - Starting async job submit for node 68d4cca2a5c2c05277bd0680 2025-09-26 19:31:35,716 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:31:35,729 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4cca2a5c2c05277bd0680 HTTP/1.1" 200 None 2025-09-26 19:31:35,729 - lava_callback - DEBUG - Processing node 68d4cca2a5c2c05277bd0680: baseline-arm 2025-09-26 19:31:35,957 - lava_callback - ERROR - Error processing callback for node 68d4cca2a5c2c05277bd0680: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:56838 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56852 - "GET / HTTP/1.0" 200 OK 2025-09-26 19:31:43,870 - lava_callback - INFO - Received callback for node 68d469d9a5c2c05277bcb9d6 INFO: 172.21.0.1:40906 - "POST /node/68d469d9a5c2c05277bcb9d6 HTTP/1.0" 202 Accepted 2025-09-26 19:31:43,876 - lava_callback - INFO - Starting async job submit for node 68d469d9a5c2c05277bcb9d6 2025-09-26 19:31:43,897 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:31:43,909 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d469d9a5c2c05277bcb9d6 HTTP/1.1" 200 None 2025-09-26 19:31:43,910 - lava_callback - DEBUG - Processing node 68d469d9a5c2c05277bcb9d6: baseline-arm64 2025-09-26 19:31:44,220 - lava_callback - ERROR - Error processing callback for node 68d469d9a5c2c05277bcb9d6: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:40920 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48082 - "GET / HTTP/1.0" 200 OK 2025-09-26 19:32:06,010 - lava_callback - INFO - Received callback for node 68d6e8943c94e462c68822b4 INFO: 172.21.0.1:54588 - "POST /node/68d6e8943c94e462c68822b4 HTTP/1.0" 202 Accepted 2025-09-26 19:32:06,017 - lava_callback - INFO - Starting async job submit for node 68d6e8943c94e462c68822b4 2025-09-26 19:32:06,051 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:32:06,063 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6e8943c94e462c68822b4 HTTP/1.1" 200 None 2025-09-26 19:32:06,064 - lava_callback - DEBUG - Processing node 68d6e8943c94e462c68822b4: baseline-arm64-broonie INFO: 172.21.0.1:54590 - "GET / HTTP/1.0" 200 OK 2025-09-26 19:32:06,681 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 19:32:06,799 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 19:32:06,800 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-68d6e8943c94e462c68822b4/log.txt.gz 2025-09-26 19:32:06,812 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 19:32:06,938 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 19:32:06,939 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-68d6e8943c94e462c68822b4/lava_callback.json.gz 2025-09-26 19:32:06,973 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:32:06,985 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6e8943c94e462c68822b4 HTTP/1.1" 200 None 2025-09-26 19:32:06,986 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:32:07,020 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6b9dd2f83206171a92b48 HTTP/1.1" 200 None 2025-09-26 19:32:07,021 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:32:07,069 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6e8943c94e462c68822b4 HTTP/1.1" 200 None 2025-09-26 19:32:07,070 - lava_callback - INFO - Completed processing callback for node 68d6e8943c94e462c68822b4 INFO: 172.21.0.1:47748 - "GET /api/metrics HTTP/1.0" 200 OK 2025-09-26 19:32:16,176 - lava_callback - INFO - Received callback for node 68d469dea5c2c05277bcb9d9 INFO: 172.21.0.1:47760 - "POST /node/68d469dea5c2c05277bcb9d9 HTTP/1.0" 202 Accepted 2025-09-26 19:32:16,183 - lava_callback - INFO - Starting async job submit for node 68d469dea5c2c05277bcb9d9 2025-09-26 19:32:16,202 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:32:16,238 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d469dea5c2c05277bcb9d9 HTTP/1.1" 200 None 2025-09-26 19:32:16,238 - lava_callback - DEBUG - Processing node 68d469dea5c2c05277bcb9d9: baseline-arm64 INFO: 172.21.0.1:47768 - "GET / HTTP/1.0" 200 OK 2025-09-26 19:32:17,251 - lava_callback - ERROR - Error processing callback for node 68d469dea5c2c05277bcb9d9: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None 2025-09-26 19:32:26,424 - lava_callback - INFO - Received callback for node 68d469e0a5c2c05277bcb9dc INFO: 172.21.0.1:59726 - "POST /node/68d469e0a5c2c05277bcb9dc HTTP/1.0" 202 Accepted 2025-09-26 19:32:26,430 - lava_callback - INFO - Starting async job submit for node 68d469e0a5c2c05277bcb9dc 2025-09-26 19:32:26,451 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:32:26,462 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d469e0a5c2c05277bcb9dc HTTP/1.1" 200 None 2025-09-26 19:32:26,463 - lava_callback - DEBUG - Processing node 68d469e0a5c2c05277bcb9dc: baseline-arm64 INFO: 172.21.0.1:59730 - "GET / HTTP/1.0" 200 OK 2025-09-26 19:32:26,763 - lava_callback - ERROR - Error processing callback for node 68d469e0a5c2c05277bcb9dc: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:45068 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45072 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36172 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36184 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37578 - "GET / HTTP/1.0" 200 OK 2025-09-26 19:33:06,293 - lava_callback - INFO - Received callback for node 68d46a0aa5c2c05277bcba12 INFO: 172.21.0.1:44338 - "POST /node/68d46a0aa5c2c05277bcba12 HTTP/1.0" 202 Accepted 2025-09-26 19:33:06,299 - lava_callback - INFO - Starting async job submit for node 68d46a0aa5c2c05277bcba12 2025-09-26 19:33:06,319 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:33:06,360 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d46a0aa5c2c05277bcba12 HTTP/1.1" 200 None 2025-09-26 19:33:06,361 - lava_callback - DEBUG - Processing node 68d46a0aa5c2c05277bcba12: baseline-arm64 INFO: 172.21.0.1:44348 - "GET / HTTP/1.0" 200 OK 2025-09-26 19:33:06,683 - lava_callback - ERROR - Error processing callback for node 68d46a0aa5c2c05277bcba12: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:41196 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41202 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33056 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55922 - "GET / HTTP/1.0" 200 OK 2025-09-26 19:33:33,154 - lava_callback - INFO - Received callback for node 68d46a12a5c2c05277bcba18 INFO: 172.21.0.1:55926 - "POST /node/68d46a12a5c2c05277bcba18 HTTP/1.0" 202 Accepted 2025-09-26 19:33:33,161 - lava_callback - INFO - Starting async job submit for node 68d46a12a5c2c05277bcba18 2025-09-26 19:33:33,181 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:33:33,198 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d46a12a5c2c05277bcba18 HTTP/1.1" 200 None 2025-09-26 19:33:33,198 - lava_callback - DEBUG - Processing node 68d46a12a5c2c05277bcba18: baseline-arm64 2025-09-26 19:33:33,492 - lava_callback - ERROR - Error processing callback for node 68d46a12a5c2c05277bcba18: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:55928 - "GET / HTTP/1.0" 200 OK 2025-09-26 19:33:39,197 - lava_callback - INFO - Received callback for node 68d6cfdc3c94e462c6881d79 INFO: 172.21.0.1:55942 - "POST /node/68d6cfdc3c94e462c6881d79 HTTP/1.0" 202 Accepted 2025-09-26 19:33:39,205 - lava_callback - INFO - Starting async job submit for node 68d6cfdc3c94e462c6881d79 2025-09-26 19:33:39,246 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:33:39,258 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6cfdc3c94e462c6881d79 HTTP/1.1" 200 None 2025-09-26 19:33:39,258 - lava_callback - DEBUG - Processing node 68d6cfdc3c94e462c6881d79: baseline-arm64 INFO: 172.21.0.1:49112 - "GET / HTTP/1.0" 200 OK 2025-09-26 19:33:40,247 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 19:33:40,390 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 19:33:40,391 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6cfdc3c94e462c6881d79/log.txt.gz 2025-09-26 19:33:40,411 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 19:33:40,591 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 19:33:40,592 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6cfdc3c94e462c6881d79/lava_callback.json.gz DEBUG: Setup failed for 68d6cfdc3c94e462c6881d79. Transitting job node result: incomplete -> fail 2025-09-26 19:33:40,692 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:33:40,704 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6cfdc3c94e462c6881d79 HTTP/1.1" 200 None 2025-09-26 19:33:40,705 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:33:40,716 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d651ddcca86fb17472aa24 HTTP/1.1" 200 None 2025-09-26 19:33:40,717 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:33:40,745 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6cfdc3c94e462c6881d79 HTTP/1.1" 200 None 2025-09-26 19:33:40,746 - lava_callback - INFO - Completed processing callback for node 68d6cfdc3c94e462c6881d79 INFO: 172.21.0.1:49122 - "GET / HTTP/1.0" 200 OK 2025-09-26 19:33:50,877 - lava_callback - INFO - Received callback for node 68d46a0da5c2c05277bcba15 INFO: 172.21.0.1:45152 - "POST /node/68d46a0da5c2c05277bcba15 HTTP/1.0" 202 Accepted 2025-09-26 19:33:50,883 - lava_callback - INFO - Starting async job submit for node 68d46a0da5c2c05277bcba15 2025-09-26 19:33:50,904 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:33:50,917 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d46a0da5c2c05277bcba15 HTTP/1.1" 200 None 2025-09-26 19:33:50,917 - lava_callback - DEBUG - Processing node 68d46a0da5c2c05277bcba15: baseline-arm64 2025-09-26 19:33:51,208 - lava_callback - ERROR - Error processing callback for node 68d46a0da5c2c05277bcba15: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None 2025-09-26 19:33:52,265 - lava_callback - INFO - Received callback for node 68d4cca2a5c2c05277bd0683 INFO: 172.21.0.1:45154 - "POST /node/68d4cca2a5c2c05277bd0683 HTTP/1.0" 202 Accepted 2025-09-26 19:33:52,271 - lava_callback - INFO - Starting async job submit for node 68d4cca2a5c2c05277bd0683 2025-09-26 19:33:52,291 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:33:52,306 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4cca2a5c2c05277bd0683 HTTP/1.1" 200 None 2025-09-26 19:33:52,307 - lava_callback - DEBUG - Processing node 68d4cca2a5c2c05277bd0683: baseline-arm 2025-09-26 19:33:52,534 - lava_callback - ERROR - Error processing callback for node 68d4cca2a5c2c05277bd0683: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:45160 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40750 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47132 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47138 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51026 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34150 - "GET / HTTP/1.0" 200 OK 2025-09-26 19:34:36,059 - lava_callback - INFO - Received callback for node 68d46a1fa5c2c05277bcba24 INFO: 172.21.0.1:34162 - "POST /node/68d46a1fa5c2c05277bcba24 HTTP/1.0" 202 Accepted 2025-09-26 19:34:36,065 - lava_callback - INFO - Starting async job submit for node 68d46a1fa5c2c05277bcba24 2025-09-26 19:34:36,086 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:34:36,118 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d46a1fa5c2c05277bcba24 HTTP/1.1" 200 None 2025-09-26 19:34:36,119 - lava_callback - DEBUG - Processing node 68d46a1fa5c2c05277bcba24: baseline-arm64 2025-09-26 19:34:36,433 - lava_callback - ERROR - Error processing callback for node 68d46a1fa5c2c05277bcba24: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:34168 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50620 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50626 - "GET / HTTP/1.0" 200 OK 2025-09-26 19:34:54,167 - lava_callback - INFO - Received callback for node 68d46a15a5c2c05277bcba1b INFO: 172.21.0.1:43816 - "POST /node/68d46a15a5c2c05277bcba1b HTTP/1.0" 202 Accepted 2025-09-26 19:34:54,174 - lava_callback - INFO - Starting async job submit for node 68d46a15a5c2c05277bcba1b 2025-09-26 19:34:54,209 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:34:54,228 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d46a15a5c2c05277bcba1b HTTP/1.1" 200 None 2025-09-26 19:34:54,228 - lava_callback - DEBUG - Processing node 68d46a15a5c2c05277bcba1b: baseline-arm64 2025-09-26 19:34:54,627 - lava_callback - ERROR - Error processing callback for node 68d46a15a5c2c05277bcba1b: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:43826 - "GET / HTTP/1.0" 200 OK 2025-09-26 19:35:05,565 - lava_callback - INFO - Received callback for node 68d46a22a5c2c05277bcba27 INFO: 172.21.0.1:43080 - "POST /node/68d46a22a5c2c05277bcba27 HTTP/1.0" 202 Accepted 2025-09-26 19:35:05,571 - lava_callback - INFO - Starting async job submit for node 68d46a22a5c2c05277bcba27 2025-09-26 19:35:05,591 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:35:05,624 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d46a22a5c2c05277bcba27 HTTP/1.1" 200 None 2025-09-26 19:35:05,624 - lava_callback - DEBUG - Processing node 68d46a22a5c2c05277bcba27: baseline-arm64 2025-09-26 19:35:05,953 - lava_callback - ERROR - Error processing callback for node 68d46a22a5c2c05277bcba27: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:43082 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33732 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33746 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51848 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49324 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49338 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32906 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32914 - "GET / HTTP/1.0" 200 OK 2025-09-26 19:35:52,681 - lava_callback - INFO - Received callback for node 68d4cca4a5c2c05277bd0686 INFO: 172.21.0.1:37862 - "POST /node/68d4cca4a5c2c05277bd0686 HTTP/1.0" 202 Accepted 2025-09-26 19:35:52,686 - lava_callback - INFO - Starting async job submit for node 68d4cca4a5c2c05277bd0686 2025-09-26 19:35:52,707 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:35:52,749 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4cca4a5c2c05277bd0686 HTTP/1.1" 200 None 2025-09-26 19:35:52,750 - lava_callback - DEBUG - Processing node 68d4cca4a5c2c05277bd0686: baseline-arm 2025-09-26 19:35:52,975 - lava_callback - ERROR - Error processing callback for node 68d4cca4a5c2c05277bd0686: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:37872 - "GET / HTTP/1.0" 200 OK 2025-09-26 19:35:58,245 - lava_callback - INFO - Received callback for node 68d46a27a5c2c05277bcba2a INFO: 172.21.0.1:37882 - "POST /node/68d46a27a5c2c05277bcba2a HTTP/1.0" 202 Accepted 2025-09-26 19:35:58,251 - lava_callback - INFO - Starting async job submit for node 68d46a27a5c2c05277bcba2a 2025-09-26 19:35:58,271 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:35:58,284 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d46a27a5c2c05277bcba2a HTTP/1.1" 200 None 2025-09-26 19:35:58,285 - lava_callback - DEBUG - Processing node 68d46a27a5c2c05277bcba2a: baseline-arm64 2025-09-26 19:35:58,583 - lava_callback - ERROR - Error processing callback for node 68d46a27a5c2c05277bcba2a: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:51140 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40544 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40560 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55232 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55570 - "GET / HTTP/1.0" 200 OK 2025-09-26 19:36:31,778 - lava_callback - INFO - Received callback for node 68d46a4fa5c2c05277bcbfc6 INFO: 172.21.0.1:55572 - "POST /node/68d46a4fa5c2c05277bcbfc6 HTTP/1.0" 202 Accepted 2025-09-26 19:36:31,784 - lava_callback - INFO - Starting async job submit for node 68d46a4fa5c2c05277bcbfc6 2025-09-26 19:36:31,804 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:36:31,820 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d46a4fa5c2c05277bcbfc6 HTTP/1.1" 200 None 2025-09-26 19:36:31,820 - lava_callback - DEBUG - Processing node 68d46a4fa5c2c05277bcbfc6: baseline-arm64 2025-09-26 19:36:32,138 - lava_callback - ERROR - Error processing callback for node 68d46a4fa5c2c05277bcbfc6: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None 2025-09-26 19:36:33,122 - lava_callback - INFO - Received callback for node 68d46a2aa5c2c05277bcba45 INFO: 172.21.0.1:55578 - "POST /node/68d46a2aa5c2c05277bcba45 HTTP/1.0" 202 Accepted 2025-09-26 19:36:33,129 - lava_callback - INFO - Starting async job submit for node 68d46a2aa5c2c05277bcba45 2025-09-26 19:36:33,149 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:36:33,162 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d46a2aa5c2c05277bcba45 HTTP/1.1" 200 None 2025-09-26 19:36:33,162 - lava_callback - DEBUG - Processing node 68d46a2aa5c2c05277bcba45: baseline-arm64 2025-09-26 19:36:33,463 - lava_callback - ERROR - Error processing callback for node 68d46a2aa5c2c05277bcba45: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:55592 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39336 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39350 - "GET / HTTP/1.0" 200 OK 2025-09-26 19:36:51,130 - lava_callback - INFO - Received callback for node 68d6ea373c94e462c68823a7 INFO: 172.21.0.1:38424 - "POST /node/68d6ea373c94e462c68823a7 HTTP/1.0" 202 Accepted 2025-09-26 19:36:51,136 - lava_callback - INFO - Starting async job submit for node 68d6ea373c94e462c68823a7 2025-09-26 19:36:51,158 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:36:51,175 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6ea373c94e462c68823a7 HTTP/1.1" 200 None 2025-09-26 19:36:51,176 - lava_callback - DEBUG - Processing node 68d6ea373c94e462c68823a7: baseline-arm64-broonie 2025-09-26 19:36:51,503 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 19:36:51,674 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 19:36:51,675 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-68d6ea373c94e462c68823a7/log.txt.gz 2025-09-26 19:36:51,683 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 19:36:51,828 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 19:36:51,829 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-68d6ea373c94e462c68823a7/lava_callback.json.gz 2025-09-26 19:36:51,851 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:36:51,863 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6ea373c94e462c68823a7 HTTP/1.1" 200 None 2025-09-26 19:36:51,864 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:36:51,876 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6b9dd2f83206171a92b48 HTTP/1.1" 200 None 2025-09-26 19:36:51,877 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:36:51,913 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6ea373c94e462c68823a7 HTTP/1.1" 200 None 2025-09-26 19:36:51,914 - lava_callback - INFO - Completed processing callback for node 68d6ea373c94e462c68823a7 INFO: 172.21.0.1:38430 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60408 - "GET / HTTP/1.0" 200 OK 2025-09-26 19:37:08,276 - lava_callback - INFO - Received callback for node 68d6cfdd3c94e462c6881d7c INFO: 172.21.0.1:60424 - "POST /node/68d6cfdd3c94e462c6881d7c HTTP/1.0" 202 Accepted 2025-09-26 19:37:08,284 - lava_callback - INFO - Starting async job submit for node 68d6cfdd3c94e462c6881d7c 2025-09-26 19:37:08,324 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:37:08,341 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6cfdd3c94e462c6881d7c HTTP/1.1" 200 None 2025-09-26 19:37:08,342 - lava_callback - DEBUG - Processing node 68d6cfdd3c94e462c6881d7c: baseline-arm64 2025-09-26 19:37:09,431 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 19:37:09,556 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 19:37:09,557 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6cfdd3c94e462c6881d7c/log.txt.gz 2025-09-26 19:37:09,578 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 19:37:09,733 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 19:37:09,734 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6cfdd3c94e462c6881d7c/lava_callback.json.gz DEBUG: Setup failed for 68d6cfdd3c94e462c6881d7c. Transitting job node result: incomplete -> fail 2025-09-26 19:37:09,816 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:37:09,829 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6cfdd3c94e462c6881d7c HTTP/1.1" 200 None 2025-09-26 19:37:09,830 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:37:09,848 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d651ddcca86fb17472aa24 HTTP/1.1" 200 None 2025-09-26 19:37:09,849 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:37:09,897 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6cfdd3c94e462c6881d7c HTTP/1.1" 200 None 2025-09-26 19:37:09,898 - lava_callback - INFO - Completed processing callback for node 68d6cfdd3c94e462c6881d7c 2025-09-26 19:37:13,085 - lava_callback - INFO - Received callback for node 68d46a53a5c2c05277bcbfc9 INFO: 172.21.0.1:54594 - "POST /node/68d46a53a5c2c05277bcbfc9 HTTP/1.0" 202 Accepted 2025-09-26 19:37:13,093 - lava_callback - INFO - Starting async job submit for node 68d46a53a5c2c05277bcbfc9 2025-09-26 19:37:13,120 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:37:13,150 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d46a53a5c2c05277bcbfc9 HTTP/1.1" 200 None 2025-09-26 19:37:13,150 - lava_callback - DEBUG - Processing node 68d46a53a5c2c05277bcbfc9: baseline-arm64 2025-09-26 19:37:13,508 - lava_callback - ERROR - Error processing callback for node 68d46a53a5c2c05277bcbfc9: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:54598 - "GET /api/metrics HTTP/1.0" 200 OK 2025-09-26 19:37:16,310 - lava_callback - INFO - Received callback for node 68d4554fa5c2c05277bca970 INFO: 172.21.0.1:54608 - "POST /node/68d4554fa5c2c05277bca970 HTTP/1.0" 202 Accepted 2025-09-26 19:37:16,317 - lava_callback - INFO - Starting async job submit for node 68d4554fa5c2c05277bca970 2025-09-26 19:37:16,423 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:37:16,441 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4554fa5c2c05277bca970 HTTP/1.1" 200 None 2025-09-26 19:37:16,441 - lava_callback - DEBUG - Processing node 68d4554fa5c2c05277bca970: ltp-containers INFO: 172.21.0.1:54624 - "GET / HTTP/1.0" 200 OK 2025-09-26 19:37:17,226 - lava_callback - ERROR - Error processing callback for node 68d4554fa5c2c05277bca970: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:40190 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49884 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49892 - "GET / HTTP/1.0" 200 OK 2025-09-26 19:37:37,275 - lava_callback - INFO - Received callback for node 68d45571a5c2c05277bca991 INFO: 172.21.0.1:49894 - "POST /node/68d45571a5c2c05277bca991 HTTP/1.0" 202 Accepted 2025-09-26 19:37:37,293 - lava_callback - INFO - Starting async job submit for node 68d45571a5c2c05277bca991 2025-09-26 19:37:38,622 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:37:38,644 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d45571a5c2c05277bca991 HTTP/1.1" 200 None 2025-09-26 19:37:38,644 - lava_callback - DEBUG - Processing node 68d45571a5c2c05277bca991: ltp-syscalls INFO: 172.21.0.1:55144 - "GET / HTTP/1.0" 200 OK 2025-09-26 19:37:45,514 - lava_callback - ERROR - Error processing callback for node 68d45571a5c2c05277bca991: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:55148 - "GET / HTTP/1.0" 200 OK 2025-09-26 19:37:47,274 - lava_callback - INFO - Received callback for node 68d46a58a5c2c05277bcbfcc INFO: 172.21.0.1:55158 - "POST /node/68d46a58a5c2c05277bcbfcc HTTP/1.0" 202 Accepted 2025-09-26 19:37:47,280 - lava_callback - INFO - Starting async job submit for node 68d46a58a5c2c05277bcbfcc 2025-09-26 19:37:47,300 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:37:47,329 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d46a58a5c2c05277bcbfcc HTTP/1.1" 200 None 2025-09-26 19:37:47,329 - lava_callback - DEBUG - Processing node 68d46a58a5c2c05277bcbfcc: baseline-arm64 2025-09-26 19:37:47,622 - lava_callback - ERROR - Error processing callback for node 68d46a58a5c2c05277bcbfcc: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None 2025-09-26 19:37:48,638 - lava_callback - INFO - Received callback for node 68d46a5ba5c2c05277bcbfcf INFO: 172.21.0.1:55164 - "POST /node/68d46a5ba5c2c05277bcbfcf HTTP/1.0" 202 Accepted 2025-09-26 19:37:48,644 - lava_callback - INFO - Starting async job submit for node 68d46a5ba5c2c05277bcbfcf 2025-09-26 19:37:48,664 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:37:48,677 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d46a5ba5c2c05277bcbfcf HTTP/1.1" 200 None 2025-09-26 19:37:48,677 - lava_callback - DEBUG - Processing node 68d46a5ba5c2c05277bcbfcf: baseline-arm64 2025-09-26 19:37:48,855 - lava_callback - INFO - Received callback for node 68d4cca4a5c2c05277bd0689 INFO: 172.21.0.1:55180 - "POST /node/68d4cca4a5c2c05277bd0689 HTTP/1.0" 202 Accepted 2025-09-26 19:37:48,876 - lava_callback - INFO - Starting async job submit for node 68d4cca4a5c2c05277bd0689 2025-09-26 19:37:48,907 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:37:48,969 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4cca4a5c2c05277bd0689 HTTP/1.1" 200 None 2025-09-26 19:37:48,985 - lava_callback - DEBUG - Processing node 68d4cca4a5c2c05277bd0689: baseline-arm 2025-09-26 19:37:49,024 - lava_callback - ERROR - Error processing callback for node 68d46a5ba5c2c05277bcbfcf: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None 2025-09-26 19:37:49,233 - lava_callback - ERROR - Error processing callback for node 68d4cca4a5c2c05277bd0689: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:48320 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49224 - "GET / HTTP/1.0" 200 OK 2025-09-26 19:38:14,943 - lava_callback - INFO - Received callback for node 68d46a67a5c2c05277bcbfe7 INFO: 172.21.0.1:39904 - "POST /node/68d46a67a5c2c05277bcbfe7 HTTP/1.0" 202 Accepted 2025-09-26 19:38:14,950 - lava_callback - INFO - Starting async job submit for node 68d46a67a5c2c05277bcbfe7 2025-09-26 19:38:14,986 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:38:14,998 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d46a67a5c2c05277bcbfe7 HTTP/1.1" 200 None 2025-09-26 19:38:14,998 - lava_callback - DEBUG - Processing node 68d46a67a5c2c05277bcbfe7: baseline-arm64 INFO: 172.21.0.1:39916 - "GET /api/metrics HTTP/1.0" 200 OK 2025-09-26 19:38:15,378 - lava_callback - ERROR - Error processing callback for node 68d46a67a5c2c05277bcbfe7: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:39928 - "GET / HTTP/1.0" 200 OK 2025-09-26 19:38:25,705 - lava_callback - INFO - Received callback for node 68d690602f83206171a91b6d INFO: 172.21.0.1:33932 - "POST /node/68d690602f83206171a91b6d HTTP/1.0" 202 Accepted 2025-09-26 19:38:25,711 - lava_callback - INFO - Starting async job submit for node 68d690602f83206171a91b6d 2025-09-26 19:38:25,746 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:38:25,758 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d690602f83206171a91b6d HTTP/1.1" 200 None 2025-09-26 19:38:25,759 - lava_callback - DEBUG - Processing node 68d690602f83206171a91b6d: baseline-arm64-broonie 2025-09-26 19:38:26,238 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 19:38:26,386 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 19:38:26,387 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-68d690602f83206171a91b6d/log.txt.gz 2025-09-26 19:38:26,397 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.21.0.1:33946 - "GET / HTTP/1.0" 200 OK 2025-09-26 19:38:26,539 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 19:38:26,540 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-68d690602f83206171a91b6d/lava_callback.json.gz 2025-09-26 19:38:26,575 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:38:26,609 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d690602f83206171a91b6d HTTP/1.1" 200 None 2025-09-26 19:38:26,610 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:38:26,622 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d5dc8530c58dac7d9410cb HTTP/1.1" 200 None 2025-09-26 19:38:26,623 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:38:26,662 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d690602f83206171a91b6d HTTP/1.1" 200 None 2025-09-26 19:38:26,663 - lava_callback - INFO - Completed processing callback for node 68d690602f83206171a91b6d INFO: 172.21.0.1:48044 - "GET / HTTP/1.0" 200 OK 2025-09-26 19:38:35,750 - lava_callback - INFO - Received callback for node 68d448a6a5c2c05277bc9f8e INFO: 172.21.0.1:48056 - "POST /node/68d448a6a5c2c05277bc9f8e HTTP/1.0" 202 Accepted 2025-09-26 19:38:35,755 - lava_callback - INFO - Starting async job submit for node 68d448a6a5c2c05277bc9f8e 2025-09-26 19:38:35,756 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:38:35,770 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d448a6a5c2c05277bc9f8e HTTP/1.1" 200 None 2025-09-26 19:38:35,770 - lava_callback - DEBUG - Processing node 68d448a6a5c2c05277bc9f8e: baseline-arm 2025-09-26 19:38:35,771 - lava_callback - ERROR - Error processing callback for node 68d448a6a5c2c05277bc9f8e: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:48058 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41512 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41526 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58312 - "GET / HTTP/1.0" 200 OK 2025-09-26 19:39:00,246 - lava_callback - INFO - Received callback for node 68d6cfe03c94e462c6881d82 INFO: 172.21.0.1:44192 - "POST /node/68d6cfe03c94e462c6881d82 HTTP/1.0" 202 Accepted 2025-09-26 19:39:00,253 - lava_callback - INFO - Starting async job submit for node 68d6cfe03c94e462c6881d82 2025-09-26 19:39:00,293 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:39:00,326 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6cfe03c94e462c6881d82 HTTP/1.1" 200 None 2025-09-26 19:39:00,327 - lava_callback - DEBUG - Processing node 68d6cfe03c94e462c6881d82: baseline-arm64 2025-09-26 19:39:01,378 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 19:39:01,527 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 19:39:01,528 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6cfe03c94e462c6881d82/log.txt.gz 2025-09-26 19:39:01,549 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 19:39:01,725 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 19:39:01,725 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6cfe03c94e462c6881d82/lava_callback.json.gz DEBUG: Setup failed for 68d6cfe03c94e462c6881d82. Transitting job node result: incomplete -> fail 2025-09-26 19:39:01,815 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:39:01,827 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6cfe03c94e462c6881d82 HTTP/1.1" 200 None 2025-09-26 19:39:01,828 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:39:01,840 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d651ddcca86fb17472aa24 HTTP/1.1" 200 None 2025-09-26 19:39:01,841 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:39:01,870 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6cfe03c94e462c6881d82 HTTP/1.1" 200 None 2025-09-26 19:39:01,871 - lava_callback - INFO - Completed processing callback for node 68d6cfe03c94e462c6881d82 INFO: 172.21.0.1:44196 - "GET / HTTP/1.0" 200 OK 2025-09-26 19:39:07,330 - lava_callback - INFO - Received callback for node 68d46a6ea5c2c05277bcbfed INFO: 172.21.0.1:44206 - "POST /node/68d46a6ea5c2c05277bcbfed HTTP/1.0" 202 Accepted 2025-09-26 19:39:07,336 - lava_callback - INFO - Starting async job submit for node 68d46a6ea5c2c05277bcbfed 2025-09-26 19:39:07,357 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:39:07,369 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d46a6ea5c2c05277bcbfed HTTP/1.1" 200 None 2025-09-26 19:39:07,369 - lava_callback - DEBUG - Processing node 68d46a6ea5c2c05277bcbfed: baseline-arm64 2025-09-26 19:39:07,668 - lava_callback - ERROR - Error processing callback for node 68d46a6ea5c2c05277bcbfed: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None 2025-09-26 19:39:08,740 - lava_callback - INFO - Received callback for node 68d46a69a5c2c05277bcbfea INFO: 172.21.0.1:44210 - "POST /node/68d46a69a5c2c05277bcbfea HTTP/1.0" 202 Accepted 2025-09-26 19:39:08,746 - lava_callback - INFO - Starting async job submit for node 68d46a69a5c2c05277bcbfea 2025-09-26 19:39:08,787 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:39:08,811 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d46a69a5c2c05277bcbfea HTTP/1.1" 200 None 2025-09-26 19:39:08,812 - lava_callback - DEBUG - Processing node 68d46a69a5c2c05277bcbfea: baseline-arm64 2025-09-26 19:39:09,109 - lava_callback - ERROR - Error processing callback for node 68d46a69a5c2c05277bcbfea: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:37196 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37210 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34496 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43216 - "GET / HTTP/1.0" 200 OK 2025-09-26 19:39:33,250 - lava_callback - INFO - Received callback for node 68d46a71a5c2c05277bcbff0 INFO: 172.21.0.1:43222 - "POST /node/68d46a71a5c2c05277bcbff0 HTTP/1.0" 202 Accepted 2025-09-26 19:39:33,256 - lava_callback - INFO - Starting async job submit for node 68d46a71a5c2c05277bcbff0 2025-09-26 19:39:33,277 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:39:33,295 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d46a71a5c2c05277bcbff0 HTTP/1.1" 200 None 2025-09-26 19:39:33,296 - lava_callback - DEBUG - Processing node 68d46a71a5c2c05277bcbff0: baseline-arm64 2025-09-26 19:39:33,613 - lava_callback - ERROR - Error processing callback for node 68d46a71a5c2c05277bcbff0: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:43238 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47568 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47578 - "GET / HTTP/1.0" 200 OK 2025-09-26 19:39:56,046 - lava_callback - INFO - Received callback for node 68d4dab5a5c2c05277bd0b6d INFO: 172.21.0.1:56620 - "POST /node/68d4dab5a5c2c05277bd0b6d HTTP/1.0" 202 Accepted 2025-09-26 19:39:56,052 - lava_callback - INFO - Starting async job submit for node 68d4dab5a5c2c05277bd0b6d 2025-09-26 19:39:56,073 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:39:56,109 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4dab5a5c2c05277bd0b6d HTTP/1.1" 200 None 2025-09-26 19:39:56,109 - lava_callback - DEBUG - Processing node 68d4dab5a5c2c05277bd0b6d: baseline-arm 2025-09-26 19:39:56,360 - lava_callback - ERROR - Error processing callback for node 68d4dab5a5c2c05277bd0b6d: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:56636 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43516 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53434 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53446 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56220 - "GET / HTTP/1.0" 200 OK 2025-09-26 19:40:30,984 - lava_callback - INFO - Received callback for node 68d6cfe03c94e462c6881d85 INFO: 172.21.0.1:40442 - "POST /node/68d6cfe03c94e462c6881d85 HTTP/1.0" 202 Accepted 2025-09-26 19:40:30,990 - lava_callback - INFO - Starting async job submit for node 68d6cfe03c94e462c6881d85 2025-09-26 19:40:31,024 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:40:31,060 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6cfe03c94e462c6881d85 HTTP/1.1" 200 None 2025-09-26 19:40:31,061 - lava_callback - DEBUG - Processing node 68d6cfe03c94e462c6881d85: baseline-arm64 INFO: 172.21.0.1:40458 - "GET / HTTP/1.0" 200 OK 2025-09-26 19:40:31,571 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 19:40:31,718 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 19:40:31,719 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6cfe03c94e462c6881d85/log.txt.gz 2025-09-26 19:40:31,730 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 19:40:31,857 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 19:40:31,858 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6cfe03c94e462c6881d85/lava_callback.json.gz DEBUG: Setup failed for 68d6cfe03c94e462c6881d85. Transitting job node result: incomplete -> fail 2025-09-26 19:40:31,924 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:40:31,944 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6cfe03c94e462c6881d85 HTTP/1.1" 200 None 2025-09-26 19:40:31,945 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:40:31,958 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d651ddcca86fb17472aa24 HTTP/1.1" 200 None 2025-09-26 19:40:31,959 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:40:31,986 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6cfe03c94e462c6881d85 HTTP/1.1" 200 None 2025-09-26 19:40:31,986 - lava_callback - INFO - Completed processing callback for node 68d6cfe03c94e462c6881d85 INFO: 172.21.0.1:40466 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35974 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35986 - "GET / HTTP/1.0" 200 OK 2025-09-26 19:40:54,693 - lava_callback - INFO - Received callback for node 68d693f22f83206171a91deb INFO: 172.21.0.1:57482 - "POST /node/68d693f22f83206171a91deb HTTP/1.0" 202 Accepted 2025-09-26 19:40:54,700 - lava_callback - INFO - Starting async job submit for node 68d693f22f83206171a91deb 2025-09-26 19:40:54,722 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:40:54,735 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d693f22f83206171a91deb HTTP/1.1" 200 None 2025-09-26 19:40:54,736 - lava_callback - DEBUG - Processing node 68d693f22f83206171a91deb: baseline-arm64-broonie 2025-09-26 19:40:55,052 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 19:40:55,201 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 19:40:55,202 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-68d693f22f83206171a91deb/log.txt.gz 2025-09-26 19:40:55,209 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 19:40:55,347 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 19:40:55,348 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-68d693f22f83206171a91deb/lava_callback.json.gz 2025-09-26 19:40:55,369 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:40:55,381 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d693f22f83206171a91deb HTTP/1.1" 200 None 2025-09-26 19:40:55,382 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:40:55,400 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d5d90f2be2da466069ac22 HTTP/1.1" 200 None 2025-09-26 19:40:55,401 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:40:55,431 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d693f22f83206171a91deb HTTP/1.1" 200 None 2025-09-26 19:40:55,432 - lava_callback - INFO - Completed processing callback for node 68d693f22f83206171a91deb INFO: 172.21.0.1:57490 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38330 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56036 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56048 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50886 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47452 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47468 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40704 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40706 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40732 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53722 - "GET / HTTP/1.0" 200 OK 2025-09-26 19:42:12,420 - lava_callback - INFO - Received callback for node 68d4dab5a5c2c05277bd0b70 INFO: 172.21.0.1:44574 - "POST /node/68d4dab5a5c2c05277bd0b70 HTTP/1.0" 202 Accepted 2025-09-26 19:42:12,426 - lava_callback - INFO - Starting async job submit for node 68d4dab5a5c2c05277bd0b70 2025-09-26 19:42:12,446 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:42:12,458 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4dab5a5c2c05277bd0b70 HTTP/1.1" 200 None 2025-09-26 19:42:12,459 - lava_callback - DEBUG - Processing node 68d4dab5a5c2c05277bd0b70: baseline-arm 2025-09-26 19:42:12,690 - lava_callback - ERROR - Error processing callback for node 68d4dab5a5c2c05277bd0b70: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:44584 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44592 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33756 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58866 - "GET / HTTP/1.0" 200 OK 2025-09-26 19:42:35,621 - lava_callback - INFO - Received callback for node 68d448a3a5c2c05277bc9f8b INFO: 172.21.0.1:58878 - "POST /node/68d448a3a5c2c05277bc9f8b HTTP/1.0" 202 Accepted 2025-09-26 19:42:35,627 - lava_callback - INFO - Starting async job submit for node 68d448a3a5c2c05277bc9f8b 2025-09-26 19:42:35,656 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:42:35,690 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d448a3a5c2c05277bc9f8b HTTP/1.1" 200 None 2025-09-26 19:42:35,691 - lava_callback - DEBUG - Processing node 68d448a3a5c2c05277bc9f8b: baseline-arm 2025-09-26 19:42:35,951 - lava_callback - ERROR - Error processing callback for node 68d448a3a5c2c05277bc9f8b: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:58888 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57690 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57698 - "GET / HTTP/1.0" 200 OK 2025-09-26 19:42:46,824 - lava_callback - INFO - Received callback for node 68d4554da5c2c05277bca96d INFO: 172.21.0.1:57714 - "POST /node/68d4554da5c2c05277bca96d HTTP/1.0" 202 Accepted 2025-09-26 19:42:46,831 - lava_callback - INFO - Starting async job submit for node 68d4554da5c2c05277bca96d 2025-09-26 19:42:46,873 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:42:46,894 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4554da5c2c05277bca96d HTTP/1.1" 200 None 2025-09-26 19:42:46,895 - lava_callback - DEBUG - Processing node 68d4554da5c2c05277bca96d: ltp-cap-bounds 2025-09-26 19:42:47,354 - lava_callback - ERROR - Error processing callback for node 68d4554da5c2c05277bca96d: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:55832 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39688 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45644 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45652 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41712 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56670 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56674 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50706 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50720 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39476 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45872 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33076 - "GET /api/metrics HTTP/1.0" 200 OK 2025-09-26 19:44:16,443 - lava_callback - INFO - Received callback for node 68d4dab7a5c2c05277bd0b73 INFO: 172.21.0.1:33080 - "POST /node/68d4dab7a5c2c05277bd0b73 HTTP/1.0" 202 Accepted 2025-09-26 19:44:16,449 - lava_callback - INFO - Starting async job submit for node 68d4dab7a5c2c05277bd0b73 2025-09-26 19:44:16,470 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:44:16,484 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4dab7a5c2c05277bd0b73 HTTP/1.1" 200 None 2025-09-26 19:44:16,484 - lava_callback - DEBUG - Processing node 68d4dab7a5c2c05277bd0b73: baseline-arm INFO: 172.21.0.1:33092 - "GET / HTTP/1.0" 200 OK 2025-09-26 19:44:16,706 - lava_callback - ERROR - Error processing callback for node 68d4dab7a5c2c05277bd0b73: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:57070 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59528 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59542 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40802 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40806 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51940 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42476 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44410 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44418 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54590 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38024 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38030 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45382 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45398 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41692 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42362 - "GET / HTTP/1.0" 200 OK 2025-09-26 19:46:08,743 - lava_callback - INFO - Received callback for node 68d694f22f83206171a91f83 INFO: 172.21.0.1:42376 - "POST /node/68d694f22f83206171a91f83 HTTP/1.0" 202 Accepted 2025-09-26 19:46:08,749 - lava_callback - INFO - Starting async job submit for node 68d694f22f83206171a91f83 2025-09-26 19:46:08,783 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:46:08,825 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d694f22f83206171a91f83 HTTP/1.1" 200 None 2025-09-26 19:46:08,826 - lava_callback - DEBUG - Processing node 68d694f22f83206171a91f83: baseline-arm64-broonie 2025-09-26 19:46:09,317 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 19:46:09,460 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 19:46:09,460 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-68d694f22f83206171a91f83/log.txt.gz 2025-09-26 19:46:09,470 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 19:46:09,610 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 19:46:09,610 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-68d694f22f83206171a91f83/lava_callback.json.gz 2025-09-26 19:46:09,645 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:46:09,658 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d694f22f83206171a91f83 HTTP/1.1" 200 None 2025-09-26 19:46:09,659 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:46:09,670 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d637475d233c363a761c21 HTTP/1.1" 200 None 2025-09-26 19:46:09,671 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:46:09,696 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d694f22f83206171a91f83 HTTP/1.1" 200 None 2025-09-26 19:46:09,696 - lava_callback - INFO - Completed processing callback for node 68d694f22f83206171a91f83 INFO: 172.21.0.1:40370 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40378 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41710 - "GET / HTTP/1.0" 200 OK 2025-09-26 19:46:28,304 - lava_callback - INFO - Received callback for node 68d6a74c2f83206171a924ea INFO: 172.21.0.1:41714 - "POST /node/68d6a74c2f83206171a924ea HTTP/1.0" 202 Accepted 2025-09-26 19:46:28,310 - lava_callback - INFO - Starting async job submit for node 68d6a74c2f83206171a924ea 2025-09-26 19:46:28,332 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:46:28,344 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6a74c2f83206171a924ea HTTP/1.1" 200 None 2025-09-26 19:46:28,345 - lava_callback - DEBUG - Processing node 68d6a74c2f83206171a924ea: baseline-arm64-broonie 2025-09-26 19:46:28,558 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 19:46:28,666 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 19:46:28,667 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-68d6a74c2f83206171a924ea/log.txt.gz 2025-09-26 19:46:28,673 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 19:46:28,803 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 19:46:28,803 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-68d6a74c2f83206171a924ea/lava_callback.json.gz 2025-09-26 19:46:28,826 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:46:28,859 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6a74c2f83206171a924ea HTTP/1.1" 200 None 2025-09-26 19:46:28,860 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:46:28,872 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d648ed5d233c363a762351 HTTP/1.1" 200 None 2025-09-26 19:46:28,873 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:46:28,900 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6a74c2f83206171a924ea HTTP/1.1" 200 None 2025-09-26 19:46:28,900 - lava_callback - INFO - Completed processing callback for node 68d6a74c2f83206171a924ea INFO: 172.21.0.1:53392 - "GET / HTTP/1.0" 200 OK 2025-09-26 19:46:33,024 - lava_callback - INFO - Received callback for node 68d4dab7a5c2c05277bd0b76 INFO: 172.21.0.1:53408 - "POST /node/68d4dab7a5c2c05277bd0b76 HTTP/1.0" 202 Accepted 2025-09-26 19:46:33,030 - lava_callback - INFO - Starting async job submit for node 68d4dab7a5c2c05277bd0b76 2025-09-26 19:46:33,050 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:46:33,063 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4dab7a5c2c05277bd0b76 HTTP/1.1" 200 None 2025-09-26 19:46:33,063 - lava_callback - DEBUG - Processing node 68d4dab7a5c2c05277bd0b76: baseline-arm 2025-09-26 19:46:33,312 - lava_callback - ERROR - Error processing callback for node 68d4dab7a5c2c05277bd0b76: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:53414 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42818 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42822 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56904 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41226 - "GET / HTTP/1.0" 200 OK 2025-09-26 19:47:09,344 - lava_callback - INFO - Received callback for node 68d6e9943c94e462c6882303 INFO: 172.21.0.1:44556 - "POST /node/68d6e9943c94e462c6882303 HTTP/1.0" 202 Accepted 2025-09-26 19:47:09,351 - lava_callback - INFO - Starting async job submit for node 68d6e9943c94e462c6882303 2025-09-26 19:47:09,409 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:47:09,421 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6e9943c94e462c6882303 HTTP/1.1" 200 None 2025-09-26 19:47:09,422 - lava_callback - DEBUG - Processing node 68d6e9943c94e462c6882303: ltp-smoketest 2025-09-26 19:47:09,885 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 19:47:09,994 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 19:47:09,995 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoketest-68d6e9943c94e462c6882303/log.txt.gz 2025-09-26 19:47:10,007 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 19:47:10,135 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 19:47:10,135 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoketest-68d6e9943c94e462c6882303/lava_callback.json.gz 2025-09-26 19:47:10,251 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:47:10,272 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6e9943c94e462c6882303 HTTP/1.1" 200 None 2025-09-26 19:47:10,273 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:47:10,284 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6bd813c94e462c68814e8 HTTP/1.1" 200 None 2025-09-26 19:47:10,285 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:47:10,315 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6e9943c94e462c6882303 HTTP/1.1" 200 None 2025-09-26 19:47:10,316 - lava_callback - INFO - Completed processing callback for node 68d6e9943c94e462c6882303 INFO: 172.21.0.1:44560 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44568 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53830 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56214 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56220 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51516 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51530 - "GET / HTTP/1.0" 200 OK 2025-09-26 19:47:56,508 - lava_callback - INFO - Received callback for node 68d4541ba5c2c05277bca8af INFO: 172.21.0.1:50494 - "POST /node/68d4541ba5c2c05277bca8af HTTP/1.0" 202 Accepted 2025-09-26 19:47:56,514 - lava_callback - INFO - Starting async job submit for node 68d4541ba5c2c05277bca8af INFO: 172.21.0.1:50500 - "GET / HTTP/1.0" 200 OK 2025-09-26 19:47:56,546 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:47:56,559 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4541ba5c2c05277bca8af HTTP/1.1" 200 None 2025-09-26 19:47:56,559 - lava_callback - DEBUG - Processing node 68d4541ba5c2c05277bca8af: ltp-sched 2025-09-26 19:47:56,910 - lava_callback - ERROR - Error processing callback for node 68d4541ba5c2c05277bca8af: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:42230 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37266 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37274 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47790 - "GET / HTTP/1.0" 200 OK 2025-09-26 19:48:30,380 - lava_callback - INFO - Received callback for node 68d6a8592f83206171a9263c INFO: 172.21.0.1:46080 - "POST /node/68d6a8592f83206171a9263c HTTP/1.0" 202 Accepted 2025-09-26 19:48:30,386 - lava_callback - INFO - Starting async job submit for node 68d6a8592f83206171a9263c 2025-09-26 19:48:30,408 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:48:30,420 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6a8592f83206171a9263c HTTP/1.1" 200 None 2025-09-26 19:48:30,420 - lava_callback - DEBUG - Processing node 68d6a8592f83206171a9263c: baseline-arm64-broonie 2025-09-26 19:48:30,626 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 19:48:30,780 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 19:48:30,780 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-68d6a8592f83206171a9263c/log.txt.gz 2025-09-26 19:48:30,786 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 19:48:30,915 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 19:48:30,916 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-68d6a8592f83206171a9263c/lava_callback.json.gz 2025-09-26 19:48:30,938 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:48:30,950 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6a8592f83206171a9263c HTTP/1.1" 200 None 2025-09-26 19:48:30,951 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:48:30,970 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d64d12a5c3ab168ecc7a3d HTTP/1.1" 200 None 2025-09-26 19:48:30,972 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:48:30,999 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6a8592f83206171a9263c HTTP/1.1" 200 None 2025-09-26 19:48:31,000 - lava_callback - INFO - Completed processing callback for node 68d6a8592f83206171a9263c INFO: 172.21.0.1:46082 - "GET / HTTP/1.0" 200 OK 2025-09-26 19:48:33,116 - lava_callback - INFO - Received callback for node 68d4dae8a5c2c05277bd0b7d INFO: 172.21.0.1:46088 - "POST /node/68d4dae8a5c2c05277bd0b7d HTTP/1.0" 202 Accepted 2025-09-26 19:48:33,122 - lava_callback - INFO - Starting async job submit for node 68d4dae8a5c2c05277bd0b7d 2025-09-26 19:48:33,143 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:48:33,155 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4dae8a5c2c05277bd0b7d HTTP/1.1" 200 None 2025-09-26 19:48:33,155 - lava_callback - DEBUG - Processing node 68d4dae8a5c2c05277bd0b7d: baseline-arm 2025-09-26 19:48:33,383 - lava_callback - ERROR - Error processing callback for node 68d4dae8a5c2c05277bd0b7d: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:46104 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55778 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55782 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55704 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55812 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42424 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42426 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55378 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58740 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58752 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38886 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38898 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48760 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55962 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55662 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55666 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54926 - "GET / HTTP/1.0" 200 OK 2025-09-26 19:50:28,244 - lava_callback - INFO - Received callback for node 68d6ae3a2f83206171a9288f INFO: 172.21.0.1:54936 - "POST /node/68d6ae3a2f83206171a9288f HTTP/1.0" 202 Accepted 2025-09-26 19:50:28,250 - lava_callback - INFO - Starting async job submit for node 68d6ae3a2f83206171a9288f 2025-09-26 19:50:28,272 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:50:28,284 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6ae3a2f83206171a9288f HTTP/1.1" 200 None 2025-09-26 19:50:28,284 - lava_callback - DEBUG - Processing node 68d6ae3a2f83206171a9288f: baseline-arm64-broonie 2025-09-26 19:50:28,515 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 19:50:28,670 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 19:50:28,671 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-68d6ae3a2f83206171a9288f/log.txt.gz 2025-09-26 19:50:28,677 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 19:50:28,812 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 19:50:28,813 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-68d6ae3a2f83206171a9288f/lava_callback.json.gz 2025-09-26 19:50:28,835 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:50:28,861 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6ae3a2f83206171a9288f HTTP/1.1" 200 None 2025-09-26 19:50:28,862 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:50:28,896 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d651ddcca86fb17472aa24 HTTP/1.1" 200 None 2025-09-26 19:50:28,897 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:50:28,924 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6ae3a2f83206171a9288f HTTP/1.1" 200 None 2025-09-26 19:50:28,927 - lava_callback - INFO - Completed processing callback for node 68d6ae3a2f83206171a9288f INFO: 172.21.0.1:43452 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43468 - "GET / HTTP/1.0" 200 OK 2025-09-26 19:50:37,588 - lava_callback - INFO - Received callback for node 68d4dae9a5c2c05277bd0b84 INFO: 172.21.0.1:43480 - "POST /node/68d4dae9a5c2c05277bd0b84 HTTP/1.0" 202 Accepted 2025-09-26 19:50:37,594 - lava_callback - INFO - Starting async job submit for node 68d4dae9a5c2c05277bd0b84 2025-09-26 19:50:37,615 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:50:37,628 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4dae9a5c2c05277bd0b84 HTTP/1.1" 200 None 2025-09-26 19:50:37,628 - lava_callback - DEBUG - Processing node 68d4dae9a5c2c05277bd0b84: baseline-arm 2025-09-26 19:50:37,853 - lava_callback - ERROR - Error processing callback for node 68d4dae9a5c2c05277bd0b84: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:37554 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37564 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43920 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36530 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50448 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50460 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60828 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43962 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43974 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38048 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38062 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43224 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52358 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42484 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42490 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40802 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37414 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37420 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55316 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55324 - "GET / HTTP/1.0" 200 OK 2025-09-26 19:52:55,867 - lava_callback - INFO - Received callback for node 68d4daeba5c2c05277bd0b87 INFO: 172.21.0.1:50810 - "POST /node/68d4daeba5c2c05277bd0b87 HTTP/1.0" 202 Accepted 2025-09-26 19:52:55,873 - lava_callback - INFO - Starting async job submit for node 68d4daeba5c2c05277bd0b87 2025-09-26 19:52:55,894 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:52:55,906 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4daeba5c2c05277bd0b87 HTTP/1.1" 200 None 2025-09-26 19:52:55,907 - lava_callback - DEBUG - Processing node 68d4daeba5c2c05277bd0b87: baseline-arm 2025-09-26 19:52:56,129 - lava_callback - ERROR - Error processing callback for node 68d4daeba5c2c05277bd0b87: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:50814 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40854 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33826 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33832 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48826 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41892 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41904 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33338 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33342 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55608 - "GET / HTTP/1.0" 200 OK 2025-09-26 19:53:57,310 - lava_callback - INFO - Received callback for node 68d45552a5c2c05277bca973 INFO: 172.21.0.1:55610 - "POST /node/68d45552a5c2c05277bca973 HTTP/1.0" 202 Accepted 2025-09-26 19:53:57,322 - lava_callback - INFO - Starting async job submit for node 68d45552a5c2c05277bca973 2025-09-26 19:53:57,694 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:53:57,715 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d45552a5c2c05277bca973 HTTP/1.1" 200 None 2025-09-26 19:53:57,715 - lava_callback - DEBUG - Processing node 68d45552a5c2c05277bca973: ltp-controllers 2025-09-26 19:54:00,563 - lava_callback - ERROR - Error processing callback for node 68d45552a5c2c05277bca973: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:59518 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53352 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53368 - "GET / HTTP/1.0" 200 OK 2025-09-26 19:54:21,320 - lava_callback - INFO - Received callback for node 68d6cfe33c94e462c6881d8e INFO: 172.21.0.1:58088 - "POST /node/68d6cfe33c94e462c6881d8e HTTP/1.0" 202 Accepted 2025-09-26 19:54:21,327 - lava_callback - INFO - Starting async job submit for node 68d6cfe33c94e462c6881d8e 2025-09-26 19:54:21,368 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:54:21,400 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6cfe33c94e462c6881d8e HTTP/1.1" 200 None 2025-09-26 19:54:21,401 - lava_callback - DEBUG - Processing node 68d6cfe33c94e462c6881d8e: baseline-arm64 2025-09-26 19:54:22,405 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 19:54:22,548 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 19:54:22,549 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6cfe33c94e462c6881d8e/log.txt.gz 2025-09-26 19:54:22,570 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 19:54:22,739 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 19:54:22,740 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6cfe33c94e462c6881d8e/lava_callback.json.gz DEBUG: Setup failed for 68d6cfe33c94e462c6881d8e. Transitting job node result: incomplete -> fail 2025-09-26 19:54:22,822 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:54:22,839 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6cfe33c94e462c6881d8e HTTP/1.1" 200 None 2025-09-26 19:54:22,840 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:54:22,852 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d651ddcca86fb17472aa24 HTTP/1.1" 200 None 2025-09-26 19:54:22,853 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:54:22,880 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6cfe33c94e462c6881d8e HTTP/1.1" 200 None 2025-09-26 19:54:22,881 - lava_callback - INFO - Completed processing callback for node 68d6cfe33c94e462c6881d8e INFO: 172.21.0.1:58102 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40704 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40706 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46066 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46068 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33928 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50696 - "GET / HTTP/1.0" 200 OK 2025-09-26 19:55:13,450 - lava_callback - INFO - Received callback for node 68d4daeba5c2c05277bd0b8a INFO: 172.21.0.1:54076 - "POST /node/68d4daeba5c2c05277bd0b8a HTTP/1.0" 202 Accepted 2025-09-26 19:55:13,456 - lava_callback - INFO - Starting async job submit for node 68d4daeba5c2c05277bd0b8a 2025-09-26 19:55:13,477 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:55:13,510 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4daeba5c2c05277bd0b8a HTTP/1.1" 200 None 2025-09-26 19:55:13,510 - lava_callback - DEBUG - Processing node 68d4daeba5c2c05277bd0b8a: baseline-arm 2025-09-26 19:55:13,760 - lava_callback - ERROR - Error processing callback for node 68d4daeba5c2c05277bd0b8a: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:54092 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54102 - "GET / HTTP/1.0" 200 OK 2025-09-26 19:55:16,833 - lava_callback - INFO - Received callback for node 68d4555aa5c2c05277bca97c INFO: 172.21.0.1:54112 - "POST /node/68d4555aa5c2c05277bca97c HTTP/1.0" 202 Accepted 2025-09-26 19:55:16,840 - lava_callback - INFO - Starting async job submit for node 68d4555aa5c2c05277bca97c 2025-09-26 19:55:16,948 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:55:16,969 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4555aa5c2c05277bca97c HTTP/1.1" 200 None 2025-09-26 19:55:16,969 - lava_callback - DEBUG - Processing node 68d4555aa5c2c05277bca97c: ltp-fs-bind 2025-09-26 19:55:17,505 - lava_callback - ERROR - Error processing callback for node 68d4555aa5c2c05277bca97c: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:36592 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56630 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56636 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50568 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50574 - "GET / HTTP/1.0" 200 OK 2025-09-26 19:55:54,698 - lava_callback - INFO - Received callback for node 68d6d47c3c94e462c6881dca INFO: 172.21.0.1:33948 - "POST /node/68d6d47c3c94e462c6881dca HTTP/1.0" 202 Accepted 2025-09-26 19:55:54,705 - lava_callback - INFO - Starting async job submit for node 68d6d47c3c94e462c6881dca 2025-09-26 19:55:54,725 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:55:54,737 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6d47c3c94e462c6881dca HTTP/1.1" 200 None 2025-09-26 19:55:54,738 - lava_callback - DEBUG - Processing node 68d6d47c3c94e462c6881dca: baseline-arm64 2025-09-26 19:55:55,152 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 19:55:55,303 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 19:55:55,304 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6d47c3c94e462c6881dca/log.txt.gz 2025-09-26 19:55:55,313 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 19:55:55,446 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 19:55:55,447 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6d47c3c94e462c6881dca/lava_callback.json.gz 2025-09-26 19:55:55,468 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:55:55,480 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6d47c3c94e462c6881dca HTTP/1.1" 200 None 2025-09-26 19:55:55,481 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:55:55,515 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6bd8e3c94e462c68814f1 HTTP/1.1" 200 None 2025-09-26 19:55:55,517 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:55:55,548 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6d47c3c94e462c6881dca HTTP/1.1" 200 None 2025-09-26 19:55:55,549 - lava_callback - INFO - Completed processing callback for node 68d6d47c3c94e462c6881dca INFO: 172.21.0.1:33954 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35390 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54024 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54034 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54504 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34198 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34202 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56840 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56842 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57844 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58630 - "GET / HTTP/1.0" 200 OK 2025-09-26 19:57:14,345 - lava_callback - INFO - Received callback for node 68d4db32a5c2c05277bd0bea INFO: 172.21.0.1:56832 - "POST /node/68d4db32a5c2c05277bd0bea HTTP/1.0" 202 Accepted 2025-09-26 19:57:14,351 - lava_callback - INFO - Starting async job submit for node 68d4db32a5c2c05277bd0bea 2025-09-26 19:57:14,372 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:57:14,405 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4db32a5c2c05277bd0bea HTTP/1.1" 200 None 2025-09-26 19:57:14,406 - lava_callback - DEBUG - Processing node 68d4db32a5c2c05277bd0bea: baseline-arm 2025-09-26 19:57:14,627 - lava_callback - ERROR - Error processing callback for node 68d4db32a5c2c05277bd0bea: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:56844 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56846 - "GET / HTTP/1.0" 200 OK 2025-09-26 19:57:20,310 - lava_callback - INFO - Received callback for node 68d6d4803c94e462c6881de5 INFO: 172.21.0.1:56370 - "POST /node/68d6d4803c94e462c6881de5 HTTP/1.0" 202 Accepted 2025-09-26 19:57:20,316 - lava_callback - INFO - Starting async job submit for node 68d6d4803c94e462c6881de5 2025-09-26 19:57:20,337 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:57:20,350 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6d4803c94e462c6881de5 HTTP/1.1" 200 None 2025-09-26 19:57:20,351 - lava_callback - DEBUG - Processing node 68d6d4803c94e462c6881de5: baseline-arm64 2025-09-26 19:57:20,781 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 19:57:20,925 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 19:57:20,926 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6d4803c94e462c6881de5/log.txt.gz 2025-09-26 19:57:20,938 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 19:57:21,055 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 19:57:21,055 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6d4803c94e462c6881de5/lava_callback.json.gz 2025-09-26 19:57:21,076 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:57:21,089 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6d4803c94e462c6881de5 HTTP/1.1" 200 None 2025-09-26 19:57:21,090 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:57:21,101 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6bd8e3c94e462c68814f1 HTTP/1.1" 200 None 2025-09-26 19:57:21,102 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:57:21,163 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6d4803c94e462c6881de5 HTTP/1.1" 200 None 2025-09-26 19:57:21,164 - lava_callback - INFO - Completed processing callback for node 68d6d4803c94e462c6881de5 INFO: 172.21.0.1:56386 - "GET / HTTP/1.0" 200 OK 2025-09-26 19:57:29,495 - lava_callback - INFO - Received callback for node 68d6cfe43c94e462c6881d91 INFO: 172.21.0.1:35896 - "POST /node/68d6cfe43c94e462c6881d91 HTTP/1.0" 202 Accepted 2025-09-26 19:57:29,502 - lava_callback - INFO - Starting async job submit for node 68d6cfe43c94e462c6881d91 2025-09-26 19:57:29,543 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:57:29,555 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6cfe43c94e462c6881d91 HTTP/1.1" 200 None 2025-09-26 19:57:29,555 - lava_callback - DEBUG - Processing node 68d6cfe43c94e462c6881d91: baseline-arm64 2025-09-26 19:57:30,607 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 19:57:30,745 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 19:57:30,745 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6cfe43c94e462c6881d91/log.txt.gz 2025-09-26 19:57:30,771 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 19:57:30,939 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 19:57:30,940 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6cfe43c94e462c6881d91/lava_callback.json.gz DEBUG: Setup failed for 68d6cfe43c94e462c6881d91. Transitting job node result: incomplete -> fail 2025-09-26 19:57:31,020 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:57:31,034 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6cfe43c94e462c6881d91 HTTP/1.1" 200 None 2025-09-26 19:57:31,035 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:57:31,077 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d651ddcca86fb17472aa24 HTTP/1.1" 200 None 2025-09-26 19:57:31,078 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:57:31,118 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6cfe43c94e462c6881d91 HTTP/1.1" 200 None 2025-09-26 19:57:31,119 - lava_callback - INFO - Completed processing callback for node 68d6cfe43c94e462c6881d91 INFO: 172.21.0.1:35898 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35914 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37330 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37340 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41346 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34094 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48366 - "GET /api/metrics HTTP/1.0" 200 OK 2025-09-26 19:58:16,038 - lava_callback - INFO - Received callback for node 68d44d45a5c2c05277bca664 INFO: 172.21.0.1:48378 - "POST /node/68d44d45a5c2c05277bca664 HTTP/1.0" 202 Accepted 2025-09-26 19:58:16,043 - lava_callback - INFO - Starting async job submit for node 68d44d45a5c2c05277bca664 2025-09-26 19:58:16,044 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:58:16,057 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d44d45a5c2c05277bca664 HTTP/1.1" 200 None 2025-09-26 19:58:16,057 - lava_callback - DEBUG - Processing node 68d44d45a5c2c05277bca664: baseline-arm 2025-09-26 19:58:16,057 - lava_callback - ERROR - Error processing callback for node 68d44d45a5c2c05277bca664: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:48386 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49530 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47306 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47316 - "GET / HTTP/1.0" 200 OK 2025-09-26 19:58:37,008 - lava_callback - INFO - Received callback for node 68d4555da5c2c05277bca97f INFO: 172.21.0.1:47332 - "POST /node/68d4555da5c2c05277bca97f HTTP/1.0" 202 Accepted 2025-09-26 19:58:37,014 - lava_callback - INFO - Starting async job submit for node 68d4555da5c2c05277bca97f 2025-09-26 19:58:37,040 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:58:37,074 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4555da5c2c05277bca97f HTTP/1.1" 200 None 2025-09-26 19:58:37,074 - lava_callback - DEBUG - Processing node 68d4555da5c2c05277bca97f: ltp-fsx 2025-09-26 19:58:37,422 - lava_callback - ERROR - Error processing callback for node 68d4555da5c2c05277bca97f: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:39460 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39472 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36934 - "GET / HTTP/1.0" 200 OK 2025-09-26 19:58:59,712 - lava_callback - INFO - Received callback for node 68d6e6a13c94e462c688225d INFO: 172.21.0.1:40700 - "POST /node/68d6e6a13c94e462c688225d HTTP/1.0" 202 Accepted 2025-09-26 19:58:59,718 - lava_callback - INFO - Starting async job submit for node 68d6e6a13c94e462c688225d 2025-09-26 19:58:59,739 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:58:59,751 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6e6a13c94e462c688225d HTTP/1.1" 200 None 2025-09-26 19:58:59,752 - lava_callback - DEBUG - Processing node 68d6e6a13c94e462c688225d: baseline-arm64 2025-09-26 19:59:00,153 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 19:59:00,313 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 19:59:00,314 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6e6a13c94e462c688225d/log.txt.gz 2025-09-26 19:59:00,323 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 19:59:00,459 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 19:59:00,460 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6e6a13c94e462c688225d/lava_callback.json.gz 2025-09-26 19:59:00,481 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:59:00,493 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6e6a13c94e462c688225d HTTP/1.1" 200 None 2025-09-26 19:59:00,494 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:59:00,507 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d648ed5d233c363a762351 HTTP/1.1" 200 None 2025-09-26 19:59:00,508 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:59:00,539 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6e6a13c94e462c688225d HTTP/1.1" 200 None 2025-09-26 19:59:00,540 - lava_callback - INFO - Completed processing callback for node 68d6e6a13c94e462c688225d INFO: 172.21.0.1:40716 - "GET / HTTP/1.0" 200 OK 2025-09-26 19:59:14,539 - lava_callback - INFO - Received callback for node 68d4db33a5c2c05277bd0bed INFO: 172.21.0.1:39638 - "POST /node/68d4db33a5c2c05277bd0bed HTTP/1.0" 202 Accepted 2025-09-26 19:59:14,545 - lava_callback - INFO - Starting async job submit for node 68d4db33a5c2c05277bd0bed 2025-09-26 19:59:14,565 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:59:14,578 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4db33a5c2c05277bd0bed HTTP/1.1" 200 None 2025-09-26 19:59:14,578 - lava_callback - DEBUG - Processing node 68d4db33a5c2c05277bd0bed: baseline-arm 2025-09-26 19:59:14,829 - lava_callback - ERROR - Error processing callback for node 68d4db33a5c2c05277bd0bed: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:39646 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39648 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35720 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37250 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37254 - "GET / HTTP/1.0" 200 OK 2025-09-26 19:59:37,082 - lava_callback - INFO - Received callback for node 68d4555fa5c2c05277bca982 INFO: 172.21.0.1:37268 - "POST /node/68d4555fa5c2c05277bca982 HTTP/1.0" 202 Accepted 2025-09-26 19:59:37,089 - lava_callback - INFO - Starting async job submit for node 68d4555fa5c2c05277bca982 2025-09-26 19:59:37,120 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:59:37,132 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4555fa5c2c05277bca982 HTTP/1.1" 200 None 2025-09-26 19:59:37,132 - lava_callback - DEBUG - Processing node 68d4555fa5c2c05277bca982: ltp-input 2025-09-26 19:59:37,508 - lava_callback - ERROR - Error processing callback for node 68d4555fa5c2c05277bca982: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None 2025-09-26 19:59:41,152 - lava_callback - INFO - Received callback for node 68d470a1a5c2c05277bcc6c9 INFO: 172.21.0.1:38490 - "POST /node/68d470a1a5c2c05277bcc6c9 HTTP/1.0" 202 Accepted 2025-09-26 19:59:41,160 - lava_callback - INFO - Starting async job submit for node 68d470a1a5c2c05277bcc6c9 2025-09-26 19:59:41,200 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:59:41,212 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d470a1a5c2c05277bcc6c9 HTTP/1.1" 200 None 2025-09-26 19:59:41,213 - lava_callback - DEBUG - Processing node 68d470a1a5c2c05277bcc6c9: baseline-arm64 2025-09-26 19:59:42,084 - lava_callback - ERROR - Error processing callback for node 68d470a1a5c2c05277bcc6c9: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None 2025-09-26 19:59:42,132 - lava_callback - INFO - Received callback for node 68d6e6a03c94e462c688225a INFO: 172.21.0.1:38492 - "POST /node/68d6e6a03c94e462c688225a HTTP/1.0" 202 Accepted 2025-09-26 19:59:42,138 - lava_callback - INFO - Starting async job submit for node 68d6e6a03c94e462c688225a 2025-09-26 19:59:42,160 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:59:42,173 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6e6a03c94e462c688225a HTTP/1.1" 200 None 2025-09-26 19:59:42,173 - lava_callback - DEBUG - Processing node 68d6e6a03c94e462c688225a: baseline-arm64 2025-09-26 19:59:42,613 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 19:59:42,637 - lava_callback - INFO - Received callback for node 68d6cfe63c94e462c6881d94 INFO: 172.21.0.1:38506 - "POST /node/68d6cfe63c94e462c6881d94 HTTP/1.0" 202 Accepted 2025-09-26 19:59:42,644 - lava_callback - INFO - Starting async job submit for node 68d6cfe63c94e462c6881d94 2025-09-26 19:59:42,686 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:59:42,710 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6cfe63c94e462c6881d94 HTTP/1.1" 200 None 2025-09-26 19:59:42,711 - lava_callback - DEBUG - Processing node 68d6cfe63c94e462c6881d94: baseline-arm64 2025-09-26 19:59:42,810 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 19:59:42,851 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6e6a03c94e462c688225a/log.txt.gz 2025-09-26 19:59:42,929 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 19:59:43,153 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 19:59:43,215 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6e6a03c94e462c688225a/lava_callback.json.gz 2025-09-26 19:59:43,304 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:59:43,398 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6e6a03c94e462c688225a HTTP/1.1" 200 None 2025-09-26 19:59:43,415 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:59:43,481 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d648ed5d233c363a762351 HTTP/1.1" 200 None 2025-09-26 19:59:43,492 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:59:43,585 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6e6a03c94e462c688225a HTTP/1.1" 200 None 2025-09-26 19:59:43,586 - lava_callback - INFO - Completed processing callback for node 68d6e6a03c94e462c688225a 2025-09-26 19:59:43,868 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 19:59:43,985 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 19:59:43,986 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6cfe63c94e462c6881d94/log.txt.gz 2025-09-26 19:59:44,006 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 19:59:44,158 - lava_callback - INFO - Received callback for node 68d4709fa5c2c05277bcc6c6 INFO: 172.21.0.1:38516 - "POST /node/68d4709fa5c2c05277bcc6c6 HTTP/1.0" 202 Accepted 2025-09-26 19:59:44,167 - lava_callback - INFO - Starting async job submit for node 68d4709fa5c2c05277bcc6c6 2025-09-26 19:59:44,204 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 19:59:44,211 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:59:44,212 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6cfe63c94e462c6881d94/lava_callback.json.gz 2025-09-26 19:59:44,293 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4709fa5c2c05277bcc6c6 HTTP/1.1" 200 None DEBUG: Setup failed for 68d6cfe63c94e462c6881d94. Transitting job node result: incomplete -> fail 2025-09-26 19:59:44,299 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:59:44,299 - lava_callback - DEBUG - Processing node 68d4709fa5c2c05277bcc6c6: baseline-arm64 2025-09-26 19:59:44,371 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6cfe63c94e462c6881d94 HTTP/1.1" 200 None 2025-09-26 19:59:44,393 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:59:44,465 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d651ddcca86fb17472aa24 HTTP/1.1" 200 None 2025-09-26 19:59:44,486 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:59:44,578 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6cfe63c94e462c6881d94 HTTP/1.1" 200 None 2025-09-26 19:59:44,578 - lava_callback - INFO - Completed processing callback for node 68d6cfe63c94e462c6881d94 2025-09-26 19:59:45,353 - lava_callback - ERROR - Error processing callback for node 68d4709fa5c2c05277bcc6c6: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:38530 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38542 - "GET / HTTP/1.0" 200 OK 2025-09-26 19:59:48,950 - lava_callback - INFO - Received callback for node 68d470a5a5c2c05277bcc6cc INFO: 172.21.0.1:38558 - "POST /node/68d470a5a5c2c05277bcc6cc HTTP/1.0" 202 Accepted 2025-09-26 19:59:48,957 - lava_callback - INFO - Starting async job submit for node 68d470a5a5c2c05277bcc6cc 2025-09-26 19:59:48,997 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 19:59:49,010 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d470a5a5c2c05277bcc6cc HTTP/1.1" 200 None 2025-09-26 19:59:49,010 - lava_callback - DEBUG - Processing node 68d470a5a5c2c05277bcc6cc: baseline-arm64 2025-09-26 19:59:49,872 - lava_callback - ERROR - Error processing callback for node 68d470a5a5c2c05277bcc6cc: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:42376 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34850 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60558 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60572 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42474 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40824 - "GET / HTTP/1.0" 200 OK 2025-09-26 20:00:34,837 - lava_callback - INFO - Received callback for node 68d6e6fb3c94e462c6882268 INFO: 172.21.0.1:40834 - "POST /node/68d6e6fb3c94e462c6882268 HTTP/1.0" 202 Accepted 2025-09-26 20:00:34,843 - lava_callback - INFO - Starting async job submit for node 68d6e6fb3c94e462c6882268 2025-09-26 20:00:34,864 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 20:00:34,884 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6e6fb3c94e462c6882268 HTTP/1.1" 200 None 2025-09-26 20:00:34,885 - lava_callback - DEBUG - Processing node 68d6e6fb3c94e462c6882268: baseline-arm64 2025-09-26 20:00:35,296 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 20:00:35,548 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 20:00:35,549 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6e6fb3c94e462c6882268/log.txt.gz 2025-09-26 20:00:35,558 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 20:00:35,676 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 20:00:35,677 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6e6fb3c94e462c6882268/lava_callback.json.gz 2025-09-26 20:00:35,698 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 20:00:35,714 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6e6fb3c94e462c6882268 HTTP/1.1" 200 None 2025-09-26 20:00:35,715 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 20:00:35,727 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d648ed5d233c363a762351 HTTP/1.1" 200 None 2025-09-26 20:00:35,728 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 20:00:35,755 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6e6fb3c94e462c6882268 HTTP/1.1" 200 None 2025-09-26 20:00:35,756 - lava_callback - INFO - Completed processing callback for node 68d6e6fb3c94e462c6882268 INFO: 172.21.0.1:40838 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60830 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60834 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40094 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59306 - "GET / HTTP/1.0" 200 OK 2025-09-26 20:01:07,644 - lava_callback - INFO - Received callback for node 68d6e6fb3c94e462c688226b INFO: 172.21.0.1:59322 - "POST /node/68d6e6fb3c94e462c688226b HTTP/1.0" 202 Accepted 2025-09-26 20:01:07,651 - lava_callback - INFO - Starting async job submit for node 68d6e6fb3c94e462c688226b 2025-09-26 20:01:07,672 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 20:01:07,683 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6e6fb3c94e462c688226b HTTP/1.1" 200 None 2025-09-26 20:01:07,684 - lava_callback - DEBUG - Processing node 68d6e6fb3c94e462c688226b: baseline-arm64 2025-09-26 20:01:08,085 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 20:01:08,216 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 20:01:08,217 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6e6fb3c94e462c688226b/log.txt.gz 2025-09-26 20:01:08,226 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 20:01:08,383 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 20:01:08,383 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6e6fb3c94e462c688226b/lava_callback.json.gz 2025-09-26 20:01:08,405 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 20:01:08,422 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6e6fb3c94e462c688226b HTTP/1.1" 200 None 2025-09-26 20:01:08,423 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 20:01:08,440 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d648ed5d233c363a762351 HTTP/1.1" 200 None 2025-09-26 20:01:08,441 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 20:01:08,470 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6e6fb3c94e462c688226b HTTP/1.1" 200 None 2025-09-26 20:01:08,471 - lava_callback - INFO - Completed processing callback for node 68d6e6fb3c94e462c688226b 2025-09-26 20:01:13,061 - lava_callback - INFO - Received callback for node 68d4db34a5c2c05277bd0bf0 INFO: 172.21.0.1:51036 - "POST /node/68d4db34a5c2c05277bd0bf0 HTTP/1.0" 202 Accepted 2025-09-26 20:01:13,067 - lava_callback - INFO - Starting async job submit for node 68d4db34a5c2c05277bd0bf0 2025-09-26 20:01:13,088 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 20:01:13,100 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4db34a5c2c05277bd0bf0 HTTP/1.1" 200 None 2025-09-26 20:01:13,100 - lava_callback - DEBUG - Processing node 68d4db34a5c2c05277bd0bf0: baseline-arm 2025-09-26 20:01:13,341 - lava_callback - ERROR - Error processing callback for node 68d4db34a5c2c05277bd0bf0: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:51050 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51066 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55510 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34094 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34106 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42106 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42108 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39414 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58128 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53142 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53146 - "GET / HTTP/1.0" 200 OK 2025-09-26 20:02:16,616 - lava_callback - INFO - Received callback for node 68d44d43a5c2c05277bca661 INFO: 172.21.0.1:53150 - "POST /node/68d44d43a5c2c05277bca661 HTTP/1.0" 202 Accepted 2025-09-26 20:02:16,622 - lava_callback - INFO - Starting async job submit for node 68d44d43a5c2c05277bca661 2025-09-26 20:02:16,652 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 20:02:16,688 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d44d43a5c2c05277bca661 HTTP/1.1" 200 None 2025-09-26 20:02:16,688 - lava_callback - DEBUG - Processing node 68d44d43a5c2c05277bca661: baseline-arm 2025-09-26 20:02:16,950 - lava_callback - ERROR - Error processing callback for node 68d44d43a5c2c05277bca661: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:57560 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45032 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45036 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60664 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60676 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33896 - "GET / HTTP/1.0" 200 OK 2025-09-26 20:02:57,168 - lava_callback - INFO - Received callback for node 68d45561a5c2c05277bca985 INFO: 172.21.0.1:33906 - "POST /node/68d45561a5c2c05277bca985 HTTP/1.0" 202 Accepted 2025-09-26 20:02:57,175 - lava_callback - INFO - Starting async job submit for node 68d45561a5c2c05277bca985 2025-09-26 20:02:57,200 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 20:02:57,241 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d45561a5c2c05277bca985 HTTP/1.1" 200 None 2025-09-26 20:02:57,241 - lava_callback - DEBUG - Processing node 68d45561a5c2c05277bca985: ltp-ipc 2025-09-26 20:02:57,589 - lava_callback - ERROR - Error processing callback for node 68d45561a5c2c05277bca985: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:49426 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54116 - "GET /api/metrics HTTP/1.0" 200 OK 2025-09-26 20:03:15,224 - lava_callback - INFO - Received callback for node 68d4db35a5c2c05277bd0bf3 INFO: 172.21.0.1:54118 - "POST /node/68d4db35a5c2c05277bd0bf3 HTTP/1.0" 202 Accepted 2025-09-26 20:03:15,231 - lava_callback - INFO - Starting async job submit for node 68d4db35a5c2c05277bd0bf3 2025-09-26 20:03:15,252 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 20:03:15,263 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4db35a5c2c05277bd0bf3 HTTP/1.1" 200 None 2025-09-26 20:03:15,263 - lava_callback - DEBUG - Processing node 68d4db35a5c2c05277bd0bf3: baseline-arm 2025-09-26 20:03:15,499 - lava_callback - ERROR - Error processing callback for node 68d4db35a5c2c05277bd0bf3: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:54130 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55666 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56250 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56258 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35124 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35126 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49932 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56916 - "GET / HTTP/1.0" 200 OK 2025-09-26 20:04:09,428 - lava_callback - INFO - Received callback for node 68d6cfe73c94e462c6881d97 INFO: 172.21.0.1:40422 - "POST /node/68d6cfe73c94e462c6881d97 HTTP/1.0" 202 Accepted 2025-09-26 20:04:09,434 - lava_callback - INFO - Starting async job submit for node 68d6cfe73c94e462c6881d97 2025-09-26 20:04:09,468 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 20:04:09,480 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6cfe73c94e462c6881d97 HTTP/1.1" 200 None 2025-09-26 20:04:09,481 - lava_callback - DEBUG - Processing node 68d6cfe73c94e462c6881d97: baseline-arm64 2025-09-26 20:04:10,043 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 20:04:10,197 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 20:04:10,198 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6cfe73c94e462c6881d97/log.txt.gz 2025-09-26 20:04:10,210 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 20:04:10,346 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 20:04:10,346 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6cfe73c94e462c6881d97/lava_callback.json.gz DEBUG: Setup failed for 68d6cfe73c94e462c6881d97. Transitting job node result: incomplete -> fail 2025-09-26 20:04:10,413 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 20:04:10,455 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6cfe73c94e462c6881d97 HTTP/1.1" 200 None 2025-09-26 20:04:10,456 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 20:04:10,467 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d651ddcca86fb17472aa24 HTTP/1.1" 200 None 2025-09-26 20:04:10,468 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 20:04:10,504 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6cfe73c94e462c6881d97 HTTP/1.1" 200 None 2025-09-26 20:04:10,505 - lava_callback - INFO - Completed processing callback for node 68d6cfe73c94e462c6881d97 INFO: 172.21.0.1:40426 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40430 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54012 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43942 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43944 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47002 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47008 - "GET / HTTP/1.0" 200 OK 2025-09-26 20:04:50,366 - lava_callback - INFO - Received callback for node 68d6edbe3c94e462c68823e6 INFO: 172.21.0.1:55610 - "POST /node/68d6edbe3c94e462c68823e6 HTTP/1.0" 202 Accepted 2025-09-26 20:04:50,373 - lava_callback - INFO - Starting async job submit for node 68d6edbe3c94e462c68823e6 2025-09-26 20:04:50,430 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 20:04:50,442 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6edbe3c94e462c68823e6 HTTP/1.1" 200 None 2025-09-26 20:04:50,443 - lava_callback - DEBUG - Processing node 68d6edbe3c94e462c68823e6: ltp-smoketest 2025-09-26 20:04:50,913 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 20:04:51,051 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 20:04:51,052 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoketest-68d6edbe3c94e462c68823e6/log.txt.gz 2025-09-26 20:04:51,063 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 20:04:51,197 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 20:04:51,198 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoketest-68d6edbe3c94e462c68823e6/lava_callback.json.gz 2025-09-26 20:04:51,313 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 20:04:51,331 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6edbe3c94e462c68823e6 HTTP/1.1" 200 None 2025-09-26 20:04:51,331 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 20:04:51,343 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6bd813c94e462c68814e8 HTTP/1.1" 200 None 2025-09-26 20:04:51,344 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 20:04:51,372 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6edbe3c94e462c68823e6 HTTP/1.1" 200 None 2025-09-26 20:04:51,373 - lava_callback - INFO - Completed processing callback for node 68d6edbe3c94e462c68823e6 INFO: 172.21.0.1:55612 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44444 - "GET / HTTP/1.0" 200 OK 2025-09-26 20:05:13,488 - lava_callback - INFO - Received callback for node 68d4db37a5c2c05277bd0bf6 INFO: 172.21.0.1:37954 - "POST /node/68d4db37a5c2c05277bd0bf6 HTTP/1.0" 202 Accepted 2025-09-26 20:05:13,494 - lava_callback - INFO - Starting async job submit for node 68d4db37a5c2c05277bd0bf6 2025-09-26 20:05:13,515 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 20:05:13,546 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4db37a5c2c05277bd0bf6 HTTP/1.1" 200 None 2025-09-26 20:05:13,546 - lava_callback - DEBUG - Processing node 68d4db37a5c2c05277bd0bf6: baseline-arm 2025-09-26 20:05:13,787 - lava_callback - ERROR - Error processing callback for node 68d4db37a5c2c05277bd0bf6: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:37964 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37978 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54586 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38964 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38980 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34996 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35010 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35368 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49004 - "GET / HTTP/1.0" 200 OK 2025-09-26 20:06:09,243 - lava_callback - INFO - Received callback for node 68d45555a5c2c05277bca976 INFO: 172.21.0.1:49018 - "POST /node/68d45555a5c2c05277bca976 HTTP/1.0" 202 Accepted 2025-09-26 20:06:09,250 - lava_callback - INFO - Starting async job submit for node 68d45555a5c2c05277bca976 2025-09-26 20:06:09,356 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 20:06:09,369 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d45555a5c2c05277bca976 HTTP/1.1" 200 None 2025-09-26 20:06:09,369 - lava_callback - DEBUG - Processing node 68d45555a5c2c05277bca976: ltp-cve 2025-09-26 20:06:10,102 - lava_callback - ERROR - Error processing callback for node 68d45555a5c2c05277bca976: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:54046 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54050 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32924 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32814 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32818 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56000 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56002 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47410 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43264 - "GET / HTTP/1.0" 200 OK 2025-09-26 20:07:13,907 - lava_callback - INFO - Received callback for node 68d4db37a5c2c05277bd0bf9 INFO: 172.21.0.1:42132 - "POST /node/68d4db37a5c2c05277bd0bf9 HTTP/1.0" 202 Accepted 2025-09-26 20:07:13,913 - lava_callback - INFO - Starting async job submit for node 68d4db37a5c2c05277bd0bf9 2025-09-26 20:07:13,933 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 20:07:13,955 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4db37a5c2c05277bd0bf9 HTTP/1.1" 200 None 2025-09-26 20:07:13,955 - lava_callback - DEBUG - Processing node 68d4db37a5c2c05277bd0bf9: baseline-arm 2025-09-26 20:07:14,181 - lava_callback - ERROR - Error processing callback for node 68d4db37a5c2c05277bd0bf9: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:42148 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42156 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56846 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34414 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34420 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42444 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42454 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43948 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46558 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46254 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46256 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56196 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36762 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36768 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57308 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57320 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40618 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55174 - "GET / HTTP/1.0" 200 OK 2025-09-26 20:09:13,042 - lava_callback - INFO - Received callback for node 68d4db39a5c2c05277bd0bfc INFO: 172.21.0.1:38338 - "POST /node/68d4db39a5c2c05277bd0bfc HTTP/1.0" 202 Accepted 2025-09-26 20:09:13,048 - lava_callback - INFO - Starting async job submit for node 68d4db39a5c2c05277bd0bfc 2025-09-26 20:09:13,069 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 20:09:13,081 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4db39a5c2c05277bd0bfc HTTP/1.1" 200 None 2025-09-26 20:09:13,081 - lava_callback - DEBUG - Processing node 68d4db39a5c2c05277bd0bfc: baseline-arm 2025-09-26 20:09:13,307 - lava_callback - ERROR - Error processing callback for node 68d4db39a5c2c05277bd0bfc: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:38352 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38368 - "GET / HTTP/1.0" 200 OK 2025-09-26 20:09:17,427 - lava_callback - INFO - Received callback for node 68d45565a5c2c05277bca988 INFO: 172.21.0.1:38380 - "POST /node/68d45565a5c2c05277bca988 HTTP/1.0" 202 Accepted 2025-09-26 20:09:17,434 - lava_callback - INFO - Starting async job submit for node 68d45565a5c2c05277bca988 2025-09-26 20:09:17,474 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 20:09:17,495 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d45565a5c2c05277bca988 HTTP/1.1" 200 None 2025-09-26 20:09:17,496 - lava_callback - DEBUG - Processing node 68d45565a5c2c05277bca988: ltp-pty 2025-09-26 20:09:17,888 - lava_callback - ERROR - Error processing callback for node 68d45565a5c2c05277bca988: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:52648 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55496 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55512 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59176 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59182 - "GET / HTTP/1.0" 200 OK 2025-09-26 20:09:49,266 - lava_callback - INFO - Received callback for node 68d6d4883c94e462c6881e1b INFO: 172.21.0.1:59194 - "POST /node/68d6d4883c94e462c6881e1b HTTP/1.0" 202 Accepted 2025-09-26 20:09:49,272 - lava_callback - INFO - Starting async job submit for node 68d6d4883c94e462c6881e1b 2025-09-26 20:09:49,294 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 20:09:49,314 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6d4883c94e462c6881e1b HTTP/1.1" 200 None 2025-09-26 20:09:49,314 - lava_callback - DEBUG - Processing node 68d6d4883c94e462c6881e1b: baseline-arm64-broonie 2025-09-26 20:09:49,529 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 20:09:49,687 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 20:09:49,688 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-68d6d4883c94e462c6881e1b/log.txt.gz 2025-09-26 20:09:49,694 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 20:09:49,821 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 20:09:49,822 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-68d6d4883c94e462c6881e1b/lava_callback.json.gz 2025-09-26 20:09:49,844 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 20:09:49,856 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6d4883c94e462c6881e1b HTTP/1.1" 200 None 2025-09-26 20:09:49,857 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 20:09:49,900 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6bd8e3c94e462c68814f1 HTTP/1.1" 200 None 2025-09-26 20:09:49,901 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 20:09:49,929 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6d4883c94e462c6881e1b HTTP/1.1" 200 None 2025-09-26 20:09:49,932 - lava_callback - INFO - Completed processing callback for node 68d6d4883c94e462c6881e1b INFO: 172.21.0.1:44220 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47012 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36570 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36584 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58532 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33468 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33482 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38152 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38158 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54818 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50678 - "GET / HTTP/1.0" 200 OK 2025-09-26 20:11:13,572 - lava_callback - INFO - Received callback for node 68d4db39a5c2c05277bd0bff INFO: 172.21.0.1:36388 - "POST /node/68d4db39a5c2c05277bd0bff HTTP/1.0" 202 Accepted 2025-09-26 20:11:13,581 - lava_callback - INFO - Starting async job submit for node 68d4db39a5c2c05277bd0bff 2025-09-26 20:11:13,608 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 20:11:13,651 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4db39a5c2c05277bd0bff HTTP/1.1" 200 None 2025-09-26 20:11:13,652 - lava_callback - DEBUG - Processing node 68d4db39a5c2c05277bd0bff: baseline-arm 2025-09-26 20:11:13,893 - lava_callback - ERROR - Error processing callback for node 68d4db39a5c2c05277bd0bff: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:36394 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36398 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46578 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39050 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39056 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41430 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49616 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49618 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34394 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40240 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40244 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52022 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55876 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55880 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43952 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45032 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45046 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55012 - "GET / HTTP/1.0" 200 OK 2025-09-26 20:13:10,832 - lava_callback - INFO - Received callback for node 68d4e056a5c2c05277bd0d02 INFO: 172.21.0.1:50526 - "POST /node/68d4e056a5c2c05277bd0d02 HTTP/1.0" 202 Accepted 2025-09-26 20:13:10,838 - lava_callback - INFO - Starting async job submit for node 68d4e056a5c2c05277bd0d02 2025-09-26 20:13:10,858 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 20:13:10,895 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4e056a5c2c05277bd0d02 HTTP/1.1" 200 None 2025-09-26 20:13:10,895 - lava_callback - DEBUG - Processing node 68d4e056a5c2c05277bd0d02: baseline-arm 2025-09-26 20:13:11,125 - lava_callback - ERROR - Error processing callback for node 68d4e056a5c2c05277bd0d02: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:50534 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50548 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38718 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55886 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55888 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39304 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56604 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56616 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45030 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54492 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54500 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35988 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54624 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54632 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38036 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43762 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43776 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35530 - "GET / HTTP/1.0" 200 OK 2025-09-26 20:15:14,133 - lava_callback - INFO - Received callback for node 68d4e056a5c2c05277bd0d05 INFO: 172.21.0.1:54180 - "POST /node/68d4e056a5c2c05277bd0d05 HTTP/1.0" 202 Accepted 2025-09-26 20:15:14,139 - lava_callback - INFO - Starting async job submit for node 68d4e056a5c2c05277bd0d05 2025-09-26 20:15:14,159 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 20:15:14,173 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4e056a5c2c05277bd0d05 HTTP/1.1" 200 None 2025-09-26 20:15:14,174 - lava_callback - DEBUG - Processing node 68d4e056a5c2c05277bd0d05: baseline-arm 2025-09-26 20:15:14,403 - lava_callback - ERROR - Error processing callback for node 68d4e056a5c2c05277bd0d05: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:54192 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54196 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45636 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56572 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56584 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35722 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46636 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46642 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36106 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53220 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53234 - "GET / HTTP/1.0" 200 OK 2025-09-26 20:16:17,261 - lava_callback - INFO - Received callback for node 68d4556fa5c2c05277bca98e INFO: 172.21.0.1:53246 - "POST /node/68d4556fa5c2c05277bca98e HTTP/1.0" 202 Accepted 2025-09-26 20:16:17,268 - lava_callback - INFO - Starting async job submit for node 68d4556fa5c2c05277bca98e 2025-09-26 20:16:17,304 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 20:16:17,346 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4556fa5c2c05277bca98e HTTP/1.1" 200 None 2025-09-26 20:16:17,347 - lava_callback - DEBUG - Processing node 68d4556fa5c2c05277bca98e: ltp-smoketest 2025-09-26 20:16:17,808 - lava_callback - ERROR - Error processing callback for node 68d4556fa5c2c05277bca98e: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:44028 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49724 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49726 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42218 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36506 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36514 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51326 - "GET / HTTP/1.0" 200 OK 2025-09-26 20:17:14,039 - lava_callback - INFO - Received callback for node 68d4e095a5c2c05277bd0d38 INFO: 172.21.0.1:57894 - "POST /node/68d4e095a5c2c05277bd0d38 HTTP/1.0" 202 Accepted 2025-09-26 20:17:14,048 - lava_callback - INFO - Starting async job submit for node 68d4e095a5c2c05277bd0d38 2025-09-26 20:17:14,071 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 20:17:14,086 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4e095a5c2c05277bd0d38 HTTP/1.1" 200 None 2025-09-26 20:17:14,086 - lava_callback - DEBUG - Processing node 68d4e095a5c2c05277bd0d38: baseline-arm 2025-09-26 20:17:14,312 - lava_callback - ERROR - Error processing callback for node 68d4e095a5c2c05277bd0d38: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:57904 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57912 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54488 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60276 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60286 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35440 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45938 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45944 - "GET / HTTP/1.0" 200 OK 2025-09-26 20:17:57,326 - lava_callback - INFO - Received callback for node 68d451e5a5c2c05277bca7d9 INFO: 172.21.0.1:45958 - "POST /node/68d451e5a5c2c05277bca7d9 HTTP/1.0" 202 Accepted 2025-09-26 20:17:57,332 - lava_callback - INFO - Starting async job submit for node 68d451e5a5c2c05277bca7d9 2025-09-26 20:17:57,333 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 20:17:57,345 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d451e5a5c2c05277bca7d9 HTTP/1.1" 200 None 2025-09-26 20:17:57,345 - lava_callback - DEBUG - Processing node 68d451e5a5c2c05277bca7d9: baseline-arm 2025-09-26 20:17:57,345 - lava_callback - ERROR - Error processing callback for node 68d451e5a5c2c05277bca7d9: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:54172 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40434 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40450 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42100 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35834 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35840 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55414 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47778 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47786 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56060 - "GET / HTTP/1.0" 200 OK 2025-09-26 20:19:14,268 - lava_callback - INFO - Received callback for node 68d4e095a5c2c05277bd0d3b INFO: 172.21.0.1:35734 - "POST /node/68d4e095a5c2c05277bd0d3b HTTP/1.0" 202 Accepted 2025-09-26 20:19:14,274 - lava_callback - INFO - Starting async job submit for node 68d4e095a5c2c05277bd0d3b 2025-09-26 20:19:14,295 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 20:19:14,315 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4e095a5c2c05277bd0d3b HTTP/1.1" 200 None 2025-09-26 20:19:14,315 - lava_callback - DEBUG - Processing node 68d4e095a5c2c05277bd0d3b: baseline-arm 2025-09-26 20:19:14,540 - lava_callback - ERROR - Error processing callback for node 68d4e095a5c2c05277bd0d3b: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:35750 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35760 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59710 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47160 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47166 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54426 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48696 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48708 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46408 - "GET / HTTP/1.0" 200 OK 2025-09-26 20:20:11,200 - lava_callback - INFO - Received callback for node 68d470dca5c2c05277bcc6dc INFO: 172.21.0.1:46602 - "POST /node/68d470dca5c2c05277bcc6dc HTTP/1.0" 202 Accepted 2025-09-26 20:20:11,212 - lava_callback - INFO - Starting async job submit for node 68d470dca5c2c05277bcc6dc 2025-09-26 20:20:11,255 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 20:20:11,276 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d470dca5c2c05277bcc6dc HTTP/1.1" 200 None 2025-09-26 20:20:11,277 - lava_callback - DEBUG - Processing node 68d470dca5c2c05277bcc6dc: baseline-arm64 2025-09-26 20:20:12,184 - lava_callback - ERROR - Error processing callback for node 68d470dca5c2c05277bcc6dc: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:46606 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46610 - "GET / HTTP/1.0" 200 OK 2025-09-26 20:20:20,387 - lava_callback - INFO - Received callback for node 68d6ea943c94e462c68823ad INFO: 172.21.0.1:35060 - "POST /node/68d6ea943c94e462c68823ad HTTP/1.0" 202 Accepted 2025-09-26 20:20:20,395 - lava_callback - INFO - Starting async job submit for node 68d6ea943c94e462c68823ad 2025-09-26 20:20:20,460 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 20:20:20,489 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6ea943c94e462c68823ad HTTP/1.1" 200 None 2025-09-26 20:20:20,490 - lava_callback - DEBUG - Processing node 68d6ea943c94e462c68823ad: baseline-arm64 2025-09-26 20:20:21,505 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 20:20:21,696 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 20:20:21,697 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6ea943c94e462c68823ad/log.txt.gz 2025-09-26 20:20:21,718 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 20:20:21,902 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 20:20:21,902 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6ea943c94e462c68823ad/lava_callback.json.gz DEBUG: Setup failed for 68d6ea943c94e462c68823ad. Transitting job node result: incomplete -> fail 2025-09-26 20:20:21,982 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 20:20:21,995 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6ea943c94e462c68823ad HTTP/1.1" 200 None 2025-09-26 20:20:21,996 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 20:20:21,997 - lava_callback - INFO - Received callback for node 68d470a8a5c2c05277bcc6cf INFO: 172.21.0.1:35074 - "POST /node/68d470a8a5c2c05277bcc6cf HTTP/1.0" 202 Accepted 2025-09-26 20:20:22,004 - lava_callback - INFO - Starting async job submit for node 68d470a8a5c2c05277bcc6cf 2025-09-26 20:20:22,047 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 20:20:22,058 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d651ddcca86fb17472aa24 HTTP/1.1" 200 None 2025-09-26 20:20:22,059 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 20:20:22,059 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d470a8a5c2c05277bcc6cf HTTP/1.1" 200 None 2025-09-26 20:20:22,060 - lava_callback - DEBUG - Processing node 68d470a8a5c2c05277bcc6cf: baseline-arm64 2025-09-26 20:20:22,162 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6ea943c94e462c68823ad HTTP/1.1" 200 None 2025-09-26 20:20:22,163 - lava_callback - INFO - Completed processing callback for node 68d6ea943c94e462c68823ad 2025-09-26 20:20:23,072 - lava_callback - ERROR - Error processing callback for node 68d470a8a5c2c05277bcc6cf: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None 2025-09-26 20:20:23,587 - lava_callback - INFO - Received callback for node 68d470dea5c2c05277bcc6df INFO: 172.21.0.1:35088 - "POST /node/68d470dea5c2c05277bcc6df HTTP/1.0" 202 Accepted 2025-09-26 20:20:23,594 - lava_callback - INFO - Starting async job submit for node 68d470dea5c2c05277bcc6df 2025-09-26 20:20:23,634 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 20:20:23,646 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d470dea5c2c05277bcc6df HTTP/1.1" 200 None 2025-09-26 20:20:23,647 - lava_callback - DEBUG - Processing node 68d470dea5c2c05277bcc6df: baseline-arm64 2025-09-26 20:20:24,607 - lava_callback - ERROR - Error processing callback for node 68d470dea5c2c05277bcc6df: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:35100 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53006 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53010 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55336 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32962 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32978 - "GET / HTTP/1.0" 200 OK 2025-09-26 20:21:01,718 - lava_callback - INFO - Received callback for node 68d6ea953c94e462c68823b0 INFO: 172.21.0.1:42760 - "POST /node/68d6ea953c94e462c68823b0 HTTP/1.0" 202 Accepted 2025-09-26 20:21:01,725 - lava_callback - INFO - Starting async job submit for node 68d6ea953c94e462c68823b0 2025-09-26 20:21:01,766 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 20:21:01,803 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6ea953c94e462c68823b0 HTTP/1.1" 200 None 2025-09-26 20:21:01,803 - lava_callback - DEBUG - Processing node 68d6ea953c94e462c68823b0: baseline-arm64 2025-09-26 20:21:02,902 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 20:21:03,036 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 20:21:03,036 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6ea953c94e462c68823b0/log.txt.gz 2025-09-26 20:21:03,058 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 20:21:03,230 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 20:21:03,231 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6ea953c94e462c68823b0/lava_callback.json.gz DEBUG: Setup failed for 68d6ea953c94e462c68823b0. Transitting job node result: incomplete -> fail 2025-09-26 20:21:03,313 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 20:21:03,335 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6ea953c94e462c68823b0 HTTP/1.1" 200 None 2025-09-26 20:21:03,336 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 20:21:03,371 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d651ddcca86fb17472aa24 HTTP/1.1" 200 None 2025-09-26 20:21:03,372 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 20:21:03,406 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6ea953c94e462c68823b0 HTTP/1.1" 200 None 2025-09-26 20:21:03,407 - lava_callback - INFO - Completed processing callback for node 68d6ea953c94e462c68823b0 INFO: 172.21.0.1:42772 - "GET / HTTP/1.0" 200 OK 2025-09-26 20:21:09,332 - lava_callback - INFO - Received callback for node 68d45577a5c2c05277bca997 INFO: 172.21.0.1:51580 - "POST /node/68d45577a5c2c05277bca997 HTTP/1.0" 202 Accepted 2025-09-26 20:21:09,343 - lava_callback - INFO - Starting async job submit for node 68d45577a5c2c05277bca997 2025-09-26 20:21:09,371 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 20:21:09,386 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d45577a5c2c05277bca997 HTTP/1.1" 200 None 2025-09-26 20:21:09,386 - lava_callback - DEBUG - Processing node 68d45577a5c2c05277bca997: ltp-timers 2025-09-26 20:21:09,769 - lava_callback - ERROR - Error processing callback for node 68d45577a5c2c05277bca997: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None 2025-09-26 20:21:14,859 - lava_callback - INFO - Received callback for node 68d4e10ea5c2c05277bd0db9 INFO: 172.21.0.1:51596 - "POST /node/68d4e10ea5c2c05277bd0db9 HTTP/1.0" 202 Accepted 2025-09-26 20:21:14,870 - lava_callback - INFO - Starting async job submit for node 68d4e10ea5c2c05277bd0db9 2025-09-26 20:21:14,900 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 20:21:14,917 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4e10ea5c2c05277bd0db9 HTTP/1.1" 200 None 2025-09-26 20:21:14,917 - lava_callback - DEBUG - Processing node 68d4e10ea5c2c05277bd0db9: baseline-arm INFO: 172.21.0.1:51598 - "GET /api/metrics HTTP/1.0" 200 OK 2025-09-26 20:21:15,220 - lava_callback - ERROR - Error processing callback for node 68d4e10ea5c2c05277bd0db9: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:51612 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58238 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42986 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42988 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35098 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54548 - "GET / HTTP/1.0" 200 OK 2025-09-26 20:21:56,449 - lava_callback - INFO - Received callback for node 68d451e2a5c2c05277bca7d6 INFO: 172.21.0.1:54550 - "POST /node/68d451e2a5c2c05277bca7d6 HTTP/1.0" 202 Accepted 2025-09-26 20:21:56,455 - lava_callback - INFO - Starting async job submit for node 68d451e2a5c2c05277bca7d6 2025-09-26 20:21:56,484 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 20:21:56,508 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d451e2a5c2c05277bca7d6 HTTP/1.1" 200 None 2025-09-26 20:21:56,508 - lava_callback - DEBUG - Processing node 68d451e2a5c2c05277bca7d6: baseline-arm INFO: 172.21.0.1:54564 - "GET / HTTP/1.0" 200 OK 2025-09-26 20:21:56,784 - lava_callback - ERROR - Error processing callback for node 68d451e2a5c2c05277bca7d6: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None 2025-09-26 20:21:57,890 - lava_callback - INFO - Received callback for node 68d4556ba5c2c05277bca98b INFO: 172.21.0.1:54568 - "POST /node/68d4556ba5c2c05277bca98b HTTP/1.0" 202 Accepted 2025-09-26 20:21:57,897 - lava_callback - INFO - Starting async job submit for node 68d4556ba5c2c05277bca98b 2025-09-26 20:21:57,925 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 20:21:57,937 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4556ba5c2c05277bca98b HTTP/1.1" 200 None 2025-09-26 20:21:57,938 - lava_callback - DEBUG - Processing node 68d4556ba5c2c05277bca98b: ltp-sched 2025-09-26 20:21:58,275 - lava_callback - ERROR - Error processing callback for node 68d4556ba5c2c05277bca98b: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:59546 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45282 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45286 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37660 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47032 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47040 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52578 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41876 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41878 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45586 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48946 - "GET /api/metrics HTTP/1.0" 200 OK 2025-09-26 20:23:16,309 - lava_callback - INFO - Received callback for node 68d4e10ea5c2c05277bd0dbc INFO: 172.21.0.1:48950 - "POST /node/68d4e10ea5c2c05277bd0dbc HTTP/1.0" 202 Accepted 2025-09-26 20:23:16,315 - lava_callback - INFO - Starting async job submit for node 68d4e10ea5c2c05277bd0dbc 2025-09-26 20:23:16,333 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 20:23:16,346 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4e10ea5c2c05277bd0dbc HTTP/1.1" 200 None 2025-09-26 20:23:16,346 - lava_callback - DEBUG - Processing node 68d4e10ea5c2c05277bd0dbc: baseline-arm INFO: 172.21.0.1:48958 - "GET / HTTP/1.0" 200 OK 2025-09-26 20:23:16,566 - lava_callback - ERROR - Error processing callback for node 68d4e10ea5c2c05277bd0dbc: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:47572 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43676 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43686 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51684 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44594 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44596 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42972 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33340 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33350 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48032 - "GET / HTTP/1.0" 200 OK 2025-09-26 20:24:30,035 - lava_callback - INFO - Received callback for node 68d6eb663c94e462c68823ba INFO: 172.21.0.1:60022 - "POST /node/68d6eb663c94e462c68823ba HTTP/1.0" 202 Accepted 2025-09-26 20:24:30,043 - lava_callback - INFO - Starting async job submit for node 68d6eb663c94e462c68823ba 2025-09-26 20:24:30,107 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 20:24:30,122 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6eb663c94e462c68823ba HTTP/1.1" 200 None 2025-09-26 20:24:30,123 - lava_callback - DEBUG - Processing node 68d6eb663c94e462c68823ba: baseline-arm64 2025-09-26 20:24:31,194 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 20:24:31,346 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 20:24:31,346 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6eb663c94e462c68823ba/log.txt.gz 2025-09-26 20:24:31,367 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 20:24:31,668 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 20:24:31,669 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6eb663c94e462c68823ba/lava_callback.json.gz DEBUG: Setup failed for 68d6eb663c94e462c68823ba. Transitting job node result: incomplete -> fail 2025-09-26 20:24:31,756 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 20:24:31,788 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6eb663c94e462c68823ba HTTP/1.1" 200 None 2025-09-26 20:24:31,789 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 20:24:31,800 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d651ddcca86fb17472aa24 HTTP/1.1" 200 None 2025-09-26 20:24:31,802 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 20:24:31,850 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6eb663c94e462c68823ba HTTP/1.1" 200 None 2025-09-26 20:24:31,850 - lava_callback - INFO - Completed processing callback for node 68d6eb663c94e462c68823ba INFO: 172.21.0.1:60026 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60034 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51334 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41128 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41136 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58516 - "GET / HTTP/1.0" 200 OK 2025-09-26 20:25:13,954 - lava_callback - INFO - Received callback for node 68d4e110a5c2c05277bd0dbf INFO: 172.21.0.1:40330 - "POST /node/68d4e110a5c2c05277bd0dbf HTTP/1.0" 202 Accepted 2025-09-26 20:25:13,961 - lava_callback - INFO - Starting async job submit for node 68d4e110a5c2c05277bd0dbf 2025-09-26 20:25:13,981 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 20:25:13,993 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4e110a5c2c05277bd0dbf HTTP/1.1" 200 None 2025-09-26 20:25:13,994 - lava_callback - DEBUG - Processing node 68d4e110a5c2c05277bd0dbf: baseline-arm 2025-09-26 20:25:14,220 - lava_callback - ERROR - Error processing callback for node 68d4e110a5c2c05277bd0dbf: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:40346 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40358 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42384 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34180 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34188 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41528 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57496 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57508 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42754 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45758 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45762 - "GET / HTTP/1.0" 200 OK 2025-09-26 20:26:17,732 - lava_callback - INFO - Received callback for node 68d453d9a5c2c05277bca895 INFO: 172.21.0.1:45764 - "POST /node/68d453d9a5c2c05277bca895 HTTP/1.0" 202 Accepted 2025-09-26 20:26:17,738 - lava_callback - INFO - Starting async job submit for node 68d453d9a5c2c05277bca895 2025-09-26 20:26:17,738 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 20:26:17,750 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d453d9a5c2c05277bca895 HTTP/1.1" 200 None 2025-09-26 20:26:17,750 - lava_callback - DEBUG - Processing node 68d453d9a5c2c05277bca895: baseline-arm 2025-09-26 20:26:17,750 - lava_callback - ERROR - Error processing callback for node 68d453d9a5c2c05277bca895: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:57572 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35186 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35198 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39484 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40194 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40204 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48438 - "GET / HTTP/1.0" 200 OK 2025-09-26 20:27:14,463 - lava_callback - INFO - Received callback for node 68d4e111a5c2c05277bd0dc2 INFO: 172.21.0.1:37494 - "POST /node/68d4e111a5c2c05277bd0dc2 HTTP/1.0" 202 Accepted 2025-09-26 20:27:14,469 - lava_callback - INFO - Starting async job submit for node 68d4e111a5c2c05277bd0dc2 2025-09-26 20:27:14,489 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 20:27:14,528 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4e111a5c2c05277bd0dc2 HTTP/1.1" 200 None 2025-09-26 20:27:14,528 - lava_callback - DEBUG - Processing node 68d4e111a5c2c05277bd0dc2: baseline-arm 2025-09-26 20:27:14,781 - lava_callback - ERROR - Error processing callback for node 68d4e111a5c2c05277bd0dc2: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:37502 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37516 - "GET / HTTP/1.0" 200 OK 2025-09-26 20:27:17,614 - lava_callback - INFO - Received callback for node 68d45414a5c2c05277bca89d INFO: 172.21.0.1:37518 - "POST /node/68d45414a5c2c05277bca89d HTTP/1.0" 202 Accepted 2025-09-26 20:27:17,622 - lava_callback - INFO - Starting async job submit for node 68d45414a5c2c05277bca89d 2025-09-26 20:27:17,623 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 20:27:17,637 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d45414a5c2c05277bca89d HTTP/1.1" 200 None 2025-09-26 20:27:17,638 - lava_callback - DEBUG - Processing node 68d45414a5c2c05277bca89d: baseline-arm 2025-09-26 20:27:17,638 - lava_callback - ERROR - Error processing callback for node 68d45414a5c2c05277bca89d: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 20:27:18,324 - lava_callback - INFO - Received callback for node 68d45416a5c2c05277bca8ac INFO: 172.21.0.1:37530 - "POST /node/68d45416a5c2c05277bca8ac HTTP/1.0" 202 Accepted 2025-09-26 20:27:18,329 - lava_callback - INFO - Starting async job submit for node 68d45416a5c2c05277bca8ac 2025-09-26 20:27:18,330 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 20:27:18,365 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d45416a5c2c05277bca8ac HTTP/1.1" 200 None 2025-09-26 20:27:18,366 - lava_callback - DEBUG - Processing node 68d45416a5c2c05277bca8ac: baseline-arm 2025-09-26 20:27:18,366 - lava_callback - ERROR - Error processing callback for node 68d45416a5c2c05277bca8ac: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:37572 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45872 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45878 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48502 - "GET / HTTP/1.0" 200 OK 2025-09-26 20:27:48,588 - lava_callback - INFO - Received callback for node 68d6eb673c94e462c68823bd INFO: 172.21.0.1:48510 - "POST /node/68d6eb673c94e462c68823bd HTTP/1.0" 202 Accepted 2025-09-26 20:27:48,595 - lava_callback - INFO - Starting async job submit for node 68d6eb673c94e462c68823bd 2025-09-26 20:27:48,628 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 20:27:48,651 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6eb673c94e462c68823bd HTTP/1.1" 200 None 2025-09-26 20:27:48,652 - lava_callback - DEBUG - Processing node 68d6eb673c94e462c68823bd: baseline-arm64 2025-09-26 20:27:49,198 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 20:27:49,364 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 20:27:49,365 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6eb673c94e462c68823bd/log.txt.gz 2025-09-26 20:27:49,377 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 20:27:49,525 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 20:27:49,526 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6eb673c94e462c68823bd/lava_callback.json.gz DEBUG: Setup failed for 68d6eb673c94e462c68823bd. Transitting job node result: incomplete -> fail 2025-09-26 20:27:49,592 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 20:27:49,615 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6eb673c94e462c68823bd HTTP/1.1" 200 None 2025-09-26 20:27:49,616 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 20:27:49,640 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d651ddcca86fb17472aa24 HTTP/1.1" 200 None 2025-09-26 20:27:49,641 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 20:27:49,668 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6eb673c94e462c68823bd HTTP/1.1" 200 None 2025-09-26 20:27:49,668 - lava_callback - INFO - Completed processing callback for node 68d6eb673c94e462c68823bd INFO: 172.21.0.1:50180 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50188 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40440 - "GET / HTTP/1.0" 200 OK 2025-09-26 20:28:10,494 - lava_callback - INFO - Received callback for node 68d4557ca5c2c05277bca99a INFO: 172.21.0.1:53376 - "POST /node/68d4557ca5c2c05277bca99a HTTP/1.0" 202 Accepted 2025-09-26 20:28:10,501 - lava_callback - INFO - Starting async job submit for node 68d4557ca5c2c05277bca99a 2025-09-26 20:28:10,535 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 20:28:10,548 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4557ca5c2c05277bca99a HTTP/1.1" 200 None 2025-09-26 20:28:10,549 - lava_callback - DEBUG - Processing node 68d4557ca5c2c05277bca99a: ltp-watchqueue 2025-09-26 20:28:10,959 - lava_callback - ERROR - Error processing callback for node 68d4557ca5c2c05277bca99a: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:53378 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53384 - "GET / HTTP/1.0" 200 OK 2025-09-26 20:28:18,329 - lava_callback - INFO - Received callback for node 68d45574a5c2c05277bca994 INFO: 172.21.0.1:53400 - "POST /node/68d45574a5c2c05277bca994 HTTP/1.0" 202 Accepted 2025-09-26 20:28:18,336 - lava_callback - INFO - Starting async job submit for node 68d45574a5c2c05277bca994 2025-09-26 20:28:18,364 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 20:28:18,377 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d45574a5c2c05277bca994 HTTP/1.1" 200 None 2025-09-26 20:28:18,377 - lava_callback - DEBUG - Processing node 68d45574a5c2c05277bca994: ltp-syscalls-ipc 2025-09-26 20:28:19,278 - lava_callback - ERROR - Error processing callback for node 68d45574a5c2c05277bca994: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:58648 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45932 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45944 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57202 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40096 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40102 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46986 - "GET / HTTP/1.0" 200 OK 2025-09-26 20:29:13,534 - lava_callback - INFO - Received callback for node 68d4e11fa5c2c05277bd0de9 INFO: 172.21.0.1:44242 - "POST /node/68d4e11fa5c2c05277bd0de9 HTTP/1.0" 202 Accepted 2025-09-26 20:29:13,540 - lava_callback - INFO - Starting async job submit for node 68d4e11fa5c2c05277bd0de9 2025-09-26 20:29:13,558 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 20:29:13,571 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4e11fa5c2c05277bd0de9 HTTP/1.1" 200 None 2025-09-26 20:29:13,571 - lava_callback - DEBUG - Processing node 68d4e11fa5c2c05277bd0de9: baseline-arm 2025-09-26 20:29:13,791 - lava_callback - ERROR - Error processing callback for node 68d4e11fa5c2c05277bd0de9: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:44258 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44268 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60504 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49374 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49380 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44162 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53380 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53382 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39134 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39050 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39064 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57390 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37684 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37700 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32846 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45388 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45404 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48482 - "GET / HTTP/1.0" 200 OK 2025-09-26 20:31:15,131 - lava_callback - INFO - Received callback for node 68d4e11fa5c2c05277bd0dec INFO: 172.21.0.1:55582 - "POST /node/68d4e11fa5c2c05277bd0dec HTTP/1.0" 202 Accepted 2025-09-26 20:31:15,137 - lava_callback - INFO - Starting async job submit for node 68d4e11fa5c2c05277bd0dec 2025-09-26 20:31:15,157 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.21.0.1:55596 - "GET /api/metrics HTTP/1.0" 200 OK 2025-09-26 20:31:15,170 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4e11fa5c2c05277bd0dec HTTP/1.1" 200 None 2025-09-26 20:31:15,170 - lava_callback - DEBUG - Processing node 68d4e11fa5c2c05277bd0dec: baseline-arm 2025-09-26 20:31:15,426 - lava_callback - ERROR - Error processing callback for node 68d4e11fa5c2c05277bd0dec: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:55600 - "GET / HTTP/1.0" 200 OK 2025-09-26 20:31:17,994 - lava_callback - INFO - Received callback for node 68d45513a5c2c05277bca920 INFO: 172.21.0.1:55612 - "POST /node/68d45513a5c2c05277bca920 HTTP/1.0" 202 Accepted 2025-09-26 20:31:18,000 - lava_callback - INFO - Starting async job submit for node 68d45513a5c2c05277bca920 2025-09-26 20:31:18,000 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 20:31:18,034 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d45513a5c2c05277bca920 HTTP/1.1" 200 None 2025-09-26 20:31:18,035 - lava_callback - DEBUG - Processing node 68d45513a5c2c05277bca920: baseline-arm 2025-09-26 20:31:18,035 - lava_callback - ERROR - Error processing callback for node 68d45513a5c2c05277bca920: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:36436 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59380 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59390 - "GET / HTTP/1.0" 200 OK 2025-09-26 20:31:37,372 - lava_callback - INFO - Received callback for node 68d45514a5c2c05277bca923 INFO: 172.21.0.1:59402 - "POST /node/68d45514a5c2c05277bca923 HTTP/1.0" 202 Accepted 2025-09-26 20:31:37,377 - lava_callback - INFO - Starting async job submit for node 68d45514a5c2c05277bca923 2025-09-26 20:31:37,378 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 20:31:37,392 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d45514a5c2c05277bca923 HTTP/1.1" 200 None 2025-09-26 20:31:37,393 - lava_callback - DEBUG - Processing node 68d45514a5c2c05277bca923: baseline-arm 2025-09-26 20:31:37,393 - lava_callback - ERROR - Error processing callback for node 68d45514a5c2c05277bca923: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 20:31:38,038 - lava_callback - INFO - Received callback for node 68d45526a5c2c05277bca938 INFO: 172.21.0.1:59418 - "POST /node/68d45526a5c2c05277bca938 HTTP/1.0" 202 Accepted 2025-09-26 20:31:38,044 - lava_callback - INFO - Starting async job submit for node 68d45526a5c2c05277bca938 2025-09-26 20:31:38,044 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 20:31:38,101 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d45526a5c2c05277bca938 HTTP/1.1" 200 None 2025-09-26 20:31:38,102 - lava_callback - DEBUG - Processing node 68d45526a5c2c05277bca938: baseline-arm 2025-09-26 20:31:38,102 - lava_callback - ERROR - Error processing callback for node 68d45526a5c2c05277bca938: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 20:31:38,715 - lava_callback - INFO - Received callback for node 68d45526a5c2c05277bca93b INFO: 172.21.0.1:59428 - "POST /node/68d45526a5c2c05277bca93b HTTP/1.0" 202 Accepted 2025-09-26 20:31:38,720 - lava_callback - INFO - Starting async job submit for node 68d45526a5c2c05277bca93b 2025-09-26 20:31:38,721 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 20:31:38,750 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d45526a5c2c05277bca93b HTTP/1.1" 200 None 2025-09-26 20:31:38,750 - lava_callback - DEBUG - Processing node 68d45526a5c2c05277bca93b: baseline-arm 2025-09-26 20:31:38,751 - lava_callback - ERROR - Error processing callback for node 68d45526a5c2c05277bca93b: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:53576 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51846 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51862 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57542 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54536 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54538 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34462 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41408 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41414 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33548 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51000 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51012 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57346 - "GET / HTTP/1.0" 200 OK 2025-09-26 20:33:11,808 - lava_callback - INFO - Received callback for node 68d4c7f2a5c2c05277bd0217 INFO: 172.21.0.1:46114 - "POST /node/68d4c7f2a5c2c05277bd0217 HTTP/1.0" 202 Accepted 2025-09-26 20:33:11,813 - lava_callback - INFO - Starting async job submit for node 68d4c7f2a5c2c05277bd0217 2025-09-26 20:33:11,830 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 20:33:11,856 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4c7f2a5c2c05277bd0217 HTTP/1.1" 200 None 2025-09-26 20:33:11,856 - lava_callback - DEBUG - Processing node 68d4c7f2a5c2c05277bd0217: baseline-arm64-broonie 2025-09-26 20:33:12,165 - lava_callback - ERROR - Error processing callback for node 68d4c7f2a5c2c05277bd0217: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:46122 - "GET /api/metrics HTTP/1.0" 200 OK 2025-09-26 20:33:15,761 - lava_callback - INFO - Received callback for node 68d4e121a5c2c05277bd0def INFO: 172.21.0.1:46138 - "POST /node/68d4e121a5c2c05277bd0def HTTP/1.0" 202 Accepted 2025-09-26 20:33:15,767 - lava_callback - INFO - Starting async job submit for node 68d4e121a5c2c05277bd0def 2025-09-26 20:33:15,788 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 20:33:15,800 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4e121a5c2c05277bd0def HTTP/1.1" 200 None 2025-09-26 20:33:15,800 - lava_callback - DEBUG - Processing node 68d4e121a5c2c05277bd0def: baseline-arm 2025-09-26 20:33:16,032 - lava_callback - ERROR - Error processing callback for node 68d4e121a5c2c05277bd0def: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:46140 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46222 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47348 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47350 - "GET / HTTP/1.0" 200 OK 2025-09-26 20:33:39,979 - lava_callback - INFO - Received callback for node 68d6c7893c94e462c68818a1 INFO: 172.21.0.1:46456 - "POST /node/68d6c7893c94e462c68818a1 HTTP/1.0" 202 Accepted 2025-09-26 20:33:39,985 - lava_callback - INFO - Starting async job submit for node 68d6c7893c94e462c68818a1 2025-09-26 20:33:40,007 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 20:33:40,023 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6c7893c94e462c68818a1 HTTP/1.1" 200 None 2025-09-26 20:33:40,023 - lava_callback - DEBUG - Processing node 68d6c7893c94e462c68818a1: baseline-arm64-broonie 2025-09-26 20:33:40,246 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 20:33:40,391 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 20:33:40,392 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-68d6c7893c94e462c68818a1/log.txt.gz 2025-09-26 20:33:40,398 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 20:33:40,539 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 20:33:40,540 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-68d6c7893c94e462c68818a1/lava_callback.json.gz 2025-09-26 20:33:40,562 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 20:33:40,583 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6c7893c94e462c68818a1 HTTP/1.1" 200 None 2025-09-26 20:33:40,584 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 20:33:40,599 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6b9dd2f83206171a92b48 HTTP/1.1" 200 None 2025-09-26 20:33:40,600 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 20:33:40,634 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6c7893c94e462c68818a1 HTTP/1.1" 200 None 2025-09-26 20:33:40,636 - lava_callback - INFO - Completed processing callback for node 68d6c7893c94e462c68818a1 INFO: 172.21.0.1:46470 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57284 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57290 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56982 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48292 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48300 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60052 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39620 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39636 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54048 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51790 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51802 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49548 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60408 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60422 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56774 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58590 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58598 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59582 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58336 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58348 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54880 - "GET / HTTP/1.0" 200 OK 2025-09-26 20:36:09,884 - lava_callback - INFO - Received callback for node 68d4c8aaa5c2c05277bd0342 INFO: 172.21.0.1:44790 - "POST /node/68d4c8aaa5c2c05277bd0342 HTTP/1.0" 202 Accepted 2025-09-26 20:36:09,890 - lava_callback - INFO - Starting async job submit for node 68d4c8aaa5c2c05277bd0342 2025-09-26 20:36:09,908 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 20:36:09,941 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4c8aaa5c2c05277bd0342 HTTP/1.1" 200 None 2025-09-26 20:36:09,942 - lava_callback - DEBUG - Processing node 68d4c8aaa5c2c05277bd0342: baseline-arm64-broonie 2025-09-26 20:36:10,254 - lava_callback - ERROR - Error processing callback for node 68d4c8aaa5c2c05277bd0342: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None 2025-09-26 20:36:13,938 - lava_callback - INFO - Received callback for node 68d4e121a5c2c05277bd0df2 INFO: 172.21.0.1:44804 - "POST /node/68d4e121a5c2c05277bd0df2 HTTP/1.0" 202 Accepted 2025-09-26 20:36:13,944 - lava_callback - INFO - Starting async job submit for node 68d4e121a5c2c05277bd0df2 2025-09-26 20:36:13,965 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 20:36:14,002 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4e121a5c2c05277bd0df2 HTTP/1.1" 200 None 2025-09-26 20:36:14,003 - lava_callback - DEBUG - Processing node 68d4e121a5c2c05277bd0df2: baseline-arm 2025-09-26 20:36:14,231 - lava_callback - ERROR - Error processing callback for node 68d4e121a5c2c05277bd0df2: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:44814 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44826 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59852 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42476 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42490 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55814 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45108 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45116 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52876 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56528 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56542 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57508 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59368 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59380 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57970 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36456 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36468 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54190 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54550 - "GET /api/metrics HTTP/1.0" 200 OK 2025-09-26 20:38:15,587 - lava_callback - INFO - Received callback for node 68d4fcb6ef0c6c1c70206022 INFO: 172.21.0.1:54566 - "POST /node/68d4fcb6ef0c6c1c70206022 HTTP/1.0" 202 Accepted 2025-09-26 20:38:15,593 - lava_callback - INFO - Starting async job submit for node 68d4fcb6ef0c6c1c70206022 2025-09-26 20:38:15,614 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 20:38:15,627 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4fcb6ef0c6c1c70206022 HTTP/1.1" 200 None 2025-09-26 20:38:15,627 - lava_callback - DEBUG - Processing node 68d4fcb6ef0c6c1c70206022: baseline-arm 2025-09-26 20:38:15,884 - lava_callback - ERROR - Error processing callback for node 68d4fcb6ef0c6c1c70206022: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:54580 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53092 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49012 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49022 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40034 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36002 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36008 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52540 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37374 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37386 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52156 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45504 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45516 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49292 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45410 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45426 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45346 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51996 - "GET /api/metrics HTTP/1.0" 200 OK 2025-09-26 20:40:15,430 - lava_callback - INFO - Received callback for node 68d4fcb8ef0c6c1c70206031 INFO: 172.21.0.1:52000 - "POST /node/68d4fcb8ef0c6c1c70206031 HTTP/1.0" 202 Accepted 2025-09-26 20:40:15,436 - lava_callback - INFO - Starting async job submit for node 68d4fcb8ef0c6c1c70206031 2025-09-26 20:40:15,456 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 20:40:15,489 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4fcb8ef0c6c1c70206031 HTTP/1.1" 200 None 2025-09-26 20:40:15,489 - lava_callback - DEBUG - Processing node 68d4fcb8ef0c6c1c70206031: baseline-arm 2025-09-26 20:40:15,714 - lava_callback - ERROR - Error processing callback for node 68d4fcb8ef0c6c1c70206031: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:52004 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37296 - "GET / HTTP/1.0" 200 OK 2025-09-26 20:40:29,654 - lava_callback - INFO - Received callback for node 68d470e2a5c2c05277bcc6e2 INFO: 172.21.0.1:36820 - "POST /node/68d470e2a5c2c05277bcc6e2 HTTP/1.0" 202 Accepted 2025-09-26 20:40:29,662 - lava_callback - INFO - Starting async job submit for node 68d470e2a5c2c05277bcc6e2 2025-09-26 20:40:29,702 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 20:40:29,714 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d470e2a5c2c05277bcc6e2 HTTP/1.1" 200 None 2025-09-26 20:40:29,715 - lava_callback - DEBUG - Processing node 68d470e2a5c2c05277bcc6e2: baseline-arm64 2025-09-26 20:40:30,622 - lava_callback - ERROR - Error processing callback for node 68d470e2a5c2c05277bcc6e2: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:36836 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36848 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39120 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45824 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45830 - "GET / HTTP/1.0" 200 OK 2025-09-26 20:40:58,789 - lava_callback - INFO - Received callback for node 68d456f4a5c2c05277bca9f3 INFO: 172.21.0.1:45838 - "POST /node/68d456f4a5c2c05277bca9f3 HTTP/1.0" 202 Accepted 2025-09-26 20:40:58,797 - lava_callback - INFO - Starting async job submit for node 68d456f4a5c2c05277bca9f3 2025-09-26 20:40:58,824 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 20:40:58,842 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d456f4a5c2c05277bca9f3 HTTP/1.1" 200 None 2025-09-26 20:40:58,842 - lava_callback - DEBUG - Processing node 68d456f4a5c2c05277bca9f3: ltp-syscalls-ipc 2025-09-26 20:40:59,714 - lava_callback - ERROR - Error processing callback for node 68d456f4a5c2c05277bca9f3: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None 2025-09-26 20:41:03,277 - lava_callback - INFO - Received callback for node 68d47154a5c2c05277bcc6ed INFO: 172.21.0.1:48692 - "POST /node/68d47154a5c2c05277bcc6ed HTTP/1.0" 202 Accepted 2025-09-26 20:41:03,284 - lava_callback - INFO - Starting async job submit for node 68d47154a5c2c05277bcc6ed 2025-09-26 20:41:03,325 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 20:41:03,337 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d47154a5c2c05277bcc6ed HTTP/1.1" 200 None 2025-09-26 20:41:03,337 - lava_callback - DEBUG - Processing node 68d47154a5c2c05277bcc6ed: baseline-arm64 2025-09-26 20:41:04,207 - lava_callback - ERROR - Error processing callback for node 68d47154a5c2c05277bcc6ed: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None 2025-09-26 20:41:04,865 - lava_callback - INFO - Received callback for node 68d6ebd63c94e462c68823c7 INFO: 172.21.0.1:48698 - "POST /node/68d6ebd63c94e462c68823c7 HTTP/1.0" 202 Accepted 2025-09-26 20:41:04,872 - lava_callback - INFO - Starting async job submit for node 68d6ebd63c94e462c68823c7 2025-09-26 20:41:04,933 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 20:41:04,946 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6ebd63c94e462c68823c7 HTTP/1.1" 200 None 2025-09-26 20:41:04,946 - lava_callback - DEBUG - Processing node 68d6ebd63c94e462c68823c7: baseline-arm64 2025-09-26 20:41:06,049 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 20:41:06,193 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 20:41:06,193 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6ebd63c94e462c68823c7/log.txt.gz 2025-09-26 20:41:06,215 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 20:41:06,413 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 20:41:06,414 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6ebd63c94e462c68823c7/lava_callback.json.gz 2025-09-26 20:41:06,469 - lava_callback - INFO - Received callback for node 68d470e5a5c2c05277bcc6e5 INFO: 172.21.0.1:48710 - "POST /node/68d470e5a5c2c05277bcc6e5 HTTP/1.0" 202 Accepted 2025-09-26 20:41:06,502 - lava_callback - INFO - Starting async job submit for node 68d470e5a5c2c05277bcc6e5 DEBUG: Setup failed for 68d6ebd63c94e462c68823c7. Transitting job node result: incomplete -> fail 2025-09-26 20:41:06,520 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 20:41:06,547 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.21.0.1:48726 - "GET / HTTP/1.0" 200 OK 2025-09-26 20:41:06,562 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6ebd63c94e462c68823c7 HTTP/1.1" 200 None 2025-09-26 20:41:06,563 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 20:41:06,564 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d470e5a5c2c05277bcc6e5 HTTP/1.1" 200 None 2025-09-26 20:41:06,564 - lava_callback - DEBUG - Processing node 68d470e5a5c2c05277bcc6e5: baseline-arm64 2025-09-26 20:41:06,634 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d651ddcca86fb17472aa24 HTTP/1.1" 200 None 2025-09-26 20:41:06,656 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 20:41:06,782 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6ebd63c94e462c68823c7 HTTP/1.1" 200 None 2025-09-26 20:41:06,798 - lava_callback - INFO - Completed processing callback for node 68d6ebd63c94e462c68823c7 2025-09-26 20:41:07,609 - lava_callback - ERROR - Error processing callback for node 68d470e5a5c2c05277bcc6e5: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:33198 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33200 - "GET / HTTP/1.0" 200 OK 2025-09-26 20:41:18,183 - lava_callback - INFO - Received callback for node 68d453d8a5c2c05277bca892 INFO: 172.21.0.1:33216 - "POST /node/68d453d8a5c2c05277bca892 HTTP/1.0" 202 Accepted 2025-09-26 20:41:18,189 - lava_callback - INFO - Starting async job submit for node 68d453d8a5c2c05277bca892 2025-09-26 20:41:18,219 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 20:41:18,231 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d453d8a5c2c05277bca892 HTTP/1.1" 200 None 2025-09-26 20:41:18,231 - lava_callback - DEBUG - Processing node 68d453d8a5c2c05277bca892: baseline-arm 2025-09-26 20:41:18,496 - lava_callback - ERROR - Error processing callback for node 68d453d8a5c2c05277bca892: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:44068 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49800 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49806 - "GET / HTTP/1.0" 200 OK 2025-09-26 20:41:42,586 - lava_callback - INFO - Received callback for node 68d6ebd63c94e462c68823ca INFO: 172.21.0.1:53010 - "POST /node/68d6ebd63c94e462c68823ca HTTP/1.0" 202 Accepted 2025-09-26 20:41:42,593 - lava_callback - INFO - Starting async job submit for node 68d6ebd63c94e462c68823ca 2025-09-26 20:41:42,654 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 20:41:42,666 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6ebd63c94e462c68823ca HTTP/1.1" 200 None 2025-09-26 20:41:42,667 - lava_callback - DEBUG - Processing node 68d6ebd63c94e462c68823ca: baseline-arm64 2025-09-26 20:41:43,691 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 20:41:43,810 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 20:41:43,811 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6ebd63c94e462c68823ca/log.txt.gz 2025-09-26 20:41:43,831 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 20:41:44,017 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 20:41:44,018 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6ebd63c94e462c68823ca/lava_callback.json.gz DEBUG: Setup failed for 68d6ebd63c94e462c68823ca. Transitting job node result: incomplete -> fail 2025-09-26 20:41:44,102 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 20:41:44,115 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6ebd63c94e462c68823ca HTTP/1.1" 200 None 2025-09-26 20:41:44,116 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 20:41:44,128 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d651ddcca86fb17472aa24 HTTP/1.1" 200 None 2025-09-26 20:41:44,129 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 20:41:44,152 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6ebd63c94e462c68823ca HTTP/1.1" 200 None 2025-09-26 20:41:44,153 - lava_callback - INFO - Completed processing callback for node 68d6ebd63c94e462c68823ca INFO: 172.21.0.1:53012 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34218 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34226 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58702 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49752 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49766 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56278 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48952 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48960 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38418 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58404 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58408 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38932 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41936 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41946 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55098 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34466 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34478 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51476 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59394 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59408 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43658 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52540 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52556 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59246 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45870 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45878 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45496 - "GET / HTTP/1.0" 200 OK 2025-09-26 20:44:50,409 - lava_callback - INFO - Received callback for node 68d6a7662f83206171a92511 INFO: 172.21.0.1:50836 - "POST /node/68d6a7662f83206171a92511 HTTP/1.0" 202 Accepted 2025-09-26 20:44:50,415 - lava_callback - INFO - Starting async job submit for node 68d6a7662f83206171a92511 2025-09-26 20:44:50,437 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 20:44:50,459 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6a7662f83206171a92511 HTTP/1.1" 200 None 2025-09-26 20:44:50,460 - lava_callback - DEBUG - Processing node 68d6a7662f83206171a92511: baseline-arm64-broonie 2025-09-26 20:44:50,497 - lava_callback - INFO - Received callback for node 68d6ec303c94e462c68823d0 INFO: 172.21.0.1:50852 - "POST /node/68d6ec303c94e462c68823d0 HTTP/1.0" 202 Accepted 2025-09-26 20:44:50,540 - lava_callback - INFO - Starting async job submit for node 68d6ec303c94e462c68823d0 2025-09-26 20:44:50,626 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 20:44:50,696 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6ec303c94e462c68823d0 HTTP/1.1" 200 None 2025-09-26 20:44:50,712 - lava_callback - DEBUG - Processing node 68d6ec303c94e462c68823d0: baseline-arm64 2025-09-26 20:44:50,992 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 20:44:51,277 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 20:44:51,344 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-68d6a7662f83206171a92511/log.txt.gz 2025-09-26 20:44:51,443 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 20:44:51,660 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 20:44:51,712 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-68d6a7662f83206171a92511/lava_callback.json.gz 2025-09-26 20:44:51,754 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 20:44:51,839 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6a7662f83206171a92511 HTTP/1.1" 200 None 2025-09-26 20:44:51,860 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 20:44:52,021 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d648ed5d233c363a762351 HTTP/1.1" 200 None 2025-09-26 20:44:52,022 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 20:44:52,028 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 20:44:52,053 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6a7662f83206171a92511 HTTP/1.1" 200 None 2025-09-26 20:44:52,055 - lava_callback - INFO - Completed processing callback for node 68d6a7662f83206171a92511 2025-09-26 20:44:52,301 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 20:44:52,302 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6ec303c94e462c68823d0/log.txt.gz 2025-09-26 20:44:52,324 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 20:44:52,519 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 20:44:52,520 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6ec303c94e462c68823d0/lava_callback.json.gz DEBUG: Setup failed for 68d6ec303c94e462c68823d0. Transitting job node result: incomplete -> fail 2025-09-26 20:44:52,611 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 20:44:52,633 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6ec303c94e462c68823d0 HTTP/1.1" 200 None 2025-09-26 20:44:52,635 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 20:44:52,647 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d651ddcca86fb17472aa24 HTTP/1.1" 200 None 2025-09-26 20:44:52,648 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 20:44:52,674 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6ec303c94e462c68823d0 HTTP/1.1" 200 None 2025-09-26 20:44:52,675 - lava_callback - INFO - Completed processing callback for node 68d6ec303c94e462c68823d0 INFO: 172.21.0.1:50854 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50868 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56062 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42454 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42468 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44704 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48264 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48280 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42080 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48630 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48632 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46442 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53690 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53692 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60320 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40888 - "GET / HTTP/1.0" 200 OK 2025-09-26 20:46:35,417 - lava_callback - INFO - Received callback for node 68d4fe15ef0c6c1c702060ee INFO: 172.21.0.1:40900 - "POST /node/68d4fe15ef0c6c1c702060ee HTTP/1.0" 202 Accepted 2025-09-26 20:46:35,423 - lava_callback - INFO - Starting async job submit for node 68d4fe15ef0c6c1c702060ee 2025-09-26 20:46:35,444 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 20:46:35,456 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4fe15ef0c6c1c702060ee HTTP/1.1" 200 None 2025-09-26 20:46:35,457 - lava_callback - DEBUG - Processing node 68d4fe15ef0c6c1c702060ee: baseline-arm 2025-09-26 20:46:35,684 - lava_callback - ERROR - Error processing callback for node 68d4fe15ef0c6c1c702060ee: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:40906 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34146 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42552 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42566 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54290 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56008 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56020 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56202 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41808 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41820 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52890 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57052 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57060 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57264 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49764 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49770 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60792 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42318 - "GET / HTTP/1.0" 200 OK 2025-09-26 20:48:36,403 - lava_callback - INFO - Received callback for node 68d4fe15ef0c6c1c702060f1 INFO: 172.21.0.1:42320 - "POST /node/68d4fe15ef0c6c1c702060f1 HTTP/1.0" 202 Accepted 2025-09-26 20:48:36,409 - lava_callback - INFO - Starting async job submit for node 68d4fe15ef0c6c1c702060f1 2025-09-26 20:48:36,430 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 20:48:36,458 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4fe15ef0c6c1c702060f1 HTTP/1.1" 200 None 2025-09-26 20:48:36,459 - lava_callback - DEBUG - Processing node 68d4fe15ef0c6c1c702060f1: baseline-arm INFO: 172.21.0.1:42332 - "GET / HTTP/1.0" 200 OK 2025-09-26 20:48:36,696 - lava_callback - ERROR - Error processing callback for node 68d4fe15ef0c6c1c702060f1: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:32966 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34292 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34308 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60366 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54732 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54746 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56508 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43358 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43360 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57822 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49452 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49454 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55294 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36824 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36832 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57464 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47300 - "GET / HTTP/1.0" 200 OK 2025-09-26 20:50:35,372 - lava_callback - INFO - Received callback for node 68d4fe17ef0c6c1c702060f4 INFO: 172.21.0.1:47314 - "POST /node/68d4fe17ef0c6c1c702060f4 HTTP/1.0" 202 Accepted 2025-09-26 20:50:35,378 - lava_callback - INFO - Starting async job submit for node 68d4fe17ef0c6c1c702060f4 2025-09-26 20:50:35,398 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 20:50:35,410 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4fe17ef0c6c1c702060f4 HTTP/1.1" 200 None 2025-09-26 20:50:35,410 - lava_callback - DEBUG - Processing node 68d4fe17ef0c6c1c702060f4: baseline-arm 2025-09-26 20:50:35,657 - lava_callback - ERROR - Error processing callback for node 68d4fe17ef0c6c1c702060f4: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:47324 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42714 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49876 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34654 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34666 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56904 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56918 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51176 - "GET / HTTP/1.0" 200 OK 2025-09-26 20:51:29,690 - lava_callback - INFO - Received callback for node 68d6ec303c94e462c68823d3 INFO: 172.21.0.1:59018 - "POST /node/68d6ec303c94e462c68823d3 HTTP/1.0" 202 Accepted 2025-09-26 20:51:29,697 - lava_callback - INFO - Starting async job submit for node 68d6ec303c94e462c68823d3 2025-09-26 20:51:29,730 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 20:51:29,743 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6ec303c94e462c68823d3 HTTP/1.1" 200 None 2025-09-26 20:51:29,744 - lava_callback - DEBUG - Processing node 68d6ec303c94e462c68823d3: baseline-arm64 2025-09-26 20:51:30,284 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 20:51:30,432 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 20:51:30,433 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6ec303c94e462c68823d3/log.txt.gz 2025-09-26 20:51:30,444 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 20:51:30,565 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 20:51:30,565 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6ec303c94e462c68823d3/lava_callback.json.gz DEBUG: Setup failed for 68d6ec303c94e462c68823d3. Transitting job node result: incomplete -> fail 2025-09-26 20:51:30,633 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 20:51:30,645 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6ec303c94e462c68823d3 HTTP/1.1" 200 None 2025-09-26 20:51:30,646 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 20:51:30,665 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d651ddcca86fb17472aa24 HTTP/1.1" 200 None 2025-09-26 20:51:30,666 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 20:51:30,691 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6ec303c94e462c68823d3 HTTP/1.1" 200 None 2025-09-26 20:51:30,692 - lava_callback - INFO - Completed processing callback for node 68d6ec303c94e462c68823d3 INFO: 172.21.0.1:59020 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59026 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54174 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47852 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41394 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41400 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58278 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58284 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50320 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57370 - "GET / HTTP/1.0" 200 OK 2025-09-26 20:52:34,071 - lava_callback - INFO - Received callback for node 68d4fe17ef0c6c1c702060f7 INFO: 172.21.0.1:57378 - "POST /node/68d4fe17ef0c6c1c702060f7 HTTP/1.0" 202 Accepted 2025-09-26 20:52:34,077 - lava_callback - INFO - Starting async job submit for node 68d4fe17ef0c6c1c702060f7 2025-09-26 20:52:34,097 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 20:52:34,109 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4fe17ef0c6c1c702060f7 HTTP/1.1" 200 None 2025-09-26 20:52:34,109 - lava_callback - DEBUG - Processing node 68d4fe17ef0c6c1c702060f7: baseline-arm 2025-09-26 20:52:34,355 - lava_callback - ERROR - Error processing callback for node 68d4fe17ef0c6c1c702060f7: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:57380 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52824 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53026 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49576 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49584 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36550 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36558 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48486 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50004 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50008 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51010 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48168 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46366 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46382 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57018 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57032 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41690 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48602 - "GET / HTTP/1.0" 200 OK 2025-09-26 20:54:35,474 - lava_callback - INFO - Received callback for node 68d4fe19ef0c6c1c702060fa INFO: 172.21.0.1:48616 - "POST /node/68d4fe19ef0c6c1c702060fa HTTP/1.0" 202 Accepted 2025-09-26 20:54:35,480 - lava_callback - INFO - Starting async job submit for node 68d4fe19ef0c6c1c702060fa 2025-09-26 20:54:35,501 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 20:54:35,537 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4fe19ef0c6c1c702060fa HTTP/1.1" 200 None 2025-09-26 20:54:35,537 - lava_callback - DEBUG - Processing node 68d4fe19ef0c6c1c702060fa: baseline-arm 2025-09-26 20:54:35,776 - lava_callback - ERROR - Error processing callback for node 68d4fe19ef0c6c1c702060fa: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:48626 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52398 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58640 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32950 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32962 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57780 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57786 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49630 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39136 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39148 - "GET / HTTP/1.0" 200 OK 2025-09-26 20:55:39,301 - lava_callback - INFO - Received callback for node 68d6a8752f83206171a92673 INFO: 172.21.0.1:39150 - "POST /node/68d6a8752f83206171a92673 HTTP/1.0" 202 Accepted 2025-09-26 20:55:39,307 - lava_callback - INFO - Starting async job submit for node 68d6a8752f83206171a92673 2025-09-26 20:55:39,341 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 20:55:39,375 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6a8752f83206171a92673 HTTP/1.1" 200 None 2025-09-26 20:55:39,375 - lava_callback - DEBUG - Processing node 68d6a8752f83206171a92673: baseline-arm64-broonie 2025-09-26 20:55:39,854 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 20:55:40,007 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 20:55:40,008 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-68d6a8752f83206171a92673/log.txt.gz 2025-09-26 20:55:40,018 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 20:55:40,156 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 20:55:40,157 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-68d6a8752f83206171a92673/lava_callback.json.gz 2025-09-26 20:55:40,198 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 20:55:40,232 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6a8752f83206171a92673 HTTP/1.1" 200 None 2025-09-26 20:55:40,233 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 20:55:40,245 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d64d12a5c3ab168ecc7a3d HTTP/1.1" 200 None 2025-09-26 20:55:40,246 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 20:55:40,272 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6a8752f83206171a92673 HTTP/1.1" 200 None 2025-09-26 20:55:40,272 - lava_callback - INFO - Completed processing callback for node 68d6a8752f83206171a92673 INFO: 172.21.0.1:34296 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45216 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57380 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57396 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43212 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43224 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58052 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48266 - "GET / HTTP/1.0" 200 OK 2025-09-26 20:56:35,250 - lava_callback - INFO - Received callback for node 68d4fe19ef0c6c1c702060fd INFO: 172.21.0.1:48272 - "POST /node/68d4fe19ef0c6c1c702060fd HTTP/1.0" 202 Accepted 2025-09-26 20:56:35,256 - lava_callback - INFO - Starting async job submit for node 68d4fe19ef0c6c1c702060fd 2025-09-26 20:56:35,277 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 20:56:35,290 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4fe19ef0c6c1c702060fd HTTP/1.1" 200 None 2025-09-26 20:56:35,290 - lava_callback - DEBUG - Processing node 68d4fe19ef0c6c1c702060fd: baseline-arm 2025-09-26 20:56:35,529 - lava_callback - ERROR - Error processing callback for node 68d4fe19ef0c6c1c702060fd: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:48284 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45634 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57878 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49492 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49496 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36270 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36282 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56044 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37730 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37744 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60478 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54310 - "GET / HTTP/1.0" 200 OK 2025-09-26 20:57:59,712 - lava_callback - INFO - Received callback for node 68d6ae562f83206171a928c2 INFO: 172.21.0.1:43288 - "POST /node/68d6ae562f83206171a928c2 HTTP/1.0" 202 Accepted 2025-09-26 20:57:59,718 - lava_callback - INFO - Starting async job submit for node 68d6ae562f83206171a928c2 2025-09-26 20:57:59,739 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 20:57:59,752 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6ae562f83206171a928c2 HTTP/1.1" 200 None 2025-09-26 20:57:59,752 - lava_callback - DEBUG - Processing node 68d6ae562f83206171a928c2: baseline-arm64-broonie 2025-09-26 20:58:00,051 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 20:58:00,193 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 20:58:00,194 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-68d6ae562f83206171a928c2/log.txt.gz 2025-09-26 20:58:00,201 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 20:58:00,342 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 20:58:00,343 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-68d6ae562f83206171a928c2/lava_callback.json.gz 2025-09-26 20:58:00,365 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 20:58:00,403 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6ae562f83206171a928c2 HTTP/1.1" 200 None 2025-09-26 20:58:00,404 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 20:58:00,436 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d651ddcca86fb17472aa24 HTTP/1.1" 200 None 2025-09-26 20:58:00,438 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 20:58:00,506 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6ae562f83206171a928c2 HTTP/1.1" 200 None 2025-09-26 20:58:00,507 - lava_callback - INFO - Completed processing callback for node 68d6ae562f83206171a928c2 INFO: 172.21.0.1:43294 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43304 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43312 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52458 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52468 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38284 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60662 - "GET / HTTP/1.0" 200 OK 2025-09-26 20:58:35,760 - lava_callback - INFO - Received callback for node 68d4fe1bef0c6c1c70206100 INFO: 172.21.0.1:60676 - "POST /node/68d4fe1bef0c6c1c70206100 HTTP/1.0" 202 Accepted 2025-09-26 20:58:35,766 - lava_callback - INFO - Starting async job submit for node 68d4fe1bef0c6c1c70206100 2025-09-26 20:58:35,787 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 20:58:35,821 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4fe1bef0c6c1c70206100 HTTP/1.1" 200 None 2025-09-26 20:58:35,822 - lava_callback - DEBUG - Processing node 68d4fe1bef0c6c1c70206100: baseline-arm 2025-09-26 20:58:36,074 - lava_callback - ERROR - Error processing callback for node 68d4fe1bef0c6c1c70206100: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:60682 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50746 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49932 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46088 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46100 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35908 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35914 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40502 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45524 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45538 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37574 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37100 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41902 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41908 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39972 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39974 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33720 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58630 - "GET / HTTP/1.0" 200 OK 2025-09-26 21:00:35,420 - lava_callback - INFO - Received callback for node 68d4fe1bef0c6c1c70206103 INFO: 172.21.0.1:58636 - "POST /node/68d4fe1bef0c6c1c70206103 HTTP/1.0" 202 Accepted 2025-09-26 21:00:35,426 - lava_callback - INFO - Starting async job submit for node 68d4fe1bef0c6c1c70206103 2025-09-26 21:00:35,447 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 21:00:35,484 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4fe1bef0c6c1c70206103 HTTP/1.1" 200 None 2025-09-26 21:00:35,484 - lava_callback - DEBUG - Processing node 68d4fe1bef0c6c1c70206103: baseline-arm 2025-09-26 21:00:35,713 - lava_callback - ERROR - Error processing callback for node 68d4fe1bef0c6c1c70206103: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:58650 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50752 - "GET / HTTP/1.0" 200 OK 2025-09-26 21:00:50,568 - lava_callback - INFO - Received callback for node 68d47156a5c2c05277bcc6f0 INFO: 172.21.0.1:58756 - "POST /node/68d47156a5c2c05277bcc6f0 HTTP/1.0" 202 Accepted 2025-09-26 21:00:50,575 - lava_callback - INFO - Starting async job submit for node 68d47156a5c2c05277bcc6f0 2025-09-26 21:00:50,616 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 21:00:50,629 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d47156a5c2c05277bcc6f0 HTTP/1.1" 200 None 2025-09-26 21:00:50,629 - lava_callback - DEBUG - Processing node 68d47156a5c2c05277bcc6f0: baseline-arm64 2025-09-26 21:00:51,531 - lava_callback - ERROR - Error processing callback for node 68d47156a5c2c05277bcc6f0: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:58764 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58274 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58282 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37720 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37724 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59896 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60042 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60044 - "GET / HTTP/1.0" 200 OK 2025-09-26 21:01:43,701 - lava_callback - INFO - Received callback for node 68d4715ea5c2c05277bcc6f6 INFO: 172.21.0.1:36510 - "POST /node/68d4715ea5c2c05277bcc6f6 HTTP/1.0" 202 Accepted 2025-09-26 21:01:43,708 - lava_callback - INFO - Starting async job submit for node 68d4715ea5c2c05277bcc6f6 2025-09-26 21:01:43,748 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 21:01:43,761 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4715ea5c2c05277bcc6f6 HTTP/1.1" 200 None 2025-09-26 21:01:43,761 - lava_callback - DEBUG - Processing node 68d4715ea5c2c05277bcc6f6: baseline-arm64 2025-09-26 21:01:44,643 - lava_callback - ERROR - Error processing callback for node 68d4715ea5c2c05277bcc6f6: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None 2025-09-26 21:01:45,216 - lava_callback - INFO - Received callback for node 68d4715ba5c2c05277bcc6f3 INFO: 172.21.0.1:36512 - "POST /node/68d4715ba5c2c05277bcc6f3 HTTP/1.0" 202 Accepted 2025-09-26 21:01:45,224 - lava_callback - INFO - Starting async job submit for node 68d4715ba5c2c05277bcc6f3 2025-09-26 21:01:45,264 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 21:01:45,281 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4715ba5c2c05277bcc6f3 HTTP/1.1" 200 None 2025-09-26 21:01:45,282 - lava_callback - DEBUG - Processing node 68d4715ba5c2c05277bcc6f3: baseline-arm64 2025-09-26 21:01:46,196 - lava_callback - ERROR - Error processing callback for node 68d4715ba5c2c05277bcc6f3: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:36522 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47374 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46530 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46546 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39182 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39190 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51810 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51360 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51372 - "GET / HTTP/1.0" 200 OK 2025-09-26 21:02:41,829 - lava_callback - INFO - Received callback for node 68d4fe52ef0c6c1c7020616b INFO: 172.21.0.1:36002 - "POST /node/68d4fe52ef0c6c1c7020616b HTTP/1.0" 202 Accepted 2025-09-26 21:02:41,835 - lava_callback - INFO - Starting async job submit for node 68d4fe52ef0c6c1c7020616b 2025-09-26 21:02:41,855 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 21:02:41,884 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4fe52ef0c6c1c7020616b HTTP/1.1" 200 None 2025-09-26 21:02:41,885 - lava_callback - DEBUG - Processing node 68d4fe52ef0c6c1c7020616b: baseline-arm 2025-09-26 21:02:42,132 - lava_callback - ERROR - Error processing callback for node 68d4fe52ef0c6c1c7020616b: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:36006 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37458 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49616 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49620 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56912 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56926 - "GET / HTTP/1.0" 200 OK 2025-09-26 21:03:22,072 - lava_callback - INFO - Received callback for node 68d6ef6f3c94e462c68823f4 INFO: 172.21.0.1:59062 - "POST /node/68d6ef6f3c94e462c68823f4 HTTP/1.0" 202 Accepted 2025-09-26 21:03:22,079 - lava_callback - INFO - Starting async job submit for node 68d6ef6f3c94e462c68823f4 2025-09-26 21:03:22,127 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 21:03:22,143 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6ef6f3c94e462c68823f4 HTTP/1.1" 200 None 2025-09-26 21:03:22,143 - lava_callback - DEBUG - Processing node 68d6ef6f3c94e462c68823f4: baseline-arm64 2025-09-26 21:03:23,203 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 21:03:23,360 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 21:03:23,361 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6ef6f3c94e462c68823f4/log.txt.gz 2025-09-26 21:03:23,380 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 21:03:23,566 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 21:03:23,567 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6ef6f3c94e462c68823f4/lava_callback.json.gz DEBUG: Setup failed for 68d6ef6f3c94e462c68823f4. Transitting job node result: incomplete -> fail 2025-09-26 21:03:23,647 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 21:03:23,660 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6ef6f3c94e462c68823f4 HTTP/1.1" 200 None 2025-09-26 21:03:23,661 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 21:03:23,673 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d651ddcca86fb17472aa24 HTTP/1.1" 200 None 2025-09-26 21:03:23,674 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 21:03:23,704 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6ef6f3c94e462c68823f4 HTTP/1.1" 200 None 2025-09-26 21:03:23,704 - lava_callback - INFO - Completed processing callback for node 68d6ef6f3c94e462c68823f4 INFO: 172.21.0.1:59076 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60852 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60868 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35202 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39854 - "GET / HTTP/1.0" 200 OK 2025-09-26 21:04:02,347 - lava_callback - INFO - Received callback for node 68d6ef703c94e462c68823f7 INFO: 172.21.0.1:35854 - "POST /node/68d6ef703c94e462c68823f7 HTTP/1.0" 202 Accepted 2025-09-26 21:04:02,355 - lava_callback - INFO - Starting async job submit for node 68d6ef703c94e462c68823f7 2025-09-26 21:04:02,397 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 21:04:02,410 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6ef703c94e462c68823f7 HTTP/1.1" 200 None 2025-09-26 21:04:02,411 - lava_callback - DEBUG - Processing node 68d6ef703c94e462c68823f7: baseline-arm64 INFO: 172.21.0.1:35866 - "GET / HTTP/1.0" 200 OK 2025-09-26 21:04:03,506 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 21:04:03,642 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 21:04:03,642 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6ef703c94e462c68823f7/log.txt.gz 2025-09-26 21:04:03,665 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 21:04:03,835 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 21:04:03,836 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6ef703c94e462c68823f7/lava_callback.json.gz DEBUG: Setup failed for 68d6ef703c94e462c68823f7. Transitting job node result: incomplete -> fail 2025-09-26 21:04:03,917 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 21:04:03,932 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6ef703c94e462c68823f7 HTTP/1.1" 200 None 2025-09-26 21:04:03,933 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 21:04:03,949 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d651ddcca86fb17472aa24 HTTP/1.1" 200 None 2025-09-26 21:04:03,950 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 21:04:03,977 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6ef703c94e462c68823f7 HTTP/1.1" 200 None 2025-09-26 21:04:03,978 - lava_callback - INFO - Completed processing callback for node 68d6ef703c94e462c68823f7 INFO: 172.21.0.1:35874 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56632 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56646 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58166 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35574 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35578 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55910 - "GET / HTTP/1.0" 200 OK 2025-09-26 21:04:55,528 - lava_callback - INFO - Received callback for node 68d4fe53ef0c6c1c7020616e INFO: 172.21.0.1:49496 - "POST /node/68d4fe53ef0c6c1c7020616e HTTP/1.0" 202 Accepted 2025-09-26 21:04:55,534 - lava_callback - INFO - Starting async job submit for node 68d4fe53ef0c6c1c7020616e 2025-09-26 21:04:55,555 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 21:04:55,567 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4fe53ef0c6c1c7020616e HTTP/1.1" 200 None 2025-09-26 21:04:55,568 - lava_callback - DEBUG - Processing node 68d4fe53ef0c6c1c7020616e: baseline-arm 2025-09-26 21:04:55,816 - lava_callback - ERROR - Error processing callback for node 68d4fe53ef0c6c1c7020616e: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:49512 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45380 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45384 - "GET / HTTP/1.0" 200 OK 2025-09-26 21:05:10,778 - lava_callback - INFO - Received callback for node 68d6f02b3c94e462c6882405 INFO: 172.21.0.1:44626 - "POST /node/68d6f02b3c94e462c6882405 HTTP/1.0" 202 Accepted 2025-09-26 21:05:10,786 - lava_callback - INFO - Starting async job submit for node 68d6f02b3c94e462c6882405 2025-09-26 21:05:10,827 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 21:05:10,864 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6f02b3c94e462c6882405 HTTP/1.1" 200 None 2025-09-26 21:05:10,865 - lava_callback - DEBUG - Processing node 68d6f02b3c94e462c6882405: baseline-arm64 2025-09-26 21:05:11,935 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 21:05:12,112 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 21:05:12,113 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6f02b3c94e462c6882405/log.txt.gz 2025-09-26 21:05:12,135 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 21:05:12,319 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 21:05:12,320 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6f02b3c94e462c6882405/lava_callback.json.gz DEBUG: Setup failed for 68d6f02b3c94e462c6882405. Transitting job node result: incomplete -> fail 2025-09-26 21:05:12,401 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 21:05:12,421 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6f02b3c94e462c6882405 HTTP/1.1" 200 None 2025-09-26 21:05:12,422 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 21:05:12,435 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d651ddcca86fb17472aa24 HTTP/1.1" 200 None 2025-09-26 21:05:12,436 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 21:05:12,472 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6f02b3c94e462c6882405 HTTP/1.1" 200 None 2025-09-26 21:05:12,473 - lava_callback - INFO - Completed processing callback for node 68d6f02b3c94e462c6882405 INFO: 172.21.0.1:44638 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44654 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35618 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45658 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45672 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45352 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59272 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50412 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50420 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51708 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51718 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48412 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33922 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33924 - "GET / HTTP/1.0" 200 OK 2025-09-26 21:06:39,793 - lava_callback - INFO - Received callback for node 68d4596ca5c2c05277bcab21 INFO: 172.21.0.1:34828 - "POST /node/68d4596ca5c2c05277bcab21 HTTP/1.0" 202 Accepted 2025-09-26 21:06:39,801 - lava_callback - INFO - Starting async job submit for node 68d4596ca5c2c05277bcab21 2025-09-26 21:06:39,834 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 21:06:39,863 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4596ca5c2c05277bcab21 HTTP/1.1" 200 None 2025-09-26 21:06:39,863 - lava_callback - DEBUG - Processing node 68d4596ca5c2c05277bcab21: ltp-syscalls-ipc 2025-09-26 21:06:40,758 - lava_callback - ERROR - Error processing callback for node 68d4596ca5c2c05277bcab21: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:34836 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39076 - "GET / HTTP/1.0" 200 OK 2025-09-26 21:06:58,135 - lava_callback - INFO - Received callback for node 68d4fe54ef0c6c1c70206171 INFO: 172.21.0.1:39080 - "POST /node/68d4fe54ef0c6c1c70206171 HTTP/1.0" 202 Accepted 2025-09-26 21:06:58,140 - lava_callback - INFO - Starting async job submit for node 68d4fe54ef0c6c1c70206171 2025-09-26 21:06:58,161 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 21:06:58,183 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4fe54ef0c6c1c70206171 HTTP/1.1" 200 None 2025-09-26 21:06:58,183 - lava_callback - DEBUG - Processing node 68d4fe54ef0c6c1c70206171: baseline-arm 2025-09-26 21:06:58,439 - lava_callback - ERROR - Error processing callback for node 68d4fe54ef0c6c1c70206171: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:48222 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48236 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55244 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55246 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45118 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53866 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53880 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46918 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47614 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48260 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48268 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39778 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39794 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55812 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52164 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52168 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52022 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52086 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38750 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38756 - "GET / HTTP/1.0" 200 OK 2025-09-26 21:09:11,917 - lava_callback - INFO - Received callback for node 68d4fe55ef0c6c1c70206174 INFO: 172.21.0.1:44574 - "POST /node/68d4fe55ef0c6c1c70206174 HTTP/1.0" 202 Accepted 2025-09-26 21:09:11,923 - lava_callback - INFO - Starting async job submit for node 68d4fe55ef0c6c1c70206174 2025-09-26 21:09:11,943 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 21:09:11,956 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4fe55ef0c6c1c70206174 HTTP/1.1" 200 None 2025-09-26 21:09:11,957 - lava_callback - DEBUG - Processing node 68d4fe55ef0c6c1c70206174: baseline-arm 2025-09-26 21:09:12,208 - lava_callback - ERROR - Error processing callback for node 68d4fe55ef0c6c1c70206174: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:44590 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44600 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36028 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34220 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34236 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45348 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48376 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57202 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57204 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34984 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35000 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46118 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51366 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51372 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59972 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60106 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55518 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55526 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48028 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48044 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54750 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42932 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42934 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54980 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51844 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33610 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33626 - "GET / HTTP/1.0" 200 OK 2025-09-26 21:12:10,300 - lava_callback - INFO - Received callback for node 68d4fe56ef0c6c1c70206177 INFO: 172.21.0.1:42488 - "POST /node/68d4fe56ef0c6c1c70206177 HTTP/1.0" 202 Accepted 2025-09-26 21:12:10,306 - lava_callback - INFO - Starting async job submit for node 68d4fe56ef0c6c1c70206177 2025-09-26 21:12:10,327 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 21:12:10,361 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4fe56ef0c6c1c70206177 HTTP/1.1" 200 None 2025-09-26 21:12:10,361 - lava_callback - DEBUG - Processing node 68d4fe56ef0c6c1c70206177: baseline-arm 2025-09-26 21:12:10,585 - lava_callback - ERROR - Error processing callback for node 68d4fe56ef0c6c1c70206177: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:42492 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42500 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43202 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50994 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51006 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33930 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42782 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58132 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58134 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47282 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47298 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53138 - "GET /favicon.ico HTTP/1.0" 404 Not Found INFO: 172.21.0.1:53150 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46300 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46310 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44902 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60800 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48406 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48418 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34588 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34594 - "GET / HTTP/1.0" 200 OK 2025-09-26 21:14:16,804 - lava_callback - INFO - Received callback for node 68d4fe57ef0c6c1c7020617a INFO: 172.21.0.1:34610 - "POST /node/68d4fe57ef0c6c1c7020617a HTTP/1.0" 202 Accepted 2025-09-26 21:14:16,810 - lava_callback - INFO - Starting async job submit for node 68d4fe57ef0c6c1c7020617a 2025-09-26 21:14:16,830 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 21:14:16,860 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4fe57ef0c6c1c7020617a HTTP/1.1" 200 None 2025-09-26 21:14:16,861 - lava_callback - DEBUG - Processing node 68d4fe57ef0c6c1c7020617a: baseline-arm 2025-09-26 21:14:17,115 - lava_callback - ERROR - Error processing callback for node 68d4fe57ef0c6c1c7020617a: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:37566 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43286 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43288 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33394 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39994 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40018 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40034 - "GET / HTTP/1.0" 200 OK 2025-09-26 21:15:09,848 - lava_callback - INFO - Received callback for node 68d6f02c3c94e462c6882408 INFO: 172.21.0.1:44534 - "POST /node/68d6f02c3c94e462c6882408 HTTP/1.0" 202 Accepted 2025-09-26 21:15:09,854 - lava_callback - INFO - Starting async job submit for node 68d6f02c3c94e462c6882408 2025-09-26 21:15:09,888 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 21:15:09,900 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6f02c3c94e462c6882408 HTTP/1.1" 200 None 2025-09-26 21:15:09,901 - lava_callback - DEBUG - Processing node 68d6f02c3c94e462c6882408: baseline-arm64 2025-09-26 21:15:10,497 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 21:15:10,744 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 21:15:10,745 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6f02c3c94e462c6882408/log.txt.gz 2025-09-26 21:15:10,761 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 21:15:10,951 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 21:15:10,952 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6f02c3c94e462c6882408/lava_callback.json.gz DEBUG: Setup failed for 68d6f02c3c94e462c6882408. Transitting job node result: incomplete -> fail 2025-09-26 21:15:11,032 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 21:15:11,046 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6f02c3c94e462c6882408 HTTP/1.1" 200 None 2025-09-26 21:15:11,047 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 21:15:11,059 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d651ddcca86fb17472aa24 HTTP/1.1" 200 None 2025-09-26 21:15:11,060 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 21:15:11,088 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6f02c3c94e462c6882408 HTTP/1.1" 200 None 2025-09-26 21:15:11,088 - lava_callback - INFO - Completed processing callback for node 68d6f02c3c94e462c6882408 INFO: 172.21.0.1:44540 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44542 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36796 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50168 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50182 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52604 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34282 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42608 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42624 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40390 - "GET /api/metrics HTTP/1.0" 200 OK 2025-09-26 21:16:15,891 - lava_callback - INFO - Received callback for node 68d4fe58ef0c6c1c7020617d INFO: 172.21.0.1:40396 - "POST /node/68d4fe58ef0c6c1c7020617d HTTP/1.0" 202 Accepted 2025-09-26 21:16:15,897 - lava_callback - INFO - Starting async job submit for node 68d4fe58ef0c6c1c7020617d 2025-09-26 21:16:15,954 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 21:16:15,967 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4fe58ef0c6c1c7020617d HTTP/1.1" 200 None 2025-09-26 21:16:15,968 - lava_callback - DEBUG - Processing node 68d4fe58ef0c6c1c7020617d: baseline-arm 2025-09-26 21:16:16,252 - lava_callback - ERROR - Error processing callback for node 68d4fe58ef0c6c1c7020617d: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:40398 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34982 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39260 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39264 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55606 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43988 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56644 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56654 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51370 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51386 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50812 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53804 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53806 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48650 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57834 - "GET / HTTP/1.0" 200 OK 2025-09-26 21:17:59,192 - lava_callback - INFO - Received callback for node 68d45ff7a5c2c05277bcadea INFO: 172.21.0.1:57850 - "POST /node/68d45ff7a5c2c05277bcadea HTTP/1.0" 202 Accepted 2025-09-26 21:17:59,198 - lava_callback - INFO - Starting async job submit for node 68d45ff7a5c2c05277bcadea 2025-09-26 21:17:59,199 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 21:17:59,230 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d45ff7a5c2c05277bcadea HTTP/1.1" 200 None 2025-09-26 21:17:59,230 - lava_callback - DEBUG - Processing node 68d45ff7a5c2c05277bcadea: baseline-arm 2025-09-26 21:17:59,230 - lava_callback - ERROR - Error processing callback for node 68d45ff7a5c2c05277bcadea: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:36246 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36254 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33384 - "GET /api/metrics HTTP/1.0" 200 OK 2025-09-26 21:18:16,185 - lava_callback - INFO - Received callback for node 68d4fe59ef0c6c1c70206180 INFO: 172.21.0.1:33394 - "POST /node/68d4fe59ef0c6c1c70206180 HTTP/1.0" 202 Accepted 2025-09-26 21:18:16,191 - lava_callback - INFO - Starting async job submit for node 68d4fe59ef0c6c1c70206180 2025-09-26 21:18:16,212 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 21:18:16,224 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4fe59ef0c6c1c70206180 HTTP/1.1" 200 None 2025-09-26 21:18:16,225 - lava_callback - DEBUG - Processing node 68d4fe59ef0c6c1c70206180: baseline-arm 2025-09-26 21:18:16,457 - lava_callback - ERROR - Error processing callback for node 68d4fe59ef0c6c1c70206180: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:33402 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46054 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39196 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39202 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52398 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46644 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48302 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48318 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44980 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44990 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46968 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59970 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59984 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47380 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45554 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39398 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39400 - "GET / HTTP/1.0" 200 OK 2025-09-26 21:20:12,402 - lava_callback - INFO - Received callback for node 68d4ff65ef0c6c1c702063dd INFO: 172.21.0.1:60394 - "POST /node/68d4ff65ef0c6c1c702063dd HTTP/1.0" 202 Accepted 2025-09-26 21:20:12,410 - lava_callback - INFO - Starting async job submit for node 68d4ff65ef0c6c1c702063dd 2025-09-26 21:20:12,431 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 21:20:12,449 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4ff65ef0c6c1c702063dd HTTP/1.1" 200 None 2025-09-26 21:20:12,450 - lava_callback - DEBUG - Processing node 68d4ff65ef0c6c1c702063dd: baseline-arm 2025-09-26 21:20:12,674 - lava_callback - ERROR - Error processing callback for node 68d4ff65ef0c6c1c702063dd: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:60400 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60402 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45676 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38734 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38748 - "GET / HTTP/1.0" 200 OK 2025-09-26 21:20:38,578 - lava_callback - INFO - Received callback for node 68d45ff3a5c2c05277bcade7 INFO: 172.21.0.1:38758 - "POST /node/68d45ff3a5c2c05277bcade7 HTTP/1.0" 202 Accepted 2025-09-26 21:20:38,584 - lava_callback - INFO - Starting async job submit for node 68d45ff3a5c2c05277bcade7 2025-09-26 21:20:38,614 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 21:20:38,628 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d45ff3a5c2c05277bcade7 HTTP/1.1" 200 None 2025-09-26 21:20:38,629 - lava_callback - DEBUG - Processing node 68d45ff3a5c2c05277bcade7: baseline-arm 2025-09-26 21:20:38,916 - lava_callback - ERROR - Error processing callback for node 68d45ff3a5c2c05277bcade7: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:39898 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47064 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56118 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56124 - "GET / HTTP/1.0" 200 OK 2025-09-26 21:21:11,638 - lava_callback - INFO - Received callback for node 68d47190a5c2c05277bcc6fe INFO: 172.21.0.1:35210 - "POST /node/68d47190a5c2c05277bcc6fe HTTP/1.0" 202 Accepted 2025-09-26 21:21:11,645 - lava_callback - INFO - Starting async job submit for node 68d47190a5c2c05277bcc6fe 2025-09-26 21:21:11,687 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 21:21:11,728 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d47190a5c2c05277bcc6fe HTTP/1.1" 200 None 2025-09-26 21:21:11,728 - lava_callback - DEBUG - Processing node 68d47190a5c2c05277bcc6fe: baseline-arm64 2025-09-26 21:21:12,642 - lava_callback - ERROR - Error processing callback for node 68d47190a5c2c05277bcc6fe: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:35220 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35224 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37284 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34280 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34288 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41008 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43144 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38354 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38370 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59434 - "GET /api/metrics HTTP/1.0" 200 OK 2025-09-26 21:22:16,310 - lava_callback - INFO - Received callback for node 68d4ff67ef0c6c1c702063e9 INFO: 172.21.0.1:59436 - "POST /node/68d4ff67ef0c6c1c702063e9 HTTP/1.0" 202 Accepted 2025-09-26 21:22:16,316 - lava_callback - INFO - Starting async job submit for node 68d4ff67ef0c6c1c702063e9 2025-09-26 21:22:16,347 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 21:22:16,377 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4ff67ef0c6c1c702063e9 HTTP/1.1" 200 None 2025-09-26 21:22:16,377 - lava_callback - DEBUG - Processing node 68d4ff67ef0c6c1c702063e9: baseline-arm INFO: 172.21.0.1:59448 - "GET / HTTP/1.0" 200 OK 2025-09-26 21:22:16,607 - lava_callback - ERROR - Error processing callback for node 68d4ff67ef0c6c1c702063e9: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None 2025-09-26 21:22:25,103 - lava_callback - INFO - Received callback for node 68d47193a5c2c05277bcc701 INFO: 172.21.0.1:52512 - "POST /node/68d47193a5c2c05277bcc701 HTTP/1.0" 202 Accepted 2025-09-26 21:22:25,112 - lava_callback - INFO - Starting async job submit for node 68d47193a5c2c05277bcc701 2025-09-26 21:22:25,159 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 21:22:25,197 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d47193a5c2c05277bcc701 HTTP/1.1" 200 None 2025-09-26 21:22:25,197 - lava_callback - DEBUG - Processing node 68d47193a5c2c05277bcc701: baseline-arm64 2025-09-26 21:22:26,062 - lava_callback - ERROR - Error processing callback for node 68d47193a5c2c05277bcc701: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:52522 - "GET / HTTP/1.0" 200 OK 2025-09-26 21:22:26,653 - lava_callback - INFO - Received callback for node 68d47198a5c2c05277bcc704 INFO: 172.21.0.1:52526 - "POST /node/68d47198a5c2c05277bcc704 HTTP/1.0" 202 Accepted 2025-09-26 21:22:26,660 - lava_callback - INFO - Starting async job submit for node 68d47198a5c2c05277bcc704 2025-09-26 21:22:26,701 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 21:22:26,714 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d47198a5c2c05277bcc704 HTTP/1.1" 200 None 2025-09-26 21:22:26,714 - lava_callback - DEBUG - Processing node 68d47198a5c2c05277bcc704: baseline-arm64 2025-09-26 21:22:27,568 - lava_callback - ERROR - Error processing callback for node 68d47198a5c2c05277bcc704: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:60574 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60584 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34872 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38642 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47472 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47474 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47178 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47192 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35826 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59134 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59140 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58874 - "GET / HTTP/1.0" 200 OK 2025-09-26 21:23:55,029 - lava_callback - INFO - Received callback for node 68d47269a5c2c05277bcc73b INFO: 172.21.0.1:40020 - "POST /node/68d47269a5c2c05277bcc73b HTTP/1.0" 202 Accepted 2025-09-26 21:23:55,036 - lava_callback - INFO - Starting async job submit for node 68d47269a5c2c05277bcc73b 2025-09-26 21:23:55,056 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 21:23:55,068 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d47269a5c2c05277bcc73b HTTP/1.1" 200 None 2025-09-26 21:23:55,069 - lava_callback - DEBUG - Processing node 68d47269a5c2c05277bcc73b: baseline-arm64 2025-09-26 21:23:55,385 - lava_callback - ERROR - Error processing callback for node 68d47269a5c2c05277bcc73b: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:40022 - "GET / HTTP/1.0" 200 OK 2025-09-26 21:23:58,973 - lava_callback - INFO - Received callback for node 68d4726ba5c2c05277bcc73e INFO: 172.21.0.1:40036 - "POST /node/68d4726ba5c2c05277bcc73e HTTP/1.0" 202 Accepted 2025-09-26 21:23:58,981 - lava_callback - INFO - Starting async job submit for node 68d4726ba5c2c05277bcc73e 2025-09-26 21:23:59,000 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 21:23:59,013 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4726ba5c2c05277bcc73e HTTP/1.1" 200 None 2025-09-26 21:23:59,014 - lava_callback - DEBUG - Processing node 68d4726ba5c2c05277bcc73e: baseline-arm64 2025-09-26 21:24:00,411 - lava_callback - ERROR - Error processing callback for node 68d4726ba5c2c05277bcc73e: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None 2025-09-26 21:24:04,038 - lava_callback - INFO - Received callback for node 68d6f0b03c94e462c6882416 INFO: 172.21.0.1:40968 - "POST /node/68d6f0b03c94e462c6882416 HTTP/1.0" 202 Accepted 2025-09-26 21:24:04,046 - lava_callback - INFO - Starting async job submit for node 68d6f0b03c94e462c6882416 2025-09-26 21:24:04,088 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 21:24:04,102 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6f0b03c94e462c6882416 HTTP/1.1" 200 None 2025-09-26 21:24:04,102 - lava_callback - DEBUG - Processing node 68d6f0b03c94e462c6882416: baseline-arm64 2025-09-26 21:24:05,190 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 21:24:05,358 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 21:24:05,359 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6f0b03c94e462c6882416/log.txt.gz 2025-09-26 21:24:05,381 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 21:24:05,561 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 21:24:05,562 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6f0b03c94e462c6882416/lava_callback.json.gz DEBUG: Setup failed for 68d6f0b03c94e462c6882416. Transitting job node result: incomplete -> fail 2025-09-26 21:24:05,646 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 21:24:05,668 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6f0b03c94e462c6882416 HTTP/1.1" 200 None 2025-09-26 21:24:05,669 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 21:24:05,682 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d651ddcca86fb17472aa24 HTTP/1.1" 200 None 2025-09-26 21:24:05,683 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 21:24:05,714 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6f0b03c94e462c6882416 HTTP/1.1" 200 None 2025-09-26 21:24:05,715 - lava_callback - INFO - Completed processing callback for node 68d6f0b03c94e462c6882416 INFO: 172.21.0.1:40970 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40986 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60780 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60788 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42144 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56848 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56852 - "GET / HTTP/1.0" 200 OK 2025-09-26 21:24:43,760 - lava_callback - INFO - Received callback for node 68d6f0b13c94e462c6882419 INFO: 172.21.0.1:40820 - "POST /node/68d6f0b13c94e462c6882419 HTTP/1.0" 202 Accepted 2025-09-26 21:24:43,767 - lava_callback - INFO - Starting async job submit for node 68d6f0b13c94e462c6882419 2025-09-26 21:24:43,808 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 21:24:43,823 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6f0b13c94e462c6882419 HTTP/1.1" 200 None 2025-09-26 21:24:43,824 - lava_callback - DEBUG - Processing node 68d6f0b13c94e462c6882419: baseline-arm64 2025-09-26 21:24:44,889 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 21:24:45,050 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 21:24:45,051 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6f0b13c94e462c6882419/log.txt.gz 2025-09-26 21:24:45,071 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 21:24:45,242 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 21:24:45,243 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6f0b13c94e462c6882419/lava_callback.json.gz DEBUG: Setup failed for 68d6f0b13c94e462c6882419. Transitting job node result: incomplete -> fail 2025-09-26 21:24:45,327 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 21:24:45,372 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6f0b13c94e462c6882419 HTTP/1.1" 200 None 2025-09-26 21:24:45,373 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 21:24:45,407 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d651ddcca86fb17472aa24 HTTP/1.1" 200 None 2025-09-26 21:24:45,408 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 21:24:45,437 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6f0b13c94e462c6882419 HTTP/1.1" 200 None 2025-09-26 21:24:45,439 - lava_callback - INFO - Completed processing callback for node 68d6f0b13c94e462c6882419 INFO: 172.21.0.1:40836 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52804 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48948 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48958 - "GET / HTTP/1.0" 200 OK 2025-09-26 21:25:11,530 - lava_callback - INFO - Received callback for node 68d4726fa5c2c05277bcc741 INFO: 172.21.0.1:37324 - "POST /node/68d4726fa5c2c05277bcc741 HTTP/1.0" 202 Accepted 2025-09-26 21:25:11,537 - lava_callback - INFO - Starting async job submit for node 68d4726fa5c2c05277bcc741 2025-09-26 21:25:11,558 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 21:25:11,580 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4726fa5c2c05277bcc741 HTTP/1.1" 200 None 2025-09-26 21:25:11,580 - lava_callback - DEBUG - Processing node 68d4726fa5c2c05277bcc741: baseline-arm64 2025-09-26 21:25:11,916 - lava_callback - ERROR - Error processing callback for node 68d4726fa5c2c05277bcc741: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:37340 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37352 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34962 - "GET / HTTP/1.0" 200 OK 2025-09-26 21:25:29,143 - lava_callback - INFO - Received callback for node 68d47272a5c2c05277bcc744 INFO: 172.21.0.1:34966 - "POST /node/68d47272a5c2c05277bcc744 HTTP/1.0" 202 Accepted 2025-09-26 21:25:29,149 - lava_callback - INFO - Starting async job submit for node 68d47272a5c2c05277bcc744 2025-09-26 21:25:29,170 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 21:25:29,183 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d47272a5c2c05277bcc744 HTTP/1.1" 200 None 2025-09-26 21:25:29,184 - lava_callback - DEBUG - Processing node 68d47272a5c2c05277bcc744: baseline-arm64 2025-09-26 21:25:29,486 - lava_callback - ERROR - Error processing callback for node 68d47272a5c2c05277bcc744: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None 2025-09-26 21:25:31,619 - lava_callback - INFO - Received callback for node 68d6f1ba3c94e462c6882427 INFO: 172.21.0.1:46486 - "POST /node/68d6f1ba3c94e462c6882427 HTTP/1.0" 202 Accepted 2025-09-26 21:25:31,626 - lava_callback - INFO - Starting async job submit for node 68d6f1ba3c94e462c6882427 2025-09-26 21:25:31,669 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 21:25:31,681 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6f1ba3c94e462c6882427 HTTP/1.1" 200 None 2025-09-26 21:25:31,681 - lava_callback - DEBUG - Processing node 68d6f1ba3c94e462c6882427: baseline-arm64 2025-09-26 21:25:32,825 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 21:25:32,980 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 21:25:32,981 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6f1ba3c94e462c6882427/log.txt.gz 2025-09-26 21:25:33,003 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 21:25:33,205 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 21:25:33,205 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6f1ba3c94e462c6882427/lava_callback.json.gz INFO: 172.21.0.1:46502 - "GET / HTTP/1.0" 200 OK DEBUG: Setup failed for 68d6f1ba3c94e462c6882427. Transitting job node result: incomplete -> fail 2025-09-26 21:25:33,288 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 21:25:33,300 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6f1ba3c94e462c6882427 HTTP/1.1" 200 None 2025-09-26 21:25:33,301 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 21:25:33,313 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d651ddcca86fb17472aa24 HTTP/1.1" 200 None 2025-09-26 21:25:33,314 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 21:25:33,341 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6f1ba3c94e462c6882427 HTTP/1.1" 200 None 2025-09-26 21:25:33,342 - lava_callback - INFO - Completed processing callback for node 68d6f1ba3c94e462c6882427 INFO: 172.21.0.1:46518 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34206 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44038 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46344 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46356 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47492 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47498 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46318 - "GET / HTTP/1.0" 200 OK 2025-09-26 21:26:29,119 - lava_callback - INFO - Received callback for node 68d472e0a5c2c05277bcc7c4 INFO: 172.21.0.1:46322 - "POST /node/68d472e0a5c2c05277bcc7c4 HTTP/1.0" 202 Accepted 2025-09-26 21:26:29,126 - lava_callback - INFO - Starting async job submit for node 68d472e0a5c2c05277bcc7c4 2025-09-26 21:26:29,147 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 21:26:29,180 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d472e0a5c2c05277bcc7c4 HTTP/1.1" 200 None 2025-09-26 21:26:29,181 - lava_callback - DEBUG - Processing node 68d472e0a5c2c05277bcc7c4: baseline-arm64 2025-09-26 21:26:29,492 - lava_callback - ERROR - Error processing callback for node 68d472e0a5c2c05277bcc7c4: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:46808 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46820 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33160 - "GET / HTTP/1.0" 200 OK 2025-09-26 21:26:54,450 - lava_callback - INFO - Received callback for node 68d472e3a5c2c05277bcc7c7 INFO: 172.21.0.1:41352 - "POST /node/68d472e3a5c2c05277bcc7c7 HTTP/1.0" 202 Accepted 2025-09-26 21:26:54,456 - lava_callback - INFO - Starting async job submit for node 68d472e3a5c2c05277bcc7c7 2025-09-26 21:26:54,477 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 21:26:54,489 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d472e3a5c2c05277bcc7c7 HTTP/1.1" 200 None 2025-09-26 21:26:54,489 - lava_callback - DEBUG - Processing node 68d472e3a5c2c05277bcc7c7: baseline-arm64 2025-09-26 21:26:54,787 - lava_callback - ERROR - Error processing callback for node 68d472e3a5c2c05277bcc7c7: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:41362 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37712 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37722 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43990 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43992 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41258 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56626 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56636 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38376 - "GET / HTTP/1.0" 200 OK 2025-09-26 21:27:49,772 - lava_callback - INFO - Received callback for node 68d472e6a5c2c05277bcc7ca INFO: 172.21.0.1:53122 - "POST /node/68d472e6a5c2c05277bcc7ca HTTP/1.0" 202 Accepted 2025-09-26 21:27:49,779 - lava_callback - INFO - Starting async job submit for node 68d472e6a5c2c05277bcc7ca 2025-09-26 21:27:49,799 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 21:27:49,817 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d472e6a5c2c05277bcc7ca HTTP/1.1" 200 None 2025-09-26 21:27:49,817 - lava_callback - DEBUG - Processing node 68d472e6a5c2c05277bcc7ca: baseline-arm64 2025-09-26 21:27:50,146 - lava_callback - ERROR - Error processing callback for node 68d472e6a5c2c05277bcc7ca: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:53124 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58234 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58240 - "GET / HTTP/1.0" 200 OK 2025-09-26 21:28:13,434 - lava_callback - INFO - Received callback for node 68d472e8a5c2c05277bcc7cd INFO: 172.21.0.1:41882 - "POST /node/68d472e8a5c2c05277bcc7cd HTTP/1.0" 202 Accepted 2025-09-26 21:28:13,449 - lava_callback - INFO - Starting async job submit for node 68d472e8a5c2c05277bcc7cd 2025-09-26 21:28:13,521 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 21:28:13,581 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d472e8a5c2c05277bcc7cd HTTP/1.1" 200 None 2025-09-26 21:28:13,582 - lava_callback - DEBUG - Processing node 68d472e8a5c2c05277bcc7cd: baseline-arm64 2025-09-26 21:28:14,619 - lava_callback - ERROR - Error processing callback for node 68d472e8a5c2c05277bcc7cd: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:41884 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41892 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44310 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50882 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50892 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39810 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35376 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48160 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48174 - "GET / HTTP/1.0" 200 OK 2025-09-26 21:29:10,315 - lava_callback - INFO - Received callback for node 68d4739aa5c2c05277bcc892 INFO: 172.21.0.1:35772 - "POST /node/68d4739aa5c2c05277bcc892 HTTP/1.0" 202 Accepted 2025-09-26 21:29:10,321 - lava_callback - INFO - Starting async job submit for node 68d4739aa5c2c05277bcc892 2025-09-26 21:29:10,342 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 21:29:10,354 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4739aa5c2c05277bcc892 HTTP/1.1" 200 None 2025-09-26 21:29:10,355 - lava_callback - DEBUG - Processing node 68d4739aa5c2c05277bcc892: baseline-arm64 2025-09-26 21:29:10,694 - lava_callback - ERROR - Error processing callback for node 68d4739aa5c2c05277bcc892: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:35776 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35786 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58514 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43410 - "GET / HTTP/1.0" 200 OK 2025-09-26 21:29:35,205 - lava_callback - INFO - Received callback for node 68d51735ef0c6c1c702066e6 INFO: 172.21.0.1:43426 - "POST /node/68d51735ef0c6c1c702066e6 HTTP/1.0" 202 Accepted 2025-09-26 21:29:35,213 - lava_callback - INFO - Starting async job submit for node 68d51735ef0c6c1c702066e6 2025-09-26 21:29:35,245 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 21:29:35,258 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d51735ef0c6c1c702066e6 HTTP/1.1" 200 None 2025-09-26 21:29:35,259 - lava_callback - DEBUG - Processing node 68d51735ef0c6c1c702066e6: baseline-arm 2025-09-26 21:29:35,407 - lava_callback - INFO - Received callback for node 68d4739ba5c2c05277bcc895 INFO: 172.21.0.1:43442 - "POST /node/68d4739ba5c2c05277bcc895 HTTP/1.0" 202 Accepted 2025-09-26 21:29:35,426 - lava_callback - INFO - Starting async job submit for node 68d4739ba5c2c05277bcc895 2025-09-26 21:29:35,467 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 21:29:35,550 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4739ba5c2c05277bcc895 HTTP/1.1" 200 None 2025-09-26 21:29:35,566 - lava_callback - DEBUG - Processing node 68d4739ba5c2c05277bcc895: baseline-arm64 2025-09-26 21:29:35,600 - lava_callback - ERROR - Error processing callback for node 68d51735ef0c6c1c702066e6: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None 2025-09-26 21:29:36,017 - lava_callback - ERROR - Error processing callback for node 68d4739ba5c2c05277bcc895: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:43456 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42920 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54530 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38612 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38618 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54358 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54360 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36466 - "GET / HTTP/1.0" 200 OK 2025-09-26 21:30:30,471 - lava_callback - INFO - Received callback for node 68d473d6a5c2c05277bcc8ce INFO: 172.21.0.1:36356 - "POST /node/68d473d6a5c2c05277bcc8ce HTTP/1.0" 202 Accepted 2025-09-26 21:30:30,477 - lava_callback - INFO - Starting async job submit for node 68d473d6a5c2c05277bcc8ce 2025-09-26 21:30:30,497 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 21:30:30,511 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d473d6a5c2c05277bcc8ce HTTP/1.1" 200 None 2025-09-26 21:30:30,512 - lava_callback - DEBUG - Processing node 68d473d6a5c2c05277bcc8ce: baseline-arm64 2025-09-26 21:30:30,840 - lava_callback - ERROR - Error processing callback for node 68d473d6a5c2c05277bcc8ce: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:36364 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36374 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57302 - "GET / HTTP/1.0" 200 OK 2025-09-26 21:30:49,569 - lava_callback - INFO - Received callback for node 68d473d9a5c2c05277bcc8d1 INFO: 172.21.0.1:55052 - "POST /node/68d473d9a5c2c05277bcc8d1 HTTP/1.0" 202 Accepted 2025-09-26 21:30:49,575 - lava_callback - INFO - Starting async job submit for node 68d473d9a5c2c05277bcc8d1 2025-09-26 21:30:49,596 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 21:30:49,611 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d473d9a5c2c05277bcc8d1 HTTP/1.1" 200 None 2025-09-26 21:30:49,612 - lava_callback - DEBUG - Processing node 68d473d9a5c2c05277bcc8d1: baseline-arm64 2025-09-26 21:30:49,974 - lava_callback - ERROR - Error processing callback for node 68d473d9a5c2c05277bcc8d1: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:55066 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46862 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53120 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53134 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53140 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56040 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58546 - "GET / HTTP/1.0" 200 OK 2025-09-26 21:31:36,141 - lava_callback - INFO - Received callback for node 68d51736ef0c6c1c702066e9 INFO: 172.21.0.1:58548 - "POST /node/68d51736ef0c6c1c702066e9 HTTP/1.0" 202 Accepted 2025-09-26 21:31:36,146 - lava_callback - INFO - Starting async job submit for node 68d51736ef0c6c1c702066e9 2025-09-26 21:31:36,167 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 21:31:36,180 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d51736ef0c6c1c702066e9 HTTP/1.1" 200 None 2025-09-26 21:31:36,181 - lava_callback - DEBUG - Processing node 68d51736ef0c6c1c702066e9: baseline-arm 2025-09-26 21:31:36,410 - lava_callback - ERROR - Error processing callback for node 68d51736ef0c6c1c702066e9: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:58554 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55466 - "GET / HTTP/1.0" 200 OK 2025-09-26 21:31:52,559 - lava_callback - INFO - Received callback for node 68d4752ca5c2c05277bcc991 INFO: 172.21.0.1:33270 - "POST /node/68d4752ca5c2c05277bcc991 HTTP/1.0" 202 Accepted 2025-09-26 21:31:52,567 - lava_callback - INFO - Starting async job submit for node 68d4752ca5c2c05277bcc991 2025-09-26 21:31:52,591 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 21:31:52,604 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4752ca5c2c05277bcc991 HTTP/1.1" 200 None 2025-09-26 21:31:52,605 - lava_callback - DEBUG - Processing node 68d4752ca5c2c05277bcc991: baseline-arm64 2025-09-26 21:31:52,904 - lava_callback - ERROR - Error processing callback for node 68d4752ca5c2c05277bcc991: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:33276 - "GET / HTTP/1.0" 200 OK 2025-09-26 21:32:00,466 - lava_callback - INFO - Received callback for node 68d46e48a5c2c05277bcc451 INFO: 172.21.0.1:37652 - "POST /node/68d46e48a5c2c05277bcc451 HTTP/1.0" 202 Accepted 2025-09-26 21:32:00,472 - lava_callback - INFO - Starting async job submit for node 68d46e48a5c2c05277bcc451 2025-09-26 21:32:00,492 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 21:32:00,504 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d46e48a5c2c05277bcc451 HTTP/1.1" 200 None 2025-09-26 21:32:00,505 - lava_callback - DEBUG - Processing node 68d46e48a5c2c05277bcc451: baseline-arm-broonie 2025-09-26 21:32:00,713 - lava_callback - ERROR - Error processing callback for node 68d46e48a5c2c05277bcc451: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:37664 - "GET / HTTP/1.0" 200 OK 2025-09-26 21:32:07,954 - lava_callback - INFO - Received callback for node 68d4752fa5c2c05277bcc994 INFO: 172.21.0.1:37678 - "POST /node/68d4752fa5c2c05277bcc994 HTTP/1.0" 202 Accepted 2025-09-26 21:32:07,960 - lava_callback - INFO - Starting async job submit for node 68d4752fa5c2c05277bcc994 2025-09-26 21:32:07,998 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 21:32:08,019 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4752fa5c2c05277bcc994 HTTP/1.1" 200 None 2025-09-26 21:32:08,019 - lava_callback - DEBUG - Processing node 68d4752fa5c2c05277bcc994: baseline-arm64 2025-09-26 21:32:08,326 - lava_callback - ERROR - Error processing callback for node 68d4752fa5c2c05277bcc994: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:40146 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40162 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40178 - "GET / HTTP/1.0" 200 OK 2025-09-26 21:32:23,347 - lava_callback - INFO - Received callback for node 68d6d4873c94e462c6881e18 INFO: 172.21.0.1:34352 - "POST /node/68d6d4873c94e462c6881e18 HTTP/1.0" 202 Accepted 2025-09-26 21:32:23,353 - lava_callback - INFO - Starting async job submit for node 68d6d4873c94e462c6881e18 2025-09-26 21:32:23,374 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 21:32:23,387 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6d4873c94e462c6881e18 HTTP/1.1" 200 None 2025-09-26 21:32:23,387 - lava_callback - DEBUG - Processing node 68d6d4873c94e462c6881e18: baseline-arm64-broonie 2025-09-26 21:32:23,591 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 21:32:23,723 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 21:32:23,723 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-68d6d4873c94e462c6881e18/log.txt.gz 2025-09-26 21:32:23,729 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 21:32:23,889 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 21:32:23,890 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-68d6d4873c94e462c6881e18/lava_callback.json.gz 2025-09-26 21:32:23,912 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 21:32:23,924 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6d4873c94e462c6881e18 HTTP/1.1" 200 None 2025-09-26 21:32:23,925 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 21:32:23,937 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6bd8e3c94e462c68814f1 HTTP/1.1" 200 None 2025-09-26 21:32:23,938 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 21:32:23,968 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6d4873c94e462c6881e18 HTTP/1.1" 200 None 2025-09-26 21:32:23,969 - lava_callback - INFO - Completed processing callback for node 68d6d4873c94e462c6881e18 INFO: 172.21.0.1:34356 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44744 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44758 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42008 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47220 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32892 - "GET / HTTP/1.0" 200 OK 2025-09-26 21:33:08,429 - lava_callback - INFO - Received callback for node 68d47534a5c2c05277bcc997 INFO: 172.21.0.1:32898 - "POST /node/68d47534a5c2c05277bcc997 HTTP/1.0" 202 Accepted 2025-09-26 21:33:08,435 - lava_callback - INFO - Starting async job submit for node 68d47534a5c2c05277bcc997 2025-09-26 21:33:08,456 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 21:33:08,488 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d47534a5c2c05277bcc997 HTTP/1.1" 200 None 2025-09-26 21:33:08,488 - lava_callback - DEBUG - Processing node 68d47534a5c2c05277bcc997: baseline-arm64 2025-09-26 21:33:08,819 - lava_callback - ERROR - Error processing callback for node 68d47534a5c2c05277bcc997: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:51160 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51168 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51170 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33488 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56382 - "GET / HTTP/1.0" 200 OK 2025-09-26 21:33:34,784 - lava_callback - INFO - Received callback for node 68d47535a5c2c05277bcc99a INFO: 172.21.0.1:56390 - "POST /node/68d47535a5c2c05277bcc99a HTTP/1.0" 202 Accepted 2025-09-26 21:33:34,791 - lava_callback - INFO - Starting async job submit for node 68d47535a5c2c05277bcc99a 2025-09-26 21:33:34,827 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 21:33:34,840 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d47535a5c2c05277bcc99a HTTP/1.1" 200 None 2025-09-26 21:33:34,840 - lava_callback - DEBUG - Processing node 68d47535a5c2c05277bcc99a: baseline-arm64 2025-09-26 21:33:35,287 - lava_callback - ERROR - Error processing callback for node 68d47535a5c2c05277bcc99a: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:56396 - "GET / HTTP/1.0" 200 OK 2025-09-26 21:33:38,092 - lava_callback - INFO - Received callback for node 68d51737ef0c6c1c702066ec INFO: 172.21.0.1:56400 - "POST /node/68d51737ef0c6c1c702066ec HTTP/1.0" 202 Accepted 2025-09-26 21:33:38,098 - lava_callback - INFO - Starting async job submit for node 68d51737ef0c6c1c702066ec 2025-09-26 21:33:38,119 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 21:33:38,138 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d51737ef0c6c1c702066ec HTTP/1.1" 200 None 2025-09-26 21:33:38,138 - lava_callback - DEBUG - Processing node 68d51737ef0c6c1c702066ec: baseline-arm 2025-09-26 21:33:38,381 - lava_callback - ERROR - Error processing callback for node 68d51737ef0c6c1c702066ec: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None 2025-09-26 21:33:42,425 - lava_callback - INFO - Received callback for node 68d4cabfa5c2c05277bd05cb INFO: 172.21.0.1:42380 - "POST /node/68d4cabfa5c2c05277bd05cb HTTP/1.0" 202 Accepted 2025-09-26 21:33:42,431 - lava_callback - INFO - Starting async job submit for node 68d4cabfa5c2c05277bd05cb 2025-09-26 21:33:42,447 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 21:33:42,459 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4cabfa5c2c05277bd05cb HTTP/1.1" 200 None 2025-09-26 21:33:42,459 - lava_callback - DEBUG - Processing node 68d4cabfa5c2c05277bd05cb: baseline-arm64-broonie 2025-09-26 21:33:42,774 - lava_callback - ERROR - Error processing callback for node 68d4cabfa5c2c05277bd05cb: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:42394 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37576 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56184 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40142 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40156 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40158 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54526 - "GET / HTTP/1.0" 200 OK 2025-09-26 21:34:29,222 - lava_callback - INFO - Received callback for node 68d475b7a5c2c05277bcca21 INFO: 172.21.0.1:54542 - "POST /node/68d475b7a5c2c05277bcca21 HTTP/1.0" 202 Accepted 2025-09-26 21:34:29,229 - lava_callback - INFO - Starting async job submit for node 68d475b7a5c2c05277bcca21 2025-09-26 21:34:29,250 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 21:34:29,264 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d475b7a5c2c05277bcca21 HTTP/1.1" 200 None 2025-09-26 21:34:29,265 - lava_callback - DEBUG - Processing node 68d475b7a5c2c05277bcca21: baseline-arm64 2025-09-26 21:34:29,573 - lava_callback - ERROR - Error processing callback for node 68d475b7a5c2c05277bcca21: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:58896 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58898 - "GET / HTTP/1.0" 200 OK 2025-09-26 21:34:40,537 - lava_callback - INFO - Received callback for node 68d46e41a5c2c05277bcc448 INFO: 172.21.0.1:44624 - "POST /node/68d46e41a5c2c05277bcc448 HTTP/1.0" 202 Accepted 2025-09-26 21:34:40,544 - lava_callback - INFO - Starting async job submit for node 68d46e41a5c2c05277bcc448 2025-09-26 21:34:40,566 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 21:34:40,579 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d46e41a5c2c05277bcc448 HTTP/1.1" 200 None 2025-09-26 21:34:40,579 - lava_callback - DEBUG - Processing node 68d46e41a5c2c05277bcc448: baseline-arm-broonie 2025-09-26 21:34:40,768 - lava_callback - ERROR - Error processing callback for node 68d46e41a5c2c05277bcc448: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:44636 - "GET / HTTP/1.0" 200 OK 2025-09-26 21:34:49,003 - lava_callback - INFO - Received callback for node 68d475baa5c2c05277bcca7d INFO: 172.21.0.1:44640 - "POST /node/68d475baa5c2c05277bcca7d HTTP/1.0" 202 Accepted 2025-09-26 21:34:49,009 - lava_callback - INFO - Starting async job submit for node 68d475baa5c2c05277bcca7d 2025-09-26 21:34:49,029 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 21:34:49,052 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d475baa5c2c05277bcca7d HTTP/1.1" 200 None 2025-09-26 21:34:49,053 - lava_callback - DEBUG - Processing node 68d475baa5c2c05277bcca7d: baseline-arm64 2025-09-26 21:34:49,375 - lava_callback - ERROR - Error processing callback for node 68d475baa5c2c05277bcca7d: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:43364 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57852 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54476 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54488 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54496 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53964 - "GET / HTTP/1.0" 200 OK 2025-09-26 21:35:31,007 - lava_callback - INFO - Received callback for node 68d51738ef0c6c1c702066ef INFO: 172.21.0.1:44256 - "POST /node/68d51738ef0c6c1c702066ef HTTP/1.0" 202 Accepted 2025-09-26 21:35:31,013 - lava_callback - INFO - Starting async job submit for node 68d51738ef0c6c1c702066ef 2025-09-26 21:35:31,033 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 21:35:31,085 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d51738ef0c6c1c702066ef HTTP/1.1" 200 None 2025-09-26 21:35:31,086 - lava_callback - DEBUG - Processing node 68d51738ef0c6c1c702066ef: baseline-arm 2025-09-26 21:35:31,312 - lava_callback - ERROR - Error processing callback for node 68d51738ef0c6c1c702066ef: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:44268 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44270 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54248 - "GET / HTTP/1.0" 200 OK 2025-09-26 21:35:49,413 - lava_callback - INFO - Received callback for node 68d475bda5c2c05277bcca80 INFO: 172.21.0.1:59112 - "POST /node/68d475bda5c2c05277bcca80 HTTP/1.0" 202 Accepted 2025-09-26 21:35:49,419 - lava_callback - INFO - Starting async job submit for node 68d475bda5c2c05277bcca80 2025-09-26 21:35:49,439 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 21:35:49,472 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d475bda5c2c05277bcca80 HTTP/1.1" 200 None 2025-09-26 21:35:49,473 - lava_callback - DEBUG - Processing node 68d475bda5c2c05277bcca80: baseline-arm64 2025-09-26 21:35:49,794 - lava_callback - ERROR - Error processing callback for node 68d475bda5c2c05277bcca80: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:59122 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53270 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38206 - "GET / HTTP/1.0" 200 OK 2025-09-26 21:36:11,616 - lava_callback - INFO - Received callback for node 68d475c0a5c2c05277bcca83 INFO: 172.21.0.1:38210 - "POST /node/68d475c0a5c2c05277bcca83 HTTP/1.0" 202 Accepted 2025-09-26 21:36:11,622 - lava_callback - INFO - Starting async job submit for node 68d475c0a5c2c05277bcca83 2025-09-26 21:36:11,643 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 21:36:11,666 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d475c0a5c2c05277bcca83 HTTP/1.1" 200 None 2025-09-26 21:36:11,666 - lava_callback - DEBUG - Processing node 68d475c0a5c2c05277bcca83: baseline-arm64 2025-09-26 21:36:11,960 - lava_callback - ERROR - Error processing callback for node 68d475c0a5c2c05277bcca83: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:38224 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38232 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51364 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34954 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34956 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51022 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46322 - "GET / HTTP/1.0" 200 OK 2025-09-26 21:37:03,560 - lava_callback - INFO - Received callback for node 68d4cf75a5c2c05277bd0ae3 INFO: 172.21.0.1:55910 - "POST /node/68d4cf75a5c2c05277bd0ae3 HTTP/1.0" 202 Accepted 2025-09-26 21:37:03,566 - lava_callback - INFO - Starting async job submit for node 68d4cf75a5c2c05277bd0ae3 2025-09-26 21:37:03,583 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 21:37:03,595 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4cf75a5c2c05277bd0ae3 HTTP/1.1" 200 None 2025-09-26 21:37:03,596 - lava_callback - DEBUG - Processing node 68d4cf75a5c2c05277bd0ae3: baseline-arm64-broonie 2025-09-26 21:37:03,901 - lava_callback - ERROR - Error processing callback for node 68d4cf75a5c2c05277bd0ae3: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:55926 - "GET / HTTP/1.0" 200 OK 2025-09-26 21:37:10,200 - lava_callback - INFO - Received callback for node 68d478e6a5c2c05277bcce5e INFO: 172.21.0.1:54146 - "POST /node/68d478e6a5c2c05277bcce5e HTTP/1.0" 202 Accepted 2025-09-26 21:37:10,206 - lava_callback - INFO - Starting async job submit for node 68d478e6a5c2c05277bcce5e 2025-09-26 21:37:10,227 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 21:37:10,251 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d478e6a5c2c05277bcce5e HTTP/1.1" 200 None 2025-09-26 21:37:10,252 - lava_callback - DEBUG - Processing node 68d478e6a5c2c05277bcce5e: baseline-arm64 2025-09-26 21:37:10,602 - lava_callback - ERROR - Error processing callback for node 68d478e6a5c2c05277bcce5e: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:54160 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54168 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54172 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33644 - "GET / HTTP/1.0" 200 OK 2025-09-26 21:37:29,929 - lava_callback - INFO - Received callback for node 68d478e8a5c2c05277bcce61 INFO: 172.21.0.1:52292 - "POST /node/68d478e8a5c2c05277bcce61 HTTP/1.0" 202 Accepted 2025-09-26 21:37:29,935 - lava_callback - INFO - Starting async job submit for node 68d478e8a5c2c05277bcce61 2025-09-26 21:37:29,955 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 21:37:29,969 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d478e8a5c2c05277bcce61 HTTP/1.1" 200 None 2025-09-26 21:37:29,969 - lava_callback - DEBUG - Processing node 68d478e8a5c2c05277bcce61: baseline-arm64 2025-09-26 21:37:30,307 - lava_callback - ERROR - Error processing callback for node 68d478e8a5c2c05277bcce61: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None 2025-09-26 21:37:30,754 - lava_callback - INFO - Received callback for node 68d51739ef0c6c1c702066f2 INFO: 172.21.0.1:52300 - "POST /node/68d51739ef0c6c1c702066f2 HTTP/1.0" 202 Accepted 2025-09-26 21:37:30,760 - lava_callback - INFO - Starting async job submit for node 68d51739ef0c6c1c702066f2 2025-09-26 21:37:30,781 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 21:37:30,805 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d51739ef0c6c1c702066f2 HTTP/1.1" 200 None 2025-09-26 21:37:30,805 - lava_callback - DEBUG - Processing node 68d51739ef0c6c1c702066f2: baseline-arm 2025-09-26 21:37:31,030 - lava_callback - ERROR - Error processing callback for node 68d51739ef0c6c1c702066f2: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:52304 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52316 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60234 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33704 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46884 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45128 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45138 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45154 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58928 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59090 - "GET / HTTP/1.0" 200 OK 2025-09-26 21:38:33,689 - lava_callback - INFO - Received callback for node 68d478fca5c2c05277bcce91 INFO: 172.21.0.1:59094 - "POST /node/68d478fca5c2c05277bcce91 HTTP/1.0" 202 Accepted 2025-09-26 21:38:33,695 - lava_callback - INFO - Starting async job submit for node 68d478fca5c2c05277bcce91 2025-09-26 21:38:33,715 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 21:38:33,729 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d478fca5c2c05277bcce91 HTTP/1.1" 200 None 2025-09-26 21:38:33,730 - lava_callback - DEBUG - Processing node 68d478fca5c2c05277bcce91: baseline-arm64 2025-09-26 21:38:34,050 - lava_callback - ERROR - Error processing callback for node 68d478fca5c2c05277bcce91: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:59096 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33594 - "GET / HTTP/1.0" 200 OK 2025-09-26 21:38:49,176 - lava_callback - INFO - Received callback for node 68d478ffa5c2c05277bcce94 INFO: 172.21.0.1:33598 - "POST /node/68d478ffa5c2c05277bcce94 HTTP/1.0" 202 Accepted 2025-09-26 21:38:49,183 - lava_callback - INFO - Starting async job submit for node 68d478ffa5c2c05277bcce94 2025-09-26 21:38:49,203 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 21:38:49,215 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d478ffa5c2c05277bcce94 HTTP/1.1" 200 None 2025-09-26 21:38:49,215 - lava_callback - DEBUG - Processing node 68d478ffa5c2c05277bcce94: baseline-arm64 2025-09-26 21:38:49,514 - lava_callback - ERROR - Error processing callback for node 68d478ffa5c2c05277bcce94: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None 2025-09-26 21:38:49,607 - lava_callback - INFO - Received callback for node 68d6f1bb3c94e462c688242a INFO: 172.21.0.1:37836 - "POST /node/68d6f1bb3c94e462c688242a HTTP/1.0" 202 Accepted 2025-09-26 21:38:49,613 - lava_callback - INFO - Starting async job submit for node 68d6f1bb3c94e462c688242a 2025-09-26 21:38:49,649 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 21:38:49,680 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6f1bb3c94e462c688242a HTTP/1.1" 200 None 2025-09-26 21:38:49,681 - lava_callback - DEBUG - Processing node 68d6f1bb3c94e462c688242a: baseline-arm64 2025-09-26 21:38:50,076 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 21:38:50,214 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 21:38:50,215 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6f1bb3c94e462c688242a/log.txt.gz 2025-09-26 21:38:50,223 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 21:38:50,425 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 21:38:50,426 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d6f1bb3c94e462c688242a/lava_callback.json.gz DEBUG: Setup failed for 68d6f1bb3c94e462c688242a. Transitting job node result: incomplete -> fail 2025-09-26 21:38:50,496 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 21:38:50,509 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6f1bb3c94e462c688242a HTTP/1.1" 200 None 2025-09-26 21:38:50,510 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 21:38:50,521 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d651ddcca86fb17472aa24 HTTP/1.1" 200 None 2025-09-26 21:38:50,522 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 21:38:50,549 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6f1bb3c94e462c688242a HTTP/1.1" 200 None 2025-09-26 21:38:50,550 - lava_callback - INFO - Completed processing callback for node 68d6f1bb3c94e462c688242a INFO: 172.21.0.1:37842 - "GET / HTTP/1.0" 200 OK 2025-09-26 21:39:00,868 - lava_callback - INFO - Received callback for node 68d46e59a5c2c05277bcc463 INFO: 172.21.0.1:34106 - "POST /node/68d46e59a5c2c05277bcc463 HTTP/1.0" 202 Accepted 2025-09-26 21:39:00,875 - lava_callback - INFO - Starting async job submit for node 68d46e59a5c2c05277bcc463 2025-09-26 21:39:00,900 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 21:39:00,915 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d46e59a5c2c05277bcc463 HTTP/1.1" 200 None 2025-09-26 21:39:00,915 - lava_callback - DEBUG - Processing node 68d46e59a5c2c05277bcc463: ltp-cap-bounds 2025-09-26 21:39:01,237 - lava_callback - ERROR - Error processing callback for node 68d46e59a5c2c05277bcc463: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:34122 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45148 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45162 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45178 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35130 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55460 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55474 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49268 - "GET / HTTP/1.0" 200 OK 2025-09-26 21:39:54,835 - lava_callback - INFO - Received callback for node 68d47907a5c2c05277bcce9d INFO: 172.21.0.1:47124 - "POST /node/68d47907a5c2c05277bcce9d HTTP/1.0" 202 Accepted 2025-09-26 21:39:54,841 - lava_callback - INFO - Starting async job submit for node 68d47907a5c2c05277bcce9d 2025-09-26 21:39:54,861 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 21:39:54,895 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d47907a5c2c05277bcce9d HTTP/1.1" 200 None 2025-09-26 21:39:54,895 - lava_callback - DEBUG - Processing node 68d47907a5c2c05277bcce9d: baseline-arm64 2025-09-26 21:39:55,231 - lava_callback - ERROR - Error processing callback for node 68d47907a5c2c05277bcce9d: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:47128 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37726 - "GET / HTTP/1.0" 200 OK 2025-09-26 21:40:09,063 - lava_callback - INFO - Received callback for node 68d47909a5c2c05277bccea0 INFO: 172.21.0.1:37736 - "POST /node/68d47909a5c2c05277bccea0 HTTP/1.0" 202 Accepted 2025-09-26 21:40:09,069 - lava_callback - INFO - Starting async job submit for node 68d47909a5c2c05277bccea0 2025-09-26 21:40:09,090 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 21:40:09,113 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d47909a5c2c05277bccea0 HTTP/1.1" 200 None 2025-09-26 21:40:09,113 - lava_callback - DEBUG - Processing node 68d47909a5c2c05277bccea0: baseline-arm64 2025-09-26 21:40:09,423 - lava_callback - ERROR - Error processing callback for node 68d47909a5c2c05277bccea0: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:55886 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55888 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55904 - "GET / HTTP/1.0" 200 OK 2025-09-26 21:40:22,567 - lava_callback - INFO - Received callback for node 68d4d32aa5c2c05277bd0b3e INFO: 172.21.0.1:45924 - "POST /node/68d4d32aa5c2c05277bd0b3e HTTP/1.0" 202 Accepted 2025-09-26 21:40:22,573 - lava_callback - INFO - Starting async job submit for node 68d4d32aa5c2c05277bd0b3e 2025-09-26 21:40:22,590 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 21:40:22,607 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4d32aa5c2c05277bd0b3e HTTP/1.1" 200 None 2025-09-26 21:40:22,607 - lava_callback - DEBUG - Processing node 68d4d32aa5c2c05277bd0b3e: baseline-arm64-broonie 2025-09-26 21:40:22,917 - lava_callback - ERROR - Error processing callback for node 68d4d32aa5c2c05277bd0b3e: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None 2025-09-26 21:40:25,152 - lava_callback - INFO - Received callback for node 68d6c78e3c94e462c68818bc INFO: 172.21.0.1:45940 - "POST /node/68d6c78e3c94e462c68818bc HTTP/1.0" 202 Accepted 2025-09-26 21:40:25,159 - lava_callback - INFO - Starting async job submit for node 68d6c78e3c94e462c68818bc 2025-09-26 21:40:25,193 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 21:40:25,206 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6c78e3c94e462c68818bc HTTP/1.1" 200 None 2025-09-26 21:40:25,206 - lava_callback - DEBUG - Processing node 68d6c78e3c94e462c68818bc: baseline-arm64-broonie 2025-09-26 21:40:25,664 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 21:40:25,816 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 21:40:25,817 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-68d6c78e3c94e462c68818bc/log.txt.gz 2025-09-26 21:40:25,827 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 21:40:25,978 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 21:40:25,979 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-68d6c78e3c94e462c68818bc/lava_callback.json.gz 2025-09-26 21:40:26,014 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 21:40:26,043 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6c78e3c94e462c68818bc HTTP/1.1" 200 None 2025-09-26 21:40:26,044 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 21:40:26,056 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6b9dd2f83206171a92b48 HTTP/1.1" 200 None 2025-09-26 21:40:26,057 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 21:40:26,081 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6c78e3c94e462c68818bc HTTP/1.1" 200 None 2025-09-26 21:40:26,081 - lava_callback - INFO - Completed processing callback for node 68d6c78e3c94e462c68818bc INFO: 172.21.0.1:45948 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40976 - "GET / HTTP/1.0" 200 OK 2025-09-26 21:40:35,721 - lava_callback - INFO - Received callback for node 68d5173aef0c6c1c702066f5 INFO: 172.21.0.1:40992 - "POST /node/68d5173aef0c6c1c702066f5 HTTP/1.0" 202 Accepted 2025-09-26 21:40:35,727 - lava_callback - INFO - Starting async job submit for node 68d5173aef0c6c1c702066f5 2025-09-26 21:40:35,750 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 21:40:35,763 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d5173aef0c6c1c702066f5 HTTP/1.1" 200 None 2025-09-26 21:40:35,763 - lava_callback - DEBUG - Processing node 68d5173aef0c6c1c702066f5: baseline-arm 2025-09-26 21:40:35,997 - lava_callback - ERROR - Error processing callback for node 68d5173aef0c6c1c702066f5: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:40996 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42030 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33354 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46582 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59668 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59670 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59684 - "GET / HTTP/1.0" 200 OK 2025-09-26 21:41:18,389 - lava_callback - INFO - Received callback for node 68d47929a5c2c05277bccec1 INFO: 172.21.0.1:59692 - "POST /node/68d47929a5c2c05277bccec1 HTTP/1.0" 202 Accepted 2025-09-26 21:41:18,395 - lava_callback - INFO - Starting async job submit for node 68d47929a5c2c05277bccec1 2025-09-26 21:41:18,416 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 21:41:18,456 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d47929a5c2c05277bccec1 HTTP/1.1" 200 None 2025-09-26 21:41:18,457 - lava_callback - DEBUG - Processing node 68d47929a5c2c05277bccec1: baseline-arm64 2025-09-26 21:41:18,811 - lava_callback - ERROR - Error processing callback for node 68d47929a5c2c05277bccec1: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:47484 - "GET / HTTP/1.0" 200 OK 2025-09-26 21:41:29,714 - lava_callback - INFO - Received callback for node 68d4792ca5c2c05277bcced0 INFO: 172.21.0.1:45052 - "POST /node/68d4792ca5c2c05277bcced0 HTTP/1.0" 202 Accepted 2025-09-26 21:41:29,720 - lava_callback - INFO - Starting async job submit for node 68d4792ca5c2c05277bcced0 2025-09-26 21:41:29,741 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 21:41:29,775 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4792ca5c2c05277bcced0 HTTP/1.1" 200 None 2025-09-26 21:41:29,775 - lava_callback - DEBUG - Processing node 68d4792ca5c2c05277bcced0: baseline-arm64 2025-09-26 21:41:30,111 - lava_callback - ERROR - Error processing callback for node 68d4792ca5c2c05277bcced0: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None 2025-09-26 21:41:31,009 - lava_callback - INFO - Received callback for node 68d4719aa5c2c05277bcc707 INFO: 172.21.0.1:45064 - "POST /node/68d4719aa5c2c05277bcc707 HTTP/1.0" 202 Accepted 2025-09-26 21:41:31,016 - lava_callback - INFO - Starting async job submit for node 68d4719aa5c2c05277bcc707 2025-09-26 21:41:31,057 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 21:41:31,071 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4719aa5c2c05277bcc707 HTTP/1.1" 200 None 2025-09-26 21:41:31,072 - lava_callback - DEBUG - Processing node 68d4719aa5c2c05277bcc707: baseline-arm64 2025-09-26 21:41:31,868 - lava_callback - ERROR - Error processing callback for node 68d4719aa5c2c05277bcc707: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:45070 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45072 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60510 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50064 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57610 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49188 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49192 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49198 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58784 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59076 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59088 - "GET / HTTP/1.0" 200 OK 2025-09-26 21:42:40,050 - lava_callback - INFO - Received callback for node 68d5173bef0c6c1c702066f8 INFO: 172.21.0.1:60802 - "POST /node/68d5173bef0c6c1c702066f8 HTTP/1.0" 202 Accepted 2025-09-26 21:42:40,056 - lava_callback - INFO - Starting async job submit for node 68d5173bef0c6c1c702066f8 2025-09-26 21:42:40,075 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 21:42:40,110 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d5173bef0c6c1c702066f8 HTTP/1.1" 200 None 2025-09-26 21:42:40,111 - lava_callback - DEBUG - Processing node 68d5173bef0c6c1c702066f8: baseline-arm 2025-09-26 21:42:40,360 - lava_callback - ERROR - Error processing callback for node 68d5173bef0c6c1c702066f8: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:60808 - "GET / HTTP/1.0" 200 OK 2025-09-26 21:42:54,338 - lava_callback - INFO - Received callback for node 68d47a62a5c2c05277bcd130 INFO: 172.21.0.1:39320 - "POST /node/68d47a62a5c2c05277bcd130 HTTP/1.0" 202 Accepted 2025-09-26 21:42:54,344 - lava_callback - INFO - Starting async job submit for node 68d47a62a5c2c05277bcd130 2025-09-26 21:42:54,365 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 21:42:54,398 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d47a62a5c2c05277bcd130 HTTP/1.1" 200 None 2025-09-26 21:42:54,399 - lava_callback - DEBUG - Processing node 68d47a62a5c2c05277bcd130: baseline-arm64 2025-09-26 21:42:54,692 - lava_callback - ERROR - Error processing callback for node 68d47a62a5c2c05277bcd130: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:39322 - "GET / HTTP/1.0" 200 OK 2025-09-26 21:42:57,673 - lava_callback - INFO - Received callback for node 68d47a62a5c2c05277bcd133 INFO: 172.21.0.1:39338 - "POST /node/68d47a62a5c2c05277bcd133 HTTP/1.0" 202 Accepted 2025-09-26 21:42:57,679 - lava_callback - INFO - Starting async job submit for node 68d47a62a5c2c05277bcd133 2025-09-26 21:42:57,700 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 21:42:57,733 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d47a62a5c2c05277bcd133 HTTP/1.1" 200 None 2025-09-26 21:42:57,733 - lava_callback - DEBUG - Processing node 68d47a62a5c2c05277bcd133: baseline-arm64 2025-09-26 21:42:58,029 - lava_callback - ERROR - Error processing callback for node 68d47a62a5c2c05277bcd133: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:54802 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47220 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47234 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47244 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48446 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56698 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56710 - "GET / HTTP/1.0" 200 OK 2025-09-26 21:43:44,585 - lava_callback - INFO - Received callback for node 68d4e861a5c2c05277bd0f14 INFO: 172.21.0.1:36452 - "POST /node/68d4e861a5c2c05277bd0f14 HTTP/1.0" 202 Accepted 2025-09-26 21:43:44,590 - lava_callback - INFO - Starting async job submit for node 68d4e861a5c2c05277bd0f14 2025-09-26 21:43:44,608 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 21:43:44,624 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4e861a5c2c05277bd0f14 HTTP/1.1" 200 None 2025-09-26 21:43:44,624 - lava_callback - DEBUG - Processing node 68d4e861a5c2c05277bd0f14: baseline-arm64-broonie 2025-09-26 21:43:44,939 - lava_callback - ERROR - Error processing callback for node 68d4e861a5c2c05277bd0f14: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:36466 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53582 - "GET / HTTP/1.0" 200 OK 2025-09-26 21:44:01,721 - lava_callback - INFO - Received callback for node 68d47a65a5c2c05277bcd136 INFO: 172.21.0.1:36350 - "POST /node/68d47a65a5c2c05277bcd136 HTTP/1.0" 202 Accepted 2025-09-26 21:44:01,729 - lava_callback - INFO - Starting async job submit for node 68d47a65a5c2c05277bcd136 2025-09-26 21:44:01,747 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 21:44:01,759 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d47a65a5c2c05277bcd136 HTTP/1.1" 200 None 2025-09-26 21:44:01,760 - lava_callback - DEBUG - Processing node 68d47a65a5c2c05277bcd136: baseline-arm64 2025-09-26 21:44:03,074 - lava_callback - ERROR - Error processing callback for node 68d47a65a5c2c05277bcd136: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:36366 - "GET / HTTP/1.0" 200 OK 2025-09-26 21:44:09,337 - lava_callback - INFO - Received callback for node 68d47a67a5c2c05277bcd139 INFO: 172.21.0.1:51376 - "POST /node/68d47a67a5c2c05277bcd139 HTTP/1.0" 202 Accepted 2025-09-26 21:44:09,343 - lava_callback - INFO - Starting async job submit for node 68d47a67a5c2c05277bcd139 2025-09-26 21:44:09,363 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 21:44:09,400 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d47a67a5c2c05277bcd139 HTTP/1.1" 200 None 2025-09-26 21:44:09,400 - lava_callback - DEBUG - Processing node 68d47a67a5c2c05277bcd139: baseline-arm64 2025-09-26 21:44:09,711 - lava_callback - ERROR - Error processing callback for node 68d47a67a5c2c05277bcd139: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:51382 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51390 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51394 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32878 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53468 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53474 - "GET / HTTP/1.0" 200 OK 2025-09-26 21:44:41,256 - lava_callback - INFO - Received callback for node 68d46e5ba5c2c05277bcc466 INFO: 172.21.0.1:51748 - "POST /node/68d46e5ba5c2c05277bcc466 HTTP/1.0" 202 Accepted 2025-09-26 21:44:41,264 - lava_callback - INFO - Starting async job submit for node 68d46e5ba5c2c05277bcc466 2025-09-26 21:44:41,364 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 21:44:41,382 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d46e5ba5c2c05277bcc466 HTTP/1.1" 200 None 2025-09-26 21:44:41,382 - lava_callback - DEBUG - Processing node 68d46e5ba5c2c05277bcc466: ltp-containers 2025-09-26 21:44:42,191 - lava_callback - ERROR - Error processing callback for node 68d46e5ba5c2c05277bcc466: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:51760 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45150 - "GET / HTTP/1.0" 200 OK 2025-09-26 21:44:56,785 - lava_callback - INFO - Received callback for node 68d5173cef0c6c1c702066fb INFO: 172.21.0.1:45156 - "POST /node/68d5173cef0c6c1c702066fb HTTP/1.0" 202 Accepted 2025-09-26 21:44:56,790 - lava_callback - INFO - Starting async job submit for node 68d5173cef0c6c1c702066fb 2025-09-26 21:44:56,811 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 21:44:56,823 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d5173cef0c6c1c702066fb HTTP/1.1" 200 None 2025-09-26 21:44:56,824 - lava_callback - DEBUG - Processing node 68d5173cef0c6c1c702066fb: baseline-arm 2025-09-26 21:44:57,052 - lava_callback - ERROR - Error processing callback for node 68d5173cef0c6c1c702066fb: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:39234 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60704 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60710 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60716 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56190 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40140 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40152 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52222 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55156 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41952 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50984 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50996 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50998 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59274 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54794 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54810 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49710 - "GET / HTTP/1.0" 200 OK 2025-09-26 21:46:55,701 - lava_callback - INFO - Received callback for node 68d5177bef0c6c1c7020676a INFO: 172.21.0.1:42984 - "POST /node/68d5177bef0c6c1c7020676a HTTP/1.0" 202 Accepted 2025-09-26 21:46:55,707 - lava_callback - INFO - Starting async job submit for node 68d5177bef0c6c1c7020676a 2025-09-26 21:46:55,727 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 21:46:55,740 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d5177bef0c6c1c7020676a HTTP/1.1" 200 None 2025-09-26 21:46:55,741 - lava_callback - DEBUG - Processing node 68d5177bef0c6c1c7020676a: baseline-arm 2025-09-26 21:46:55,990 - lava_callback - ERROR - Error processing callback for node 68d5177bef0c6c1c7020676a: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:42998 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35674 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36146 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36156 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36168 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43904 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57918 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57924 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42102 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43708 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37152 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40026 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40032 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40046 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55616 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58426 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58438 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49074 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44124 - "GET / HTTP/1.0" 200 OK 2025-09-26 21:48:57,423 - lava_callback - INFO - Received callback for node 68d5177bef0c6c1c7020676d INFO: 172.21.0.1:44130 - "POST /node/68d5177bef0c6c1c7020676d HTTP/1.0" 202 Accepted 2025-09-26 21:48:57,429 - lava_callback - INFO - Starting async job submit for node 68d5177bef0c6c1c7020676d 2025-09-26 21:48:57,450 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 21:48:57,464 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d5177bef0c6c1c7020676d HTTP/1.1" 200 None 2025-09-26 21:48:57,465 - lava_callback - DEBUG - Processing node 68d5177bef0c6c1c7020676d: baseline-arm 2025-09-26 21:48:57,694 - lava_callback - ERROR - Error processing callback for node 68d5177bef0c6c1c7020676d: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:53026 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48868 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48874 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48888 - "GET / HTTP/1.0" 200 OK 2025-09-26 21:49:24,509 - lava_callback - INFO - Received callback for node 68d46dbea5c2c05277bcc3a2 INFO: 172.21.0.1:33398 - "POST /node/68d46dbea5c2c05277bcc3a2 HTTP/1.0" 202 Accepted 2025-09-26 21:49:24,516 - lava_callback - INFO - Starting async job submit for node 68d46dbea5c2c05277bcc3a2 2025-09-26 21:49:24,544 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 21:49:24,562 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d46dbea5c2c05277bcc3a2 HTTP/1.1" 200 None 2025-09-26 21:49:24,563 - lava_callback - DEBUG - Processing node 68d46dbea5c2c05277bcc3a2: ltp-syscalls-ipc 2025-09-26 21:49:25,375 - lava_callback - ERROR - Error processing callback for node 68d46dbea5c2c05277bcc3a2: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:33410 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33192 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33202 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50600 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56558 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40336 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54860 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54864 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54876 - "GET / HTTP/1.0" 200 OK 2025-09-26 21:50:16,713 - lava_callback - INFO - Received callback for node 68d4678da5c2c05277bcb73d INFO: 172.21.0.1:54886 - "POST /node/68d4678da5c2c05277bcb73d HTTP/1.0" 202 Accepted 2025-09-26 21:50:16,718 - lava_callback - INFO - Starting async job submit for node 68d4678da5c2c05277bcb73d 2025-09-26 21:50:16,719 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 21:50:16,755 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4678da5c2c05277bcb73d HTTP/1.1" 200 None 2025-09-26 21:50:16,755 - lava_callback - DEBUG - Processing node 68d4678da5c2c05277bcb73d: baseline-arm64-baylibre 2025-09-26 21:50:16,755 - lava_callback - ERROR - Error processing callback for node 68d4678da5c2c05277bcb73d: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:38408 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59074 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59086 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40306 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42216 - "GET / HTTP/1.0" 200 OK 2025-09-26 21:50:57,167 - lava_callback - INFO - Received callback for node 68d5177def0c6c1c70206770 INFO: 172.21.0.1:42226 - "POST /node/68d5177def0c6c1c70206770 HTTP/1.0" 202 Accepted 2025-09-26 21:50:57,173 - lava_callback - INFO - Starting async job submit for node 68d5177def0c6c1c70206770 2025-09-26 21:50:57,193 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 21:50:57,206 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d5177def0c6c1c70206770 HTTP/1.1" 200 None 2025-09-26 21:50:57,206 - lava_callback - DEBUG - Processing node 68d5177def0c6c1c70206770: baseline-arm 2025-09-26 21:50:57,438 - lava_callback - ERROR - Error processing callback for node 68d5177def0c6c1c70206770: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:36774 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46796 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46800 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46812 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60010 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50802 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50818 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58272 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57548 - "GET / HTTP/1.0" 200 OK 2025-09-26 21:51:56,769 - lava_callback - INFO - Received callback for node 68d467f4a5c2c05277bcb7a5 INFO: 172.21.0.1:57556 - "POST /node/68d467f4a5c2c05277bcb7a5 HTTP/1.0" 202 Accepted 2025-09-26 21:51:56,774 - lava_callback - INFO - Starting async job submit for node 68d467f4a5c2c05277bcb7a5 2025-09-26 21:51:56,775 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 21:51:56,788 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d467f4a5c2c05277bcb7a5 HTTP/1.1" 200 None 2025-09-26 21:51:56,789 - lava_callback - DEBUG - Processing node 68d467f4a5c2c05277bcb7a5: baseline-arm64-baylibre 2025-09-26 21:51:56,789 - lava_callback - ERROR - Error processing callback for node 68d467f4a5c2c05277bcb7a5: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:50936 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54590 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54602 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54610 - "GET / HTTP/1.0" 200 OK 2025-09-26 21:52:16,759 - lava_callback - INFO - Received callback for node 68d46809a5c2c05277bcb7c5 INFO: 172.21.0.1:54612 - "POST /node/68d46809a5c2c05277bcb7c5 HTTP/1.0" 202 Accepted 2025-09-26 21:52:16,764 - lava_callback - INFO - Starting async job submit for node 68d46809a5c2c05277bcb7c5 2025-09-26 21:52:16,765 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 21:52:16,786 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d46809a5c2c05277bcb7c5 HTTP/1.1" 200 None 2025-09-26 21:52:16,786 - lava_callback - DEBUG - Processing node 68d46809a5c2c05277bcb7c5: baseline-arm64-baylibre 2025-09-26 21:52:16,787 - lava_callback - ERROR - Error processing callback for node 68d46809a5c2c05277bcb7c5: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:52532 - "GET / HTTP/1.0" 200 OK 2025-09-26 21:52:27,055 - lava_callback - INFO - Received callback for node 68d5019cef0c6c1c70206546 INFO: 172.21.0.1:52534 - "POST /node/68d5019cef0c6c1c70206546 HTTP/1.0" 202 Accepted 2025-09-26 21:52:27,060 - lava_callback - INFO - Starting async job submit for node 68d5019cef0c6c1c70206546 2025-09-26 21:52:27,078 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 21:52:27,091 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d5019cef0c6c1c70206546 HTTP/1.1" 200 None 2025-09-26 21:52:27,091 - lava_callback - DEBUG - Processing node 68d5019cef0c6c1c70206546: baseline-arm64-broonie 2025-09-26 21:52:27,404 - lava_callback - ERROR - Error processing callback for node 68d5019cef0c6c1c70206546: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:36062 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36076 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37836 - "GET / HTTP/1.0" 200 OK 2025-09-26 21:52:56,019 - lava_callback - INFO - Received callback for node 68d5177def0c6c1c70206773 INFO: 172.21.0.1:39486 - "POST /node/68d5177def0c6c1c70206773 HTTP/1.0" 202 Accepted 2025-09-26 21:52:56,025 - lava_callback - INFO - Starting async job submit for node 68d5177def0c6c1c70206773 2025-09-26 21:52:56,045 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 21:52:56,058 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d5177def0c6c1c70206773 HTTP/1.1" 200 None 2025-09-26 21:52:56,058 - lava_callback - DEBUG - Processing node 68d5177def0c6c1c70206773: baseline-arm 2025-09-26 21:52:56,290 - lava_callback - ERROR - Error processing callback for node 68d5177def0c6c1c70206773: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:39492 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49632 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53454 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53468 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53482 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41272 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46384 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46392 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54172 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59356 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44312 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48076 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48082 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48098 - "GET / HTTP/1.0" 200 OK 2025-09-26 21:54:20,269 - lava_callback - INFO - Received callback for node 68d46883a5c2c05277bcb84b INFO: 172.21.0.1:59134 - "POST /node/68d46883a5c2c05277bcb84b HTTP/1.0" 202 Accepted 2025-09-26 21:54:20,274 - lava_callback - INFO - Starting async job submit for node 68d46883a5c2c05277bcb84b 2025-09-26 21:54:20,275 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 21:54:20,287 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d46883a5c2c05277bcb84b HTTP/1.1" 200 None 2025-09-26 21:54:20,288 - lava_callback - DEBUG - Processing node 68d46883a5c2c05277bcb84b: baseline-arm 2025-09-26 21:54:20,288 - lava_callback - ERROR - Error processing callback for node 68d46883a5c2c05277bcb84b: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:59150 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53494 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53502 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34030 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39222 - "GET / HTTP/1.0" 200 OK 2025-09-26 21:54:59,420 - lava_callback - INFO - Received callback for node 68d5177fef0c6c1c70206776 INFO: 172.21.0.1:51998 - "POST /node/68d5177fef0c6c1c70206776 HTTP/1.0" 202 Accepted 2025-09-26 21:54:59,426 - lava_callback - INFO - Starting async job submit for node 68d5177fef0c6c1c70206776 2025-09-26 21:54:59,446 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 21:54:59,477 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d5177fef0c6c1c70206776 HTTP/1.1" 200 None 2025-09-26 21:54:59,477 - lava_callback - DEBUG - Processing node 68d5177fef0c6c1c70206776: baseline-arm 2025-09-26 21:54:59,714 - lava_callback - ERROR - Error processing callback for node 68d5177fef0c6c1c70206776: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:52006 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41364 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41368 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41370 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34150 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44356 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44364 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43962 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37784 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55898 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59582 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59588 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59598 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45290 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47342 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47356 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36642 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42330 - "GET / HTTP/1.0" 200 OK 2025-09-26 21:57:04,327 - lava_callback - INFO - Received callback for node 68d4ce44a5c2c05277bd0932 INFO: 172.21.0.1:37534 - "POST /node/68d4ce44a5c2c05277bd0932 HTTP/1.0" 202 Accepted 2025-09-26 21:57:04,334 - lava_callback - INFO - Starting async job submit for node 68d4ce44a5c2c05277bd0932 2025-09-26 21:57:04,356 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 21:57:04,373 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4ce44a5c2c05277bd0932 HTTP/1.1" 200 None 2025-09-26 21:57:04,373 - lava_callback - DEBUG - Processing node 68d4ce44a5c2c05277bd0932: baseline-arm64-broonie 2025-09-26 21:57:04,783 - lava_callback - ERROR - Error processing callback for node 68d4ce44a5c2c05277bd0932: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:37536 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50664 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50668 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50674 - "GET / HTTP/1.0" 200 OK 2025-09-26 21:57:17,024 - lava_callback - INFO - Received callback for node 68d46926a5c2c05277bcb901 INFO: 172.21.0.1:50680 - "POST /node/68d46926a5c2c05277bcb901 HTTP/1.0" 202 Accepted 2025-09-26 21:57:17,030 - lava_callback - INFO - Starting async job submit for node 68d46926a5c2c05277bcb901 2025-09-26 21:57:17,030 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 21:57:17,043 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d46926a5c2c05277bcb901 HTTP/1.1" 200 None 2025-09-26 21:57:17,044 - lava_callback - DEBUG - Processing node 68d46926a5c2c05277bcb901: baseline-arm64-baylibre 2025-09-26 21:57:17,044 - lava_callback - ERROR - Error processing callback for node 68d46926a5c2c05277bcb901: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 21:57:17,064 - lava_callback - INFO - Received callback for node 68d5177fef0c6c1c70206779 INFO: 172.21.0.1:50694 - "POST /node/68d5177fef0c6c1c70206779 HTTP/1.0" 202 Accepted 2025-09-26 21:57:17,070 - lava_callback - INFO - Starting async job submit for node 68d5177fef0c6c1c70206779 2025-09-26 21:57:17,090 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 21:57:17,106 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d5177fef0c6c1c70206779 HTTP/1.1" 200 None 2025-09-26 21:57:17,106 - lava_callback - DEBUG - Processing node 68d5177fef0c6c1c70206779: baseline-arm 2025-09-26 21:57:17,337 - lava_callback - ERROR - Error processing callback for node 68d5177fef0c6c1c70206779: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:39020 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43532 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43542 - "GET / HTTP/1.0" 200 OK 2025-09-26 21:57:37,057 - lava_callback - INFO - Received callback for node 68d46947a5c2c05277bcb930 INFO: 172.21.0.1:43548 - "POST /node/68d46947a5c2c05277bcb930 HTTP/1.0" 202 Accepted 2025-09-26 21:57:37,063 - lava_callback - INFO - Starting async job submit for node 68d46947a5c2c05277bcb930 2025-09-26 21:57:37,063 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 21:57:37,081 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d46947a5c2c05277bcb930 HTTP/1.1" 200 None 2025-09-26 21:57:37,082 - lava_callback - DEBUG - Processing node 68d46947a5c2c05277bcb930: baseline-arm64-baylibre 2025-09-26 21:57:37,082 - lava_callback - ERROR - Error processing callback for node 68d46947a5c2c05277bcb930: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:41356 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38060 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45274 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58280 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58282 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58288 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33602 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44060 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44072 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34830 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43218 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38152 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46816 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46824 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46832 - "GET / HTTP/1.0" 200 OK 2025-09-26 21:59:17,709 - lava_callback - INFO - Received callback for node 68d51781ef0c6c1c7020677c INFO: 172.21.0.1:46836 - "POST /node/68d51781ef0c6c1c7020677c HTTP/1.0" 202 Accepted 2025-09-26 21:59:17,715 - lava_callback - INFO - Starting async job submit for node 68d51781ef0c6c1c7020677c 2025-09-26 21:59:17,736 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 21:59:17,770 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d51781ef0c6c1c7020677c HTTP/1.1" 200 None 2025-09-26 21:59:17,771 - lava_callback - DEBUG - Processing node 68d51781ef0c6c1c7020677c: baseline-arm 2025-09-26 21:59:17,996 - lava_callback - ERROR - Error processing callback for node 68d51781ef0c6c1c7020677c: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:42672 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51404 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51412 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44252 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56114 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52372 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57096 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57102 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57110 - "GET / HTTP/1.0" 200 OK 2025-09-26 22:00:22,576 - lava_callback - INFO - Received callback for node 68d46e5fa5c2c05277bcc469 INFO: 172.21.0.1:60610 - "POST /node/68d46e5fa5c2c05277bcc469 HTTP/1.0" 202 Accepted 2025-09-26 22:00:22,587 - lava_callback - INFO - Starting async job submit for node 68d46e5fa5c2c05277bcc469 2025-09-26 22:00:22,952 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 22:00:22,982 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d46e5fa5c2c05277bcc469 HTTP/1.1" 200 None 2025-09-26 22:00:22,983 - lava_callback - DEBUG - Processing node 68d46e5fa5c2c05277bcc469: ltp-controllers 2025-09-26 22:00:25,814 - lava_callback - ERROR - Error processing callback for node 68d46e5fa5c2c05277bcc469: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:60612 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57010 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57026 - "GET / HTTP/1.0" 200 OK 2025-09-26 22:00:40,726 - lava_callback - INFO - Received callback for node 68d46881a5c2c05277bcb848 INFO: 172.21.0.1:46760 - "POST /node/68d46881a5c2c05277bcb848 HTTP/1.0" 202 Accepted 2025-09-26 22:00:40,732 - lava_callback - INFO - Starting async job submit for node 68d46881a5c2c05277bcb848 2025-09-26 22:00:40,762 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 22:00:40,774 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d46881a5c2c05277bcb848 HTTP/1.1" 200 None 2025-09-26 22:00:40,774 - lava_callback - DEBUG - Processing node 68d46881a5c2c05277bcb848: baseline-arm 2025-09-26 22:00:41,032 - lava_callback - ERROR - Error processing callback for node 68d46881a5c2c05277bcb848: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:46766 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33622 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60130 - "GET / HTTP/1.0" 200 OK 2025-09-26 22:01:11,796 - lava_callback - INFO - Received callback for node 68d51782ef0c6c1c7020677f INFO: 172.21.0.1:42960 - "POST /node/68d51782ef0c6c1c7020677f HTTP/1.0" 202 Accepted 2025-09-26 22:01:11,802 - lava_callback - INFO - Starting async job submit for node 68d51782ef0c6c1c7020677f 2025-09-26 22:01:11,822 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 22:01:11,852 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d51782ef0c6c1c7020677f HTTP/1.1" 200 None 2025-09-26 22:01:11,853 - lava_callback - DEBUG - Processing node 68d51782ef0c6c1c7020677f: baseline-arm 2025-09-26 22:01:12,113 - lava_callback - ERROR - Error processing callback for node 68d51782ef0c6c1c7020677f: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:42972 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42980 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42994 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34538 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48264 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48272 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36544 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37910 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46028 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45528 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45538 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45548 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52016 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36518 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36534 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59440 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58548 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41654 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45496 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45502 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45514 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54278 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42808 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42820 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58756 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60808 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52548 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51540 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51554 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51564 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46774 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44824 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44836 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36200 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59014 - "GET / HTTP/1.0" 200 OK 2025-09-26 22:04:57,449 - lava_callback - INFO - Received callback for node 68d46afca5c2c05277bcc087 INFO: 172.21.0.1:59018 - "POST /node/68d46afca5c2c05277bcc087 HTTP/1.0" 202 Accepted 2025-09-26 22:04:57,454 - lava_callback - INFO - Starting async job submit for node 68d46afca5c2c05277bcc087 2025-09-26 22:04:57,455 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 22:04:57,468 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d46afca5c2c05277bcc087 HTTP/1.1" 200 None 2025-09-26 22:04:57,468 - lava_callback - DEBUG - Processing node 68d46afca5c2c05277bcc087: baseline-arm64-baylibre 2025-09-26 22:04:57,468 - lava_callback - ERROR - Error processing callback for node 68d46afca5c2c05277bcc087: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:59108 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36322 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36334 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36348 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39516 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52988 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52992 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37698 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45376 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38712 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54110 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54120 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54132 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35774 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36906 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36910 - "GET / HTTP/1.0" 200 OK 2025-09-26 22:06:42,072 - lava_callback - INFO - Received callback for node 68d46e61a5c2c05277bcc46c INFO: 172.21.0.1:58902 - "POST /node/68d46e61a5c2c05277bcc46c HTTP/1.0" 202 Accepted 2025-09-26 22:06:42,080 - lava_callback - INFO - Starting async job submit for node 68d46e61a5c2c05277bcc46c 2025-09-26 22:06:42,193 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 22:06:42,205 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d46e61a5c2c05277bcc46c HTTP/1.1" 200 None 2025-09-26 22:06:42,206 - lava_callback - DEBUG - Processing node 68d46e61a5c2c05277bcc46c: ltp-cve 2025-09-26 22:06:42,958 - lava_callback - ERROR - Error processing callback for node 68d46e61a5c2c05277bcc46c: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:58918 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51028 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40340 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38554 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38564 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38572 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56330 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52096 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52102 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51750 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35978 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53526 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39678 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39686 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39700 - "GET / HTTP/1.0" 200 OK 2025-09-26 22:08:24,739 - lava_callback - INFO - Received callback for node 68d486dda5c2c05277bcdd50 INFO: 172.21.0.1:40428 - "POST /node/68d486dda5c2c05277bcdd50 HTTP/1.0" 202 Accepted 2025-09-26 22:08:24,748 - lava_callback - INFO - Starting async job submit for node 68d486dda5c2c05277bcdd50 2025-09-26 22:08:24,795 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 22:08:24,808 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d486dda5c2c05277bcdd50 HTTP/1.1" 200 None 2025-09-26 22:08:24,808 - lava_callback - DEBUG - Processing node 68d486dda5c2c05277bcdd50: baseline-arm64 2025-09-26 22:08:25,846 - lava_callback - ERROR - Error processing callback for node 68d486dda5c2c05277bcdd50: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:40432 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43612 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43628 - "GET / HTTP/1.0" 200 OK 2025-09-26 22:08:44,197 - lava_callback - INFO - Received callback for node 68d518ddb70fb2a1e20f10f2 INFO: 172.21.0.1:60228 - "POST /node/68d518ddb70fb2a1e20f10f2 HTTP/1.0" 202 Accepted 2025-09-26 22:08:44,203 - lava_callback - INFO - Starting async job submit for node 68d518ddb70fb2a1e20f10f2 2025-09-26 22:08:44,224 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 22:08:44,237 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d518ddb70fb2a1e20f10f2 HTTP/1.1" 200 None 2025-09-26 22:08:44,237 - lava_callback - DEBUG - Processing node 68d518ddb70fb2a1e20f10f2: baseline-arm 2025-09-26 22:08:44,470 - lava_callback - ERROR - Error processing callback for node 68d518ddb70fb2a1e20f10f2: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:60230 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46512 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45510 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34798 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34802 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34810 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51118 - "GET / HTTP/1.0" 200 OK 2025-09-26 22:09:33,482 - lava_callback - INFO - Received callback for node 68d486dda5c2c05277bcdd53 INFO: 172.21.0.1:39264 - "POST /node/68d486dda5c2c05277bcdd53 HTTP/1.0" 202 Accepted 2025-09-26 22:09:33,490 - lava_callback - INFO - Starting async job submit for node 68d486dda5c2c05277bcdd53 2025-09-26 22:09:33,539 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 22:09:33,552 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d486dda5c2c05277bcdd53 HTTP/1.1" 200 None 2025-09-26 22:09:33,553 - lava_callback - DEBUG - Processing node 68d486dda5c2c05277bcdd53: baseline-arm64 INFO: 172.21.0.1:39270 - "GET / HTTP/1.0" 200 OK 2025-09-26 22:09:34,591 - lava_callback - ERROR - Error processing callback for node 68d486dda5c2c05277bcdd53: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:39278 - "GET / HTTP/1.0" 200 OK 2025-09-26 22:09:37,735 - lava_callback - INFO - Received callback for node 68d46c1da5c2c05277bcc209 INFO: 172.21.0.1:39290 - "POST /node/68d46c1da5c2c05277bcc209 HTTP/1.0" 202 Accepted 2025-09-26 22:09:37,741 - lava_callback - INFO - Starting async job submit for node 68d46c1da5c2c05277bcc209 2025-09-26 22:09:37,742 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 22:09:37,753 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d46c1da5c2c05277bcc209 HTTP/1.1" 200 None 2025-09-26 22:09:37,754 - lava_callback - DEBUG - Processing node 68d46c1da5c2c05277bcc209: baseline-arm64-baylibre 2025-09-26 22:09:37,754 - lava_callback - ERROR - Error processing callback for node 68d46c1da5c2c05277bcc209: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:42010 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37282 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59732 - "GET / HTTP/1.0" 200 OK 2025-09-26 22:10:09,806 - lava_callback - INFO - Received callback for node 68d46e66a5c2c05277bcc472 INFO: 172.21.0.1:36608 - "POST /node/68d46e66a5c2c05277bcc472 HTTP/1.0" 202 Accepted 2025-09-26 22:10:09,814 - lava_callback - INFO - Starting async job submit for node 68d46e66a5c2c05277bcc472 2025-09-26 22:10:09,923 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 22:10:09,943 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d46e66a5c2c05277bcc472 HTTP/1.1" 200 None 2025-09-26 22:10:09,944 - lava_callback - DEBUG - Processing node 68d46e66a5c2c05277bcc472: ltp-fs-bind 2025-09-26 22:10:10,466 - lava_callback - ERROR - Error processing callback for node 68d46e66a5c2c05277bcc472: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:36616 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36632 - "GET / HTTP/1.0" 200 OK 2025-09-26 22:10:17,641 - lava_callback - INFO - Received callback for node 68d46c47a5c2c05277bcc22a INFO: 172.21.0.1:36642 - "POST /node/68d46c47a5c2c05277bcc22a HTTP/1.0" 202 Accepted 2025-09-26 22:10:17,647 - lava_callback - INFO - Starting async job submit for node 68d46c47a5c2c05277bcc22a 2025-09-26 22:10:17,647 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 22:10:17,660 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d46c47a5c2c05277bcc22a HTTP/1.1" 200 None 2025-09-26 22:10:17,660 - lava_callback - DEBUG - Processing node 68d46c47a5c2c05277bcc22a: baseline-arm64-baylibre 2025-09-26 22:10:17,660 - lava_callback - ERROR - Error processing callback for node 68d46c47a5c2c05277bcc22a: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:36646 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40780 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52272 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52274 - "GET / HTTP/1.0" 200 OK 2025-09-26 22:10:37,700 - lava_callback - INFO - Received callback for node 68d46c4fa5c2c05277bcc230 INFO: 172.21.0.1:52284 - "POST /node/68d46c4fa5c2c05277bcc230 HTTP/1.0" 202 Accepted 2025-09-26 22:10:37,706 - lava_callback - INFO - Starting async job submit for node 68d46c4fa5c2c05277bcc230 2025-09-26 22:10:37,707 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 22:10:37,739 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d46c4fa5c2c05277bcc230 HTTP/1.1" 200 None 2025-09-26 22:10:37,739 - lava_callback - DEBUG - Processing node 68d46c4fa5c2c05277bcc230: baseline-arm64-baylibre 2025-09-26 22:10:37,739 - lava_callback - ERROR - Error processing callback for node 68d46c4fa5c2c05277bcc230: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:39012 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58788 - "GET / HTTP/1.0" 200 OK 2025-09-26 22:10:58,361 - lava_callback - INFO - Received callback for node 68d518ddb70fb2a1e20f10f5 INFO: 172.21.0.1:58804 - "POST /node/68d518ddb70fb2a1e20f10f5 HTTP/1.0" 202 Accepted 2025-09-26 22:10:58,367 - lava_callback - INFO - Starting async job submit for node 68d518ddb70fb2a1e20f10f5 2025-09-26 22:10:58,390 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 22:10:58,402 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d518ddb70fb2a1e20f10f5 HTTP/1.1" 200 None 2025-09-26 22:10:58,403 - lava_callback - DEBUG - Processing node 68d518ddb70fb2a1e20f10f5: baseline-arm 2025-09-26 22:10:58,649 - lava_callback - ERROR - Error processing callback for node 68d518ddb70fb2a1e20f10f5: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None 2025-09-26 22:11:02,165 - lava_callback - INFO - Received callback for node 68d46e6aa5c2c05277bcc475 INFO: 172.21.0.1:34102 - "POST /node/68d46e6aa5c2c05277bcc475 HTTP/1.0" 202 Accepted 2025-09-26 22:11:02,172 - lava_callback - INFO - Starting async job submit for node 68d46e6aa5c2c05277bcc475 2025-09-26 22:11:02,198 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 22:11:02,211 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d46e6aa5c2c05277bcc475 HTTP/1.1" 200 None 2025-09-26 22:11:02,211 - lava_callback - DEBUG - Processing node 68d46e6aa5c2c05277bcc475: ltp-fsx 2025-09-26 22:11:02,557 - lava_callback - ERROR - Error processing callback for node 68d46e6aa5c2c05277bcc475: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None 2025-09-26 22:11:05,278 - lava_callback - INFO - Received callback for node 68d4871ca5c2c05277bcdd9c INFO: 172.21.0.1:34114 - "POST /node/68d4871ca5c2c05277bcdd9c HTTP/1.0" 202 Accepted 2025-09-26 22:11:05,285 - lava_callback - INFO - Starting async job submit for node 68d4871ca5c2c05277bcdd9c 2025-09-26 22:11:05,326 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 22:11:05,339 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4871ca5c2c05277bcdd9c HTTP/1.1" 200 None 2025-09-26 22:11:05,339 - lava_callback - DEBUG - Processing node 68d4871ca5c2c05277bcdd9c: baseline-arm64 2025-09-26 22:11:06,181 - lava_callback - ERROR - Error processing callback for node 68d4871ca5c2c05277bcdd9c: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:34122 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56598 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56600 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34728 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34744 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39118 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39134 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54544 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41926 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53404 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56788 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56800 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33100 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33110 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60292 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60302 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56852 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48584 - "GET / HTTP/1.0" 200 OK 2025-09-26 22:12:57,615 - lava_callback - INFO - Received callback for node 68d518dfb70fb2a1e20f10fb INFO: 172.21.0.1:48598 - "POST /node/68d518dfb70fb2a1e20f10fb HTTP/1.0" 202 Accepted 2025-09-26 22:12:57,621 - lava_callback - INFO - Starting async job submit for node 68d518dfb70fb2a1e20f10fb 2025-09-26 22:12:57,642 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 22:12:57,664 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d518dfb70fb2a1e20f10fb HTTP/1.1" 200 None 2025-09-26 22:12:57,664 - lava_callback - DEBUG - Processing node 68d518dfb70fb2a1e20f10fb: baseline-arm 2025-09-26 22:12:57,888 - lava_callback - ERROR - Error processing callback for node 68d518dfb70fb2a1e20f10fb: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None 2025-09-26 22:12:57,980 - lava_callback - INFO - Received callback for node 68d46ce5a5c2c05277bcc2d2 INFO: 172.21.0.1:48614 - "POST /node/68d46ce5a5c2c05277bcc2d2 HTTP/1.0" 202 Accepted 2025-09-26 22:12:57,985 - lava_callback - INFO - Starting async job submit for node 68d46ce5a5c2c05277bcc2d2 2025-09-26 22:12:57,986 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 22:12:58,007 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d46ce5a5c2c05277bcc2d2 HTTP/1.1" 200 None 2025-09-26 22:12:58,008 - lava_callback - DEBUG - Processing node 68d46ce5a5c2c05277bcc2d2: baseline-arm64-baylibre 2025-09-26 22:12:58,008 - lava_callback - ERROR - Error processing callback for node 68d46ce5a5c2c05277bcc2d2: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:48366 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34664 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34678 - "GET / HTTP/1.0" 200 OK 2025-09-26 22:13:17,979 - lava_callback - INFO - Received callback for node 68d46cf4a5c2c05277bcc2db INFO: 172.21.0.1:34694 - "POST /node/68d46cf4a5c2c05277bcc2db HTTP/1.0" 202 Accepted 2025-09-26 22:13:17,984 - lava_callback - INFO - Starting async job submit for node 68d46cf4a5c2c05277bcc2db 2025-09-26 22:13:17,985 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 22:13:18,008 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d46cf4a5c2c05277bcc2db HTTP/1.1" 200 None 2025-09-26 22:13:18,008 - lava_callback - DEBUG - Processing node 68d46cf4a5c2c05277bcc2db: baseline-arm64-baylibre 2025-09-26 22:13:18,008 - lava_callback - ERROR - Error processing callback for node 68d46cf4a5c2c05277bcc2db: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:37594 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37598 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57260 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57262 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60846 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52440 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33864 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47326 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47342 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36498 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36504 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57810 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57822 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41108 - "GET / HTTP/1.0" 200 OK 2025-09-26 22:14:48,983 - lava_callback - INFO - Received callback for node 68d46e6da5c2c05277bcc478 INFO: 172.21.0.1:41118 - "POST /node/68d46e6da5c2c05277bcc478 HTTP/1.0" 202 Accepted 2025-09-26 22:14:48,990 - lava_callback - INFO - Starting async job submit for node 68d46e6da5c2c05277bcc478 2025-09-26 22:14:49,021 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 22:14:49,060 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d46e6da5c2c05277bcc478 HTTP/1.1" 200 None 2025-09-26 22:14:49,060 - lava_callback - DEBUG - Processing node 68d46e6da5c2c05277bcc478: ltp-input 2025-09-26 22:14:49,461 - lava_callback - ERROR - Error processing callback for node 68d46e6da5c2c05277bcc478: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:58360 - "GET / HTTP/1.0" 200 OK 2025-09-26 22:14:58,139 - lava_callback - INFO - Received callback for node 68d46d59a5c2c05277bcc34d INFO: 172.21.0.1:58376 - "POST /node/68d46d59a5c2c05277bcc34d HTTP/1.0" 202 Accepted 2025-09-26 22:14:58,145 - lava_callback - INFO - Starting async job submit for node 68d46d59a5c2c05277bcc34d 2025-09-26 22:14:58,146 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 22:14:58,158 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d46d59a5c2c05277bcc34d HTTP/1.1" 200 None 2025-09-26 22:14:58,158 - lava_callback - DEBUG - Processing node 68d46d59a5c2c05277bcc34d: baseline-arm64-baylibre 2025-09-26 22:14:58,158 - lava_callback - ERROR - Error processing callback for node 68d46d59a5c2c05277bcc34d: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 22:15:01,462 - lava_callback - INFO - Received callback for node 68d518e0b70fb2a1e20f10fe INFO: 172.21.0.1:45980 - "POST /node/68d518e0b70fb2a1e20f10fe HTTP/1.0" 202 Accepted 2025-09-26 22:15:01,468 - lava_callback - INFO - Starting async job submit for node 68d518e0b70fb2a1e20f10fe 2025-09-26 22:15:01,489 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 22:15:01,522 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d518e0b70fb2a1e20f10fe HTTP/1.1" 200 None 2025-09-26 22:15:01,522 - lava_callback - DEBUG - Processing node 68d518e0b70fb2a1e20f10fe: baseline-arm 2025-09-26 22:15:01,754 - lava_callback - ERROR - Error processing callback for node 68d518e0b70fb2a1e20f10fe: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:45986 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43496 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43510 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34814 - "GET / HTTP/1.0" 200 OK 2025-09-26 22:15:22,280 - lava_callback - INFO - Received callback for node 68d46e6ea5c2c05277bcc47b INFO: 172.21.0.1:34816 - "POST /node/68d46e6ea5c2c05277bcc47b HTTP/1.0" 202 Accepted 2025-09-26 22:15:22,287 - lava_callback - INFO - Starting async job submit for node 68d46e6ea5c2c05277bcc47b 2025-09-26 22:15:22,312 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 22:15:22,324 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d46e6ea5c2c05277bcc47b HTTP/1.1" 200 None 2025-09-26 22:15:22,325 - lava_callback - DEBUG - Processing node 68d46e6ea5c2c05277bcc47b: ltp-ipc 2025-09-26 22:15:22,647 - lava_callback - ERROR - Error processing callback for node 68d46e6ea5c2c05277bcc47b: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:34830 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60464 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60476 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34390 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60644 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39720 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47680 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47682 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57960 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57964 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55484 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55490 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42796 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57474 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51374 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39536 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39550 - "GET / HTTP/1.0" 200 OK 2025-09-26 22:17:18,848 - lava_callback - INFO - Received callback for node 68d518e2b70fb2a1e20f1101 INFO: 172.21.0.1:39558 - "POST /node/68d518e2b70fb2a1e20f1101 HTTP/1.0" 202 Accepted 2025-09-26 22:17:18,854 - lava_callback - INFO - Starting async job submit for node 68d518e2b70fb2a1e20f1101 2025-09-26 22:17:18,875 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 22:17:18,898 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d518e2b70fb2a1e20f1101 HTTP/1.1" 200 None 2025-09-26 22:17:18,898 - lava_callback - DEBUG - Processing node 68d518e2b70fb2a1e20f1101: baseline-arm 2025-09-26 22:17:19,152 - lava_callback - ERROR - Error processing callback for node 68d518e2b70fb2a1e20f1101: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:55294 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55304 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37638 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37654 - "GET / HTTP/1.0" 200 OK 2025-09-26 22:17:41,815 - lava_callback - INFO - Received callback for node 68d46dfba5c2c05277bcc3e4 INFO: 172.21.0.1:60314 - "POST /node/68d46dfba5c2c05277bcc3e4 HTTP/1.0" 202 Accepted 2025-09-26 22:17:41,820 - lava_callback - INFO - Starting async job submit for node 68d46dfba5c2c05277bcc3e4 2025-09-26 22:17:41,821 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 22:17:41,860 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d46dfba5c2c05277bcc3e4 HTTP/1.1" 200 None 2025-09-26 22:17:41,860 - lava_callback - DEBUG - Processing node 68d46dfba5c2c05277bcc3e4: baseline-arm 2025-09-26 22:17:41,860 - lava_callback - ERROR - Error processing callback for node 68d46dfba5c2c05277bcc3e4: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:60322 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50588 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56828 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55514 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55528 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42910 - "GET / HTTP/1.0" 200 OK 2025-09-26 22:18:21,948 - lava_callback - INFO - Received callback for node 68d46e2ba5c2c05277bcc41c INFO: 172.21.0.1:42920 - "POST /node/68d46e2ba5c2c05277bcc41c HTTP/1.0" 202 Accepted 2025-09-26 22:18:21,954 - lava_callback - INFO - Starting async job submit for node 68d46e2ba5c2c05277bcc41c 2025-09-26 22:18:21,954 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 22:18:22,103 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d46e2ba5c2c05277bcc41c HTTP/1.1" 200 None 2025-09-26 22:18:22,103 - lava_callback - DEBUG - Processing node 68d46e2ba5c2c05277bcc41c: baseline-arm 2025-09-26 22:18:22,103 - lava_callback - ERROR - Error processing callback for node 68d46e2ba5c2c05277bcc41c: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:42936 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57492 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57502 - "GET / HTTP/1.0" 200 OK 2025-09-26 22:18:41,589 - lava_callback - INFO - Received callback for node 68d46e2da5c2c05277bcc41f INFO: 172.21.0.1:58866 - "POST /node/68d46e2da5c2c05277bcc41f HTTP/1.0" 202 Accepted 2025-09-26 22:18:41,594 - lava_callback - INFO - Starting async job submit for node 68d46e2da5c2c05277bcc41f 2025-09-26 22:18:41,595 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 22:18:41,630 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d46e2da5c2c05277bcc41f HTTP/1.1" 200 None 2025-09-26 22:18:41,630 - lava_callback - DEBUG - Processing node 68d46e2da5c2c05277bcc41f: baseline-arm 2025-09-26 22:18:41,630 - lava_callback - ERROR - Error processing callback for node 68d46e2da5c2c05277bcc41f: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 22:18:42,266 - lava_callback - INFO - Received callback for node 68d46e3ba5c2c05277bcc434 INFO: 172.21.0.1:58872 - "POST /node/68d46e3ba5c2c05277bcc434 HTTP/1.0" 202 Accepted 2025-09-26 22:18:42,271 - lava_callback - INFO - Starting async job submit for node 68d46e3ba5c2c05277bcc434 2025-09-26 22:18:42,272 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 22:18:42,285 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d46e3ba5c2c05277bcc434 HTTP/1.1" 200 None 2025-09-26 22:18:42,285 - lava_callback - DEBUG - Processing node 68d46e3ba5c2c05277bcc434: baseline-arm 2025-09-26 22:18:42,285 - lava_callback - ERROR - Error processing callback for node 68d46e3ba5c2c05277bcc434: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 22:18:42,923 - lava_callback - INFO - Received callback for node 68d46e3ca5c2c05277bcc437 INFO: 172.21.0.1:58874 - "POST /node/68d46e3ca5c2c05277bcc437 HTTP/1.0" 202 Accepted 2025-09-26 22:18:42,929 - lava_callback - INFO - Starting async job submit for node 68d46e3ca5c2c05277bcc437 2025-09-26 22:18:42,929 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 22:18:42,963 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d46e3ca5c2c05277bcc437 HTTP/1.1" 200 None 2025-09-26 22:18:42,964 - lava_callback - DEBUG - Processing node 68d46e3ca5c2c05277bcc437: baseline-arm 2025-09-26 22:18:42,964 - lava_callback - ERROR - Error processing callback for node 68d46e3ca5c2c05277bcc437: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:58888 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40056 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32998 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60898 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60906 - "GET / HTTP/1.0" 200 OK 2025-09-26 22:19:17,760 - lava_callback - INFO - Received callback for node 68d518e3b70fb2a1e20f1104 INFO: 172.21.0.1:60920 - "POST /node/68d518e3b70fb2a1e20f1104 HTTP/1.0" 202 Accepted 2025-09-26 22:19:17,766 - lava_callback - INFO - Starting async job submit for node 68d518e3b70fb2a1e20f1104 2025-09-26 22:19:17,786 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 22:19:17,814 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d518e3b70fb2a1e20f1104 HTTP/1.1" 200 None 2025-09-26 22:19:17,815 - lava_callback - DEBUG - Processing node 68d518e3b70fb2a1e20f1104: baseline-arm 2025-09-26 22:19:18,044 - lava_callback - ERROR - Error processing callback for node 68d518e3b70fb2a1e20f1104: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:42200 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42214 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32780 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32794 - "GET / HTTP/1.0" 200 OK 2025-09-26 22:19:45,877 - lava_callback - INFO - Received callback for node 68d46e7da5c2c05277bcc497 INFO: 172.21.0.1:58474 - "POST /node/68d46e7da5c2c05277bcc497 HTTP/1.0" 202 Accepted 2025-09-26 22:19:45,884 - lava_callback - INFO - Starting async job submit for node 68d46e7da5c2c05277bcc497 2025-09-26 22:19:45,885 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 22:19:45,914 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d46e7da5c2c05277bcc497 HTTP/1.1" 200 None 2025-09-26 22:19:45,914 - lava_callback - DEBUG - Processing node 68d46e7da5c2c05277bcc497: ltp-syscalls-ipc 2025-09-26 22:19:45,914 - lava_callback - ERROR - Error processing callback for node 68d46e7da5c2c05277bcc497: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:58486 - "GET / HTTP/1.0" 200 OK 2025-09-26 22:19:46,677 - lava_callback - INFO - Received callback for node 68d46e80a5c2c05277bcc49a INFO: 172.21.0.1:58500 - "POST /node/68d46e80a5c2c05277bcc49a HTTP/1.0" 202 Accepted 2025-09-26 22:19:46,683 - lava_callback - INFO - Starting async job submit for node 68d46e80a5c2c05277bcc49a 2025-09-26 22:19:46,684 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 22:19:46,697 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d46e80a5c2c05277bcc49a HTTP/1.1" 200 None 2025-09-26 22:19:46,698 - lava_callback - DEBUG - Processing node 68d46e80a5c2c05277bcc49a: ltp-timers 2025-09-26 22:19:46,698 - lava_callback - ERROR - Error processing callback for node 68d46e80a5c2c05277bcc49a: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:52044 - "GET / HTTP/1.0" 200 OK 2025-09-26 22:20:01,836 - lava_callback - INFO - Received callback for node 68d46e84a5c2c05277bcc4a0 INFO: 172.21.0.1:41242 - "POST /node/68d46e84a5c2c05277bcc4a0 HTTP/1.0" 202 Accepted 2025-09-26 22:20:01,842 - lava_callback - INFO - Starting async job submit for node 68d46e84a5c2c05277bcc4a0 2025-09-26 22:20:01,843 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 22:20:01,857 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d46e84a5c2c05277bcc4a0 HTTP/1.1" 200 None 2025-09-26 22:20:01,858 - lava_callback - DEBUG - Processing node 68d46e84a5c2c05277bcc4a0: baseline-arm 2025-09-26 22:20:01,858 - lava_callback - ERROR - Error processing callback for node 68d46e84a5c2c05277bcc4a0: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 22:20:02,631 - lava_callback - INFO - Received callback for node 68d46e87a5c2c05277bcc4a3 INFO: 172.21.0.1:41248 - "POST /node/68d46e87a5c2c05277bcc4a3 HTTP/1.0" 202 Accepted 2025-09-26 22:20:02,637 - lava_callback - INFO - Starting async job submit for node 68d46e87a5c2c05277bcc4a3 2025-09-26 22:20:02,638 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 22:20:02,668 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d46e87a5c2c05277bcc4a3 HTTP/1.1" 200 None 2025-09-26 22:20:02,669 - lava_callback - DEBUG - Processing node 68d46e87a5c2c05277bcc4a3: baseline-arm 2025-09-26 22:20:02,669 - lava_callback - ERROR - Error processing callback for node 68d46e87a5c2c05277bcc4a3: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 22:20:05,807 - lava_callback - INFO - Received callback for node 68d46e83a5c2c05277bcc49d INFO: 172.21.0.1:41260 - "POST /node/68d46e83a5c2c05277bcc49d HTTP/1.0" 202 Accepted 2025-09-26 22:20:05,813 - lava_callback - INFO - Starting async job submit for node 68d46e83a5c2c05277bcc49d 2025-09-26 22:20:05,814 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 22:20:05,826 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d46e83a5c2c05277bcc49d HTTP/1.1" 200 None 2025-09-26 22:20:05,827 - lava_callback - DEBUG - Processing node 68d46e83a5c2c05277bcc49d: ltp-watchqueue 2025-09-26 22:20:05,827 - lava_callback - ERROR - Error processing callback for node 68d46e83a5c2c05277bcc49d: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:41274 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34984 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34990 - "GET / HTTP/1.0" 200 OK 2025-09-26 22:20:21,432 - lava_callback - INFO - Received callback for node 68d46dfba5c2c05277bcc3e1 INFO: 172.21.0.1:40512 - "POST /node/68d46dfba5c2c05277bcc3e1 HTTP/1.0" 202 Accepted 2025-09-26 22:20:21,438 - lava_callback - INFO - Starting async job submit for node 68d46dfba5c2c05277bcc3e1 2025-09-26 22:20:21,468 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 22:20:21,502 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d46dfba5c2c05277bcc3e1 HTTP/1.1" 200 None 2025-09-26 22:20:21,502 - lava_callback - DEBUG - Processing node 68d46dfba5c2c05277bcc3e1: baseline-arm INFO: 172.21.0.1:40528 - "GET / HTTP/1.0" 200 OK 2025-09-26 22:20:21,772 - lava_callback - ERROR - Error processing callback for node 68d46dfba5c2c05277bcc3e1: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:40544 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53944 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53960 - "GET / HTTP/1.0" 200 OK 2025-09-26 22:20:41,475 - lava_callback - INFO - Received callback for node 68d501e5ef0c6c1c702065fe INFO: 172.21.0.1:56536 - "POST /node/68d501e5ef0c6c1c702065fe HTTP/1.0" 202 Accepted 2025-09-26 22:20:41,480 - lava_callback - INFO - Starting async job submit for node 68d501e5ef0c6c1c702065fe 2025-09-26 22:20:41,497 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 22:20:41,516 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d501e5ef0c6c1c702065fe HTTP/1.1" 200 None 2025-09-26 22:20:41,516 - lava_callback - DEBUG - Processing node 68d501e5ef0c6c1c702065fe: baseline-arm64-broonie 2025-09-26 22:20:41,843 - lava_callback - ERROR - Error processing callback for node 68d501e5ef0c6c1c702065fe: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None 2025-09-26 22:20:44,187 - lava_callback - INFO - Received callback for node 68d46ea6a5c2c05277bcc4bb INFO: 172.21.0.1:56540 - "POST /node/68d46ea6a5c2c05277bcc4bb HTTP/1.0" 202 Accepted 2025-09-26 22:20:44,192 - lava_callback - INFO - Starting async job submit for node 68d46ea6a5c2c05277bcc4bb 2025-09-26 22:20:44,193 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 22:20:44,205 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d46ea6a5c2c05277bcc4bb HTTP/1.1" 200 None 2025-09-26 22:20:44,205 - lava_callback - DEBUG - Processing node 68d46ea6a5c2c05277bcc4bb: baseline-arm64-clabbe 2025-09-26 22:20:44,205 - lava_callback - ERROR - Error processing callback for node 68d46ea6a5c2c05277bcc4bb: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:56544 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38696 - "GET / HTTP/1.0" 200 OK 2025-09-26 22:21:04,141 - lava_callback - INFO - Received callback for node 68d46eb9a5c2c05277bcc4cf INFO: 172.21.0.1:46832 - "POST /node/68d46eb9a5c2c05277bcc4cf HTTP/1.0" 202 Accepted 2025-09-26 22:21:04,146 - lava_callback - INFO - Starting async job submit for node 68d46eb9a5c2c05277bcc4cf 2025-09-26 22:21:04,147 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 22:21:04,185 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d46eb9a5c2c05277bcc4cf HTTP/1.1" 200 None 2025-09-26 22:21:04,185 - lava_callback - DEBUG - Processing node 68d46eb9a5c2c05277bcc4cf: baseline-arm64-clabbe 2025-09-26 22:21:04,186 - lava_callback - ERROR - Error processing callback for node 68d46eb9a5c2c05277bcc4cf: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 22:21:04,799 - lava_callback - INFO - Received callback for node 68d46ec9a5c2c05277bcc4d8 INFO: 172.21.0.1:46838 - "POST /node/68d46ec9a5c2c05277bcc4d8 HTTP/1.0" 202 Accepted 2025-09-26 22:21:04,804 - lava_callback - INFO - Starting async job submit for node 68d46ec9a5c2c05277bcc4d8 2025-09-26 22:21:04,805 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 22:21:04,817 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d46ec9a5c2c05277bcc4d8 HTTP/1.1" 200 None 2025-09-26 22:21:04,818 - lava_callback - DEBUG - Processing node 68d46ec9a5c2c05277bcc4d8: baseline-arm64-clabbe 2025-09-26 22:21:04,818 - lava_callback - ERROR - Error processing callback for node 68d46ec9a5c2c05277bcc4d8: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:46840 - "GET / HTTP/1.0" 200 OK 2025-09-26 22:21:15,002 - lava_callback - INFO - Received callback for node 68d518e4b70fb2a1e20f110a INFO: 172.21.0.1:40800 - "POST /node/68d518e4b70fb2a1e20f110a HTTP/1.0" 202 Accepted 2025-09-26 22:21:15,008 - lava_callback - INFO - Starting async job submit for node 68d518e4b70fb2a1e20f110a 2025-09-26 22:21:15,027 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 22:21:15,039 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d518e4b70fb2a1e20f110a HTTP/1.1" 200 None 2025-09-26 22:21:15,039 - lava_callback - DEBUG - Processing node 68d518e4b70fb2a1e20f110a: baseline-arm INFO: 172.21.0.1:40810 - "GET /api/metrics HTTP/1.0" 200 OK 2025-09-26 22:21:15,260 - lava_callback - ERROR - Error processing callback for node 68d518e4b70fb2a1e20f110a: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:40822 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44480 - "GET / HTTP/1.0" 200 OK 2025-09-26 22:21:24,143 - lava_callback - INFO - Received callback for node 68d46ecfa5c2c05277bcc4db INFO: 172.21.0.1:44494 - "POST /node/68d46ecfa5c2c05277bcc4db HTTP/1.0" 202 Accepted 2025-09-26 22:21:24,149 - lava_callback - INFO - Starting async job submit for node 68d46ecfa5c2c05277bcc4db 2025-09-26 22:21:24,150 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 22:21:24,162 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d46ecfa5c2c05277bcc4db HTTP/1.1" 200 None 2025-09-26 22:21:24,163 - lava_callback - DEBUG - Processing node 68d46ecfa5c2c05277bcc4db: baseline-arm64-clabbe 2025-09-26 22:21:24,163 - lava_callback - ERROR - Error processing callback for node 68d46ecfa5c2c05277bcc4db: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 22:21:24,798 - lava_callback - INFO - Received callback for node 68d46ed3a5c2c05277bcc4de INFO: 172.21.0.1:44502 - "POST /node/68d46ed3a5c2c05277bcc4de HTTP/1.0" 202 Accepted 2025-09-26 22:21:24,804 - lava_callback - INFO - Starting async job submit for node 68d46ed3a5c2c05277bcc4de 2025-09-26 22:21:24,805 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 22:21:24,816 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d46ed3a5c2c05277bcc4de HTTP/1.1" 200 None 2025-09-26 22:21:24,817 - lava_callback - DEBUG - Processing node 68d46ed3a5c2c05277bcc4de: baseline-arm64-clabbe 2025-09-26 22:21:24,817 - lava_callback - ERROR - Error processing callback for node 68d46ed3a5c2c05277bcc4de: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:44508 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39978 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39992 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47310 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44044 - "GET / HTTP/1.0" 200 OK 2025-09-26 22:22:01,680 - lava_callback - INFO - Received callback for node 68d46f05a5c2c05277bcc524 INFO: 172.21.0.1:50652 - "POST /node/68d46f05a5c2c05277bcc524 HTTP/1.0" 202 Accepted 2025-09-26 22:22:01,686 - lava_callback - INFO - Starting async job submit for node 68d46f05a5c2c05277bcc524 2025-09-26 22:22:01,686 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 22:22:01,718 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d46f05a5c2c05277bcc524 HTTP/1.1" 200 None 2025-09-26 22:22:01,718 - lava_callback - DEBUG - Processing node 68d46f05a5c2c05277bcc524: baseline-arm 2025-09-26 22:22:01,718 - lava_callback - ERROR - Error processing callback for node 68d46f05a5c2c05277bcc524: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:50662 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50588 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50594 - "GET / HTTP/1.0" 200 OK 2025-09-26 22:22:21,967 - lava_callback - INFO - Received callback for node 68d46f09a5c2c05277bcc527 INFO: 172.21.0.1:33308 - "POST /node/68d46f09a5c2c05277bcc527 HTTP/1.0" 202 Accepted 2025-09-26 22:22:21,972 - lava_callback - INFO - Starting async job submit for node 68d46f09a5c2c05277bcc527 2025-09-26 22:22:21,973 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 22:22:21,986 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d46f09a5c2c05277bcc527 HTTP/1.1" 200 None 2025-09-26 22:22:21,986 - lava_callback - DEBUG - Processing node 68d46f09a5c2c05277bcc527: baseline-arm 2025-09-26 22:22:21,986 - lava_callback - ERROR - Error processing callback for node 68d46f09a5c2c05277bcc527: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:33312 - "GET / HTTP/1.0" 200 OK 2025-09-26 22:22:22,507 - lava_callback - INFO - Received callback for node 68d46e75a5c2c05277bcc481 INFO: 172.21.0.1:33326 - "POST /node/68d46e75a5c2c05277bcc481 HTTP/1.0" 202 Accepted 2025-09-26 22:22:22,514 - lava_callback - INFO - Starting async job submit for node 68d46e75a5c2c05277bcc481 2025-09-26 22:22:22,548 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 22:22:22,561 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d46e75a5c2c05277bcc481 HTTP/1.1" 200 None 2025-09-26 22:22:22,562 - lava_callback - DEBUG - Processing node 68d46e75a5c2c05277bcc481: ltp-sched 2025-09-26 22:22:22,653 - lava_callback - INFO - Received callback for node 68d46f0aa5c2c05277bcc52a 2025-09-26 22:22:22,664 - lava_callback - INFO - Starting async job submit for node 68d46f0aa5c2c05277bcc52a INFO: 172.21.0.1:33334 - "POST /node/68d46f0aa5c2c05277bcc52a HTTP/1.0" 202 Accepted 2025-09-26 22:22:22,685 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 22:22:22,777 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d46f0aa5c2c05277bcc52a HTTP/1.1" 200 None 2025-09-26 22:22:22,793 - lava_callback - DEBUG - Processing node 68d46f0aa5c2c05277bcc52a: baseline-arm 2025-09-26 22:22:22,798 - lava_callback - ERROR - Error processing callback for node 68d46f0aa5c2c05277bcc52a: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 22:22:23,089 - lava_callback - ERROR - Error processing callback for node 68d46e75a5c2c05277bcc481: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:33342 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38120 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38124 - "GET / HTTP/1.0" 200 OK 2025-09-26 22:22:42,313 - lava_callback - INFO - Received callback for node 68d46f26a5c2c05277bcc551 INFO: 172.21.0.1:53728 - "POST /node/68d46f26a5c2c05277bcc551 HTTP/1.0" 202 Accepted 2025-09-26 22:22:42,318 - lava_callback - INFO - Starting async job submit for node 68d46f26a5c2c05277bcc551 2025-09-26 22:22:42,319 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 22:22:42,331 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d46f26a5c2c05277bcc551 HTTP/1.1" 200 None 2025-09-26 22:22:42,332 - lava_callback - DEBUG - Processing node 68d46f26a5c2c05277bcc551: baseline-arm 2025-09-26 22:22:42,332 - lava_callback - ERROR - Error processing callback for node 68d46f26a5c2c05277bcc551: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 22:22:42,991 - lava_callback - INFO - Received callback for node 68d46f27a5c2c05277bcc554 INFO: 172.21.0.1:53742 - "POST /node/68d46f27a5c2c05277bcc554 HTTP/1.0" 202 Accepted 2025-09-26 22:22:42,996 - lava_callback - INFO - Starting async job submit for node 68d46f27a5c2c05277bcc554 2025-09-26 22:22:42,997 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 22:22:43,030 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d46f27a5c2c05277bcc554 HTTP/1.1" 200 None 2025-09-26 22:22:43,030 - lava_callback - DEBUG - Processing node 68d46f27a5c2c05277bcc554: baseline-arm 2025-09-26 22:22:43,030 - lava_callback - ERROR - Error processing callback for node 68d46f27a5c2c05277bcc554: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 22:22:43,676 - lava_callback - INFO - Received callback for node 68d46f2ba5c2c05277bcc557 INFO: 172.21.0.1:53750 - "POST /node/68d46f2ba5c2c05277bcc557 HTTP/1.0" 202 Accepted 2025-09-26 22:22:43,681 - lava_callback - INFO - Starting async job submit for node 68d46f2ba5c2c05277bcc557 2025-09-26 22:22:43,682 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 22:22:43,694 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d46f2ba5c2c05277bcc557 HTTP/1.1" 200 None 2025-09-26 22:22:43,695 - lava_callback - DEBUG - Processing node 68d46f2ba5c2c05277bcc557: baseline-arm 2025-09-26 22:22:43,695 - lava_callback - ERROR - Error processing callback for node 68d46f2ba5c2c05277bcc557: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 22:22:44,352 - lava_callback - INFO - Received callback for node 68d46f2da5c2c05277bcc55a INFO: 172.21.0.1:53756 - "POST /node/68d46f2da5c2c05277bcc55a HTTP/1.0" 202 Accepted 2025-09-26 22:22:44,357 - lava_callback - INFO - Starting async job submit for node 68d46f2da5c2c05277bcc55a 2025-09-26 22:22:44,358 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 22:22:44,371 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d46f2da5c2c05277bcc55a HTTP/1.1" 200 None 2025-09-26 22:22:44,371 - lava_callback - DEBUG - Processing node 68d46f2da5c2c05277bcc55a: baseline-arm 2025-09-26 22:22:44,371 - lava_callback - ERROR - Error processing callback for node 68d46f2da5c2c05277bcc55a: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:53768 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56694 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54374 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60336 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60340 - "GET / HTTP/1.0" 200 OK 2025-09-26 22:23:18,141 - lava_callback - INFO - Received callback for node 68d518e5b70fb2a1e20f110d INFO: 172.21.0.1:60350 - "POST /node/68d518e5b70fb2a1e20f110d HTTP/1.0" 202 Accepted 2025-09-26 22:23:18,147 - lava_callback - INFO - Starting async job submit for node 68d518e5b70fb2a1e20f110d 2025-09-26 22:23:18,168 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 22:23:18,180 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d518e5b70fb2a1e20f110d HTTP/1.1" 200 None 2025-09-26 22:23:18,180 - lava_callback - DEBUG - Processing node 68d518e5b70fb2a1e20f110d: baseline-arm 2025-09-26 22:23:18,405 - lava_callback - ERROR - Error processing callback for node 68d518e5b70fb2a1e20f110d: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:53554 - "GET / HTTP/1.0" 200 OK 2025-09-26 22:23:26,020 - lava_callback - INFO - Received callback for node 68d46f51a5c2c05277bcc595 INFO: 172.21.0.1:53568 - "POST /node/68d46f51a5c2c05277bcc595 HTTP/1.0" 202 Accepted 2025-09-26 22:23:26,026 - lava_callback - INFO - Starting async job submit for node 68d46f51a5c2c05277bcc595 2025-09-26 22:23:26,026 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 22:23:26,062 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d46f51a5c2c05277bcc595 HTTP/1.1" 200 None 2025-09-26 22:23:26,062 - lava_callback - DEBUG - Processing node 68d46f51a5c2c05277bcc595: baseline-arm-broonie 2025-09-26 22:23:26,062 - lava_callback - ERROR - Error processing callback for node 68d46f51a5c2c05277bcc595: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:53570 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36176 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36190 - "GET / HTTP/1.0" 200 OK 2025-09-26 22:23:44,310 - lava_callback - INFO - Received callback for node 68d46f5ca5c2c05277bcc5a1 INFO: 172.21.0.1:36466 - "POST /node/68d46f5ca5c2c05277bcc5a1 HTTP/1.0" 202 Accepted 2025-09-26 22:23:44,316 - lava_callback - INFO - Starting async job submit for node 68d46f5ca5c2c05277bcc5a1 2025-09-26 22:23:44,316 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 22:23:44,328 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d46f5ca5c2c05277bcc5a1 HTTP/1.1" 200 None 2025-09-26 22:23:44,329 - lava_callback - DEBUG - Processing node 68d46f5ca5c2c05277bcc5a1: baseline-arm-clabbe 2025-09-26 22:23:44,329 - lava_callback - ERROR - Error processing callback for node 68d46f5ca5c2c05277bcc5a1: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 22:23:44,981 - lava_callback - INFO - Received callback for node 68d46f62a5c2c05277bcc5a7 INFO: 172.21.0.1:36470 - "POST /node/68d46f62a5c2c05277bcc5a7 HTTP/1.0" 202 Accepted 2025-09-26 22:23:44,989 - lava_callback - INFO - Starting async job submit for node 68d46f62a5c2c05277bcc5a7 2025-09-26 22:23:44,990 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 22:23:45,030 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d46f62a5c2c05277bcc5a7 HTTP/1.1" 200 None 2025-09-26 22:23:45,031 - lava_callback - DEBUG - Processing node 68d46f62a5c2c05277bcc5a7: baseline-arm-clabbe 2025-09-26 22:23:45,031 - lava_callback - ERROR - Error processing callback for node 68d46f62a5c2c05277bcc5a7: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 22:23:45,632 - lava_callback - INFO - Received callback for node 68d46f64a5c2c05277bcc5aa INFO: 172.21.0.1:36476 - "POST /node/68d46f64a5c2c05277bcc5aa HTTP/1.0" 202 Accepted 2025-09-26 22:23:45,637 - lava_callback - INFO - Starting async job submit for node 68d46f64a5c2c05277bcc5aa 2025-09-26 22:23:45,638 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 22:23:45,685 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d46f64a5c2c05277bcc5aa HTTP/1.1" 200 None 2025-09-26 22:23:45,685 - lava_callback - DEBUG - Processing node 68d46f64a5c2c05277bcc5aa: baseline-arm-clabbe 2025-09-26 22:23:45,685 - lava_callback - ERROR - Error processing callback for node 68d46f64a5c2c05277bcc5aa: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:36482 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58430 - "GET / HTTP/1.0" 200 OK 2025-09-26 22:24:03,657 - lava_callback - INFO - Received callback for node 68d46e7ba5c2c05277bcc494 INFO: 172.21.0.1:53208 - "POST /node/68d46e7ba5c2c05277bcc494 HTTP/1.0" 202 Accepted 2025-09-26 22:24:03,677 - lava_callback - INFO - Starting async job submit for node 68d46e7ba5c2c05277bcc494 2025-09-26 22:24:04,458 - lava_callback - INFO - Received callback for node 68d46f70a5c2c05277bcc5c6 INFO: 172.21.0.1:53220 - "POST /node/68d46f70a5c2c05277bcc5c6 HTTP/1.0" 202 Accepted 2025-09-26 22:24:04,479 - lava_callback - INFO - Starting async job submit for node 68d46f70a5c2c05277bcc5c6 2025-09-26 22:24:04,480 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 22:24:04,558 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d46f70a5c2c05277bcc5c6 HTTP/1.1" 200 None 2025-09-26 22:24:04,574 - lava_callback - DEBUG - Processing node 68d46f70a5c2c05277bcc5c6: baseline-arm-clabbe 2025-09-26 22:24:04,574 - lava_callback - ERROR - Error processing callback for node 68d46f70a5c2c05277bcc5c6: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 22:24:05,046 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 22:24:05,060 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d46e7ba5c2c05277bcc494 HTTP/1.1" 200 None 2025-09-26 22:24:05,061 - lava_callback - DEBUG - Processing node 68d46e7ba5c2c05277bcc494: ltp-syscalls 2025-09-26 22:24:05,113 - lava_callback - INFO - Received callback for node 68d46f48a5c2c05277bcc58c INFO: 172.21.0.1:53234 - "POST /node/68d46f48a5c2c05277bcc58c HTTP/1.0" 202 Accepted 2025-09-26 22:24:05,139 - lava_callback - INFO - Starting async job submit for node 68d46f48a5c2c05277bcc58c 2025-09-26 22:24:05,182 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 22:24:05,253 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d46f48a5c2c05277bcc58c HTTP/1.1" 200 None 2025-09-26 22:24:05,268 - lava_callback - DEBUG - Processing node 68d46f48a5c2c05277bcc58c: baseline-arm-broonie 2025-09-26 22:24:05,729 - lava_callback - ERROR - Error processing callback for node 68d46f48a5c2c05277bcc58c: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None 2025-09-26 22:24:06,348 - lava_callback - INFO - Received callback for node 68d46f7aa5c2c05277bcc5d2 INFO: 172.21.0.1:53250 - "POST /node/68d46f7aa5c2c05277bcc5d2 HTTP/1.0" 202 Accepted 2025-09-26 22:24:06,364 - lava_callback - INFO - Starting async job submit for node 68d46f7aa5c2c05277bcc5d2 2025-09-26 22:24:06,381 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 22:24:06,448 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d46f7aa5c2c05277bcc5d2 HTTP/1.1" 200 None 2025-09-26 22:24:06,463 - lava_callback - DEBUG - Processing node 68d46f7aa5c2c05277bcc5d2: ltp-cap-bounds 2025-09-26 22:24:06,464 - lava_callback - ERROR - Error processing callback for node 68d46f7aa5c2c05277bcc5d2: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:53252 - "GET / HTTP/1.0" 200 OK 2025-09-26 22:24:07,209 - lava_callback - INFO - Received callback for node 68d46f7ea5c2c05277bcc5d5 INFO: 172.21.0.1:53258 - "POST /node/68d46f7ea5c2c05277bcc5d5 HTTP/1.0" 202 Accepted 2025-09-26 22:24:07,231 - lava_callback - INFO - Starting async job submit for node 68d46f7ea5c2c05277bcc5d5 2025-09-26 22:24:07,231 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 22:24:07,299 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d46f7ea5c2c05277bcc5d5 HTTP/1.1" 200 None 2025-09-26 22:24:07,314 - lava_callback - DEBUG - Processing node 68d46f7ea5c2c05277bcc5d5: ltp-containers 2025-09-26 22:24:07,314 - lava_callback - ERROR - Error processing callback for node 68d46f7ea5c2c05277bcc5d5: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 22:24:07,980 - lava_callback - INFO - Received callback for node 68d46f80a5c2c05277bcc5d8 INFO: 172.21.0.1:53266 - "POST /node/68d46f80a5c2c05277bcc5d8 HTTP/1.0" 202 Accepted 2025-09-26 22:24:08,007 - lava_callback - INFO - Starting async job submit for node 68d46f80a5c2c05277bcc5d8 2025-09-26 22:24:08,012 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 22:24:08,075 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d46f80a5c2c05277bcc5d8 HTTP/1.1" 200 None 2025-09-26 22:24:08,090 - lava_callback - DEBUG - Processing node 68d46f80a5c2c05277bcc5d8: ltp-controllers 2025-09-26 22:24:08,091 - lava_callback - ERROR - Error processing callback for node 68d46f80a5c2c05277bcc5d8: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 22:24:08,769 - lava_callback - INFO - Received callback for node 68d46f82a5c2c05277bcc5db INFO: 172.21.0.1:53282 - "POST /node/68d46f82a5c2c05277bcc5db HTTP/1.0" 202 Accepted 2025-09-26 22:24:08,786 - lava_callback - INFO - Starting async job submit for node 68d46f82a5c2c05277bcc5db 2025-09-26 22:24:08,792 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 22:24:08,897 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d46f82a5c2c05277bcc5db HTTP/1.1" 200 None 2025-09-26 22:24:08,908 - lava_callback - DEBUG - Processing node 68d46f82a5c2c05277bcc5db: ltp-cve 2025-09-26 22:24:08,908 - lava_callback - ERROR - Error processing callback for node 68d46f82a5c2c05277bcc5db: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 22:24:09,577 - lava_callback - INFO - Received callback for node 68d46f86a5c2c05277bcc5e1 INFO: 172.21.0.1:41618 - "POST /node/68d46f86a5c2c05277bcc5e1 HTTP/1.0" 202 Accepted 2025-09-26 22:24:09,598 - lava_callback - INFO - Starting async job submit for node 68d46f86a5c2c05277bcc5e1 2025-09-26 22:24:09,599 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 22:24:09,659 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d46f86a5c2c05277bcc5e1 HTTP/1.1" 200 None 2025-09-26 22:24:09,670 - lava_callback - DEBUG - Processing node 68d46f86a5c2c05277bcc5e1: ltp-fs-bind 2025-09-26 22:24:09,675 - lava_callback - ERROR - Error processing callback for node 68d46f86a5c2c05277bcc5e1: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 22:24:10,335 - lava_callback - INFO - Received callback for node 68d46f89a5c2c05277bcc5e4 INFO: 172.21.0.1:41630 - "POST /node/68d46f89a5c2c05277bcc5e4 HTTP/1.0" 202 Accepted 2025-09-26 22:24:10,357 - lava_callback - INFO - Starting async job submit for node 68d46f89a5c2c05277bcc5e4 2025-09-26 22:24:10,358 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 22:24:10,428 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d46f89a5c2c05277bcc5e4 HTTP/1.1" 200 None 2025-09-26 22:24:10,444 - lava_callback - DEBUG - Processing node 68d46f89a5c2c05277bcc5e4: ltp-fsx 2025-09-26 22:24:10,444 - lava_callback - ERROR - Error processing callback for node 68d46f89a5c2c05277bcc5e4: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 22:24:12,632 - lava_callback - ERROR - Error processing callback for node 68d46e7ba5c2c05277bcc494: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:41636 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41640 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47300 - "GET / HTTP/1.0" 200 OK 2025-09-26 22:24:22,847 - lava_callback - INFO - Received callback for node 68d46e77a5c2c05277bcc484 INFO: 172.21.0.1:47302 - "POST /node/68d46e77a5c2c05277bcc484 HTTP/1.0" 202 Accepted 2025-09-26 22:24:22,854 - lava_callback - INFO - Starting async job submit for node 68d46e77a5c2c05277bcc484 2025-09-26 22:24:22,890 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 22:24:22,902 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d46e77a5c2c05277bcc484 HTTP/1.1" 200 None 2025-09-26 22:24:22,903 - lava_callback - DEBUG - Processing node 68d46e77a5c2c05277bcc484: ltp-smoketest 2025-09-26 22:24:23,302 - lava_callback - ERROR - Error processing callback for node 68d46e77a5c2c05277bcc484: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None 2025-09-26 22:24:23,466 - lava_callback - INFO - Received callback for node 68d512f3ef0c6c1c702066b7 INFO: 172.21.0.1:47306 - "POST /node/68d512f3ef0c6c1c702066b7 HTTP/1.0" 202 Accepted 2025-09-26 22:24:23,472 - lava_callback - INFO - Starting async job submit for node 68d512f3ef0c6c1c702066b7 2025-09-26 22:24:23,490 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 22:24:23,505 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d512f3ef0c6c1c702066b7 HTTP/1.1" 200 None 2025-09-26 22:24:23,505 - lava_callback - DEBUG - Processing node 68d512f3ef0c6c1c702066b7: baseline-arm64-broonie 2025-09-26 22:24:23,808 - lava_callback - ERROR - Error processing callback for node 68d512f3ef0c6c1c702066b7: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None 2025-09-26 22:24:26,365 - lava_callback - INFO - Received callback for node 68d46f90a5c2c05277bcc5ed INFO: 172.21.0.1:47320 - "POST /node/68d46f90a5c2c05277bcc5ed HTTP/1.0" 202 Accepted 2025-09-26 22:24:26,371 - lava_callback - INFO - Starting async job submit for node 68d46f90a5c2c05277bcc5ed 2025-09-26 22:24:26,371 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 22:24:26,384 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d46f90a5c2c05277bcc5ed HTTP/1.1" 200 None 2025-09-26 22:24:26,384 - lava_callback - DEBUG - Processing node 68d46f90a5c2c05277bcc5ed: ltp-pty 2025-09-26 22:24:26,384 - lava_callback - ERROR - Error processing callback for node 68d46f90a5c2c05277bcc5ed: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:47330 - "GET / HTTP/1.0" 200 OK 2025-09-26 22:24:27,109 - lava_callback - INFO - Received callback for node 68d46f92a5c2c05277bcc5f0 INFO: 172.21.0.1:47338 - "POST /node/68d46f92a5c2c05277bcc5f0 HTTP/1.0" 202 Accepted 2025-09-26 22:24:27,115 - lava_callback - INFO - Starting async job submit for node 68d46f92a5c2c05277bcc5f0 2025-09-26 22:24:27,116 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 22:24:27,128 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d46f92a5c2c05277bcc5f0 HTTP/1.1" 200 None 2025-09-26 22:24:27,128 - lava_callback - DEBUG - Processing node 68d46f92a5c2c05277bcc5f0: ltp-sched 2025-09-26 22:24:27,128 - lava_callback - ERROR - Error processing callback for node 68d46f92a5c2c05277bcc5f0: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 22:24:27,864 - lava_callback - INFO - Received callback for node 68d46f96a5c2c05277bcc5f3 INFO: 172.21.0.1:47340 - "POST /node/68d46f96a5c2c05277bcc5f3 HTTP/1.0" 202 Accepted 2025-09-26 22:24:27,869 - lava_callback - INFO - Starting async job submit for node 68d46f96a5c2c05277bcc5f3 2025-09-26 22:24:27,870 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 22:24:27,883 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d46f96a5c2c05277bcc5f3 HTTP/1.1" 200 None 2025-09-26 22:24:27,884 - lava_callback - DEBUG - Processing node 68d46f96a5c2c05277bcc5f3: ltp-smoketest 2025-09-26 22:24:27,884 - lava_callback - ERROR - Error processing callback for node 68d46f96a5c2c05277bcc5f3: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 22:24:28,657 - lava_callback - INFO - Received callback for node 68d46f98a5c2c05277bcc5f6 INFO: 172.21.0.1:47352 - "POST /node/68d46f98a5c2c05277bcc5f6 HTTP/1.0" 202 Accepted 2025-09-26 22:24:28,663 - lava_callback - INFO - Starting async job submit for node 68d46f98a5c2c05277bcc5f6 2025-09-26 22:24:28,663 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 22:24:28,682 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d46f98a5c2c05277bcc5f6 HTTP/1.1" 200 None 2025-09-26 22:24:28,682 - lava_callback - DEBUG - Processing node 68d46f98a5c2c05277bcc5f6: ltp-syscalls 2025-09-26 22:24:28,682 - lava_callback - ERROR - Error processing callback for node 68d46f98a5c2c05277bcc5f6: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:46780 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46788 - "GET / HTTP/1.0" 200 OK 2025-09-26 22:24:44,567 - lava_callback - INFO - Received callback for node 68d46e72a5c2c05277bcc47e INFO: 172.21.0.1:37228 - "POST /node/68d46e72a5c2c05277bcc47e HTTP/1.0" 202 Accepted 2025-09-26 22:24:44,574 - lava_callback - INFO - Starting async job submit for node 68d46e72a5c2c05277bcc47e 2025-09-26 22:24:44,613 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 22:24:44,649 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d46e72a5c2c05277bcc47e HTTP/1.1" 200 None 2025-09-26 22:24:44,649 - lava_callback - DEBUG - Processing node 68d46e72a5c2c05277bcc47e: ltp-pty 2025-09-26 22:24:45,026 - lava_callback - ERROR - Error processing callback for node 68d46e72a5c2c05277bcc47e: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:37242 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42750 - "GET / HTTP/1.0" 200 OK 2025-09-26 22:25:06,199 - lava_callback - INFO - Received callback for node 68d46fb1a5c2c05277bcc611 INFO: 172.21.0.1:55300 - "POST /node/68d46fb1a5c2c05277bcc611 HTTP/1.0" 202 Accepted 2025-09-26 22:25:06,204 - lava_callback - INFO - Starting async job submit for node 68d46fb1a5c2c05277bcc611 2025-09-26 22:25:06,206 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 22:25:06,218 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d46fb1a5c2c05277bcc611 HTTP/1.1" 200 None 2025-09-26 22:25:06,218 - lava_callback - DEBUG - Processing node 68d46fb1a5c2c05277bcc611: ltp-timers 2025-09-26 22:25:06,218 - lava_callback - ERROR - Error processing callback for node 68d46fb1a5c2c05277bcc611: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:55310 - "GET / HTTP/1.0" 200 OK 2025-09-26 22:25:07,015 - lava_callback - INFO - Received callback for node 68d46fb3a5c2c05277bcc614 INFO: 172.21.0.1:55316 - "POST /node/68d46fb3a5c2c05277bcc614 HTTP/1.0" 202 Accepted 2025-09-26 22:25:07,021 - lava_callback - INFO - Starting async job submit for node 68d46fb3a5c2c05277bcc614 2025-09-26 22:25:07,021 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 22:25:07,055 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d46fb3a5c2c05277bcc614 HTTP/1.1" 200 None 2025-09-26 22:25:07,055 - lava_callback - DEBUG - Processing node 68d46fb3a5c2c05277bcc614: ltp-watchqueue 2025-09-26 22:25:07,055 - lava_callback - ERROR - Error processing callback for node 68d46fb3a5c2c05277bcc614: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:44288 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44290 - "GET / HTTP/1.0" 200 OK 2025-09-26 22:25:18,147 - lava_callback - INFO - Received callback for node 68d51900b70fb2a1e20f116a INFO: 172.21.0.1:44292 - "POST /node/68d51900b70fb2a1e20f116a HTTP/1.0" 202 Accepted 2025-09-26 22:25:18,153 - lava_callback - INFO - Starting async job submit for node 68d51900b70fb2a1e20f116a 2025-09-26 22:25:18,174 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 22:25:18,186 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d51900b70fb2a1e20f116a HTTP/1.1" 200 None 2025-09-26 22:25:18,187 - lava_callback - DEBUG - Processing node 68d51900b70fb2a1e20f116a: baseline-arm 2025-09-26 22:25:18,419 - lava_callback - ERROR - Error processing callback for node 68d51900b70fb2a1e20f116a: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None 2025-09-26 22:25:22,417 - lava_callback - INFO - Received callback for node 68d46fbda5c2c05277bcc629 INFO: 172.21.0.1:36126 - "POST /node/68d46fbda5c2c05277bcc629 HTTP/1.0" 202 Accepted 2025-09-26 22:25:22,422 - lava_callback - INFO - Starting async job submit for node 68d46fbda5c2c05277bcc629 2025-09-26 22:25:22,423 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 22:25:22,436 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d46fbda5c2c05277bcc629 HTTP/1.1" 200 None 2025-09-26 22:25:22,436 - lava_callback - DEBUG - Processing node 68d46fbda5c2c05277bcc629: baseline-arm 2025-09-26 22:25:22,436 - lava_callback - ERROR - Error processing callback for node 68d46fbda5c2c05277bcc629: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:36136 - "GET / HTTP/1.0" 200 OK 2025-09-26 22:25:23,098 - lava_callback - INFO - Received callback for node 68d46fc0a5c2c05277bcc62c INFO: 172.21.0.1:36144 - "POST /node/68d46fc0a5c2c05277bcc62c HTTP/1.0" 202 Accepted 2025-09-26 22:25:23,103 - lava_callback - INFO - Starting async job submit for node 68d46fc0a5c2c05277bcc62c 2025-09-26 22:25:23,104 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 22:25:23,116 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d46fc0a5c2c05277bcc62c HTTP/1.1" 200 None 2025-09-26 22:25:23,117 - lava_callback - DEBUG - Processing node 68d46fc0a5c2c05277bcc62c: baseline-arm 2025-09-26 22:25:23,117 - lava_callback - ERROR - Error processing callback for node 68d46fc0a5c2c05277bcc62c: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 22:25:23,751 - lava_callback - INFO - Received callback for node 68d46fcfa5c2c05277bcc64f INFO: 172.21.0.1:36152 - "POST /node/68d46fcfa5c2c05277bcc64f HTTP/1.0" 202 Accepted 2025-09-26 22:25:23,757 - lava_callback - INFO - Starting async job submit for node 68d46fcfa5c2c05277bcc64f 2025-09-26 22:25:23,758 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 22:25:23,774 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d46fcfa5c2c05277bcc64f HTTP/1.1" 200 None 2025-09-26 22:25:23,774 - lava_callback - DEBUG - Processing node 68d46fcfa5c2c05277bcc64f: baseline-arm 2025-09-26 22:25:23,774 - lava_callback - ERROR - Error processing callback for node 68d46fcfa5c2c05277bcc64f: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:36164 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34960 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34966 - "GET / HTTP/1.0" 200 OK 2025-09-26 22:25:42,361 - lava_callback - INFO - Received callback for node 68d46fcfa5c2c05277bcc652 INFO: 172.21.0.1:56894 - "POST /node/68d46fcfa5c2c05277bcc652 HTTP/1.0" 202 Accepted 2025-09-26 22:25:42,367 - lava_callback - INFO - Starting async job submit for node 68d46fcfa5c2c05277bcc652 2025-09-26 22:25:42,367 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 22:25:42,387 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d46fcfa5c2c05277bcc652 HTTP/1.1" 200 None 2025-09-26 22:25:42,388 - lava_callback - DEBUG - Processing node 68d46fcfa5c2c05277bcc652: baseline-arm 2025-09-26 22:25:42,388 - lava_callback - ERROR - Error processing callback for node 68d46fcfa5c2c05277bcc652: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 22:25:46,202 - lava_callback - INFO - Received callback for node 68d46fd8a5c2c05277bcc65b INFO: 172.21.0.1:56902 - "POST /node/68d46fd8a5c2c05277bcc65b HTTP/1.0" 202 Accepted 2025-09-26 22:25:46,207 - lava_callback - INFO - Starting async job submit for node 68d46fd8a5c2c05277bcc65b 2025-09-26 22:25:46,208 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 22:25:46,220 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d46fd8a5c2c05277bcc65b HTTP/1.1" 200 None 2025-09-26 22:25:46,221 - lava_callback - DEBUG - Processing node 68d46fd8a5c2c05277bcc65b: baseline-arm-broonie 2025-09-26 22:25:46,221 - lava_callback - ERROR - Error processing callback for node 68d46fd8a5c2c05277bcc65b: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:56918 - "GET / HTTP/1.0" 200 OK 2025-09-26 22:25:46,947 - lava_callback - INFO - Received callback for node 68d46fe1a5c2c05277bcc664 INFO: 172.21.0.1:56934 - "POST /node/68d46fe1a5c2c05277bcc664 HTTP/1.0" 202 Accepted 2025-09-26 22:25:46,953 - lava_callback - INFO - Starting async job submit for node 68d46fe1a5c2c05277bcc664 2025-09-26 22:25:46,953 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 22:25:46,966 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d46fe1a5c2c05277bcc664 HTTP/1.1" 200 None 2025-09-26 22:25:46,966 - lava_callback - DEBUG - Processing node 68d46fe1a5c2c05277bcc664: baseline-arm-broonie 2025-09-26 22:25:46,966 - lava_callback - ERROR - Error processing callback for node 68d46fe1a5c2c05277bcc664: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:44180 - "GET / HTTP/1.0" 200 OK 2025-09-26 22:26:06,334 - lava_callback - INFO - Received callback for node 68d46ff4a5c2c05277bcc687 INFO: 172.21.0.1:54256 - "POST /node/68d46ff4a5c2c05277bcc687 HTTP/1.0" 202 Accepted 2025-09-26 22:26:06,339 - lava_callback - INFO - Starting async job submit for node 68d46ff4a5c2c05277bcc687 2025-09-26 22:26:06,340 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 22:26:06,353 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d46ff4a5c2c05277bcc687 HTTP/1.1" 200 None 2025-09-26 22:26:06,353 - lava_callback - DEBUG - Processing node 68d46ff4a5c2c05277bcc687: ltp-cap-bounds 2025-09-26 22:26:06,353 - lava_callback - ERROR - Error processing callback for node 68d46ff4a5c2c05277bcc687: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:54264 - "GET / HTTP/1.0" 200 OK 2025-09-26 22:26:07,155 - lava_callback - INFO - Received callback for node 68d46ff7a5c2c05277bcc68a INFO: 172.21.0.1:54278 - "POST /node/68d46ff7a5c2c05277bcc68a HTTP/1.0" 202 Accepted 2025-09-26 22:26:07,161 - lava_callback - INFO - Starting async job submit for node 68d46ff7a5c2c05277bcc68a 2025-09-26 22:26:07,161 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 22:26:07,299 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d46ff7a5c2c05277bcc68a HTTP/1.1" 200 None 2025-09-26 22:26:07,299 - lava_callback - DEBUG - Processing node 68d46ff7a5c2c05277bcc68a: ltp-containers 2025-09-26 22:26:07,299 - lava_callback - ERROR - Error processing callback for node 68d46ff7a5c2c05277bcc68a: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 22:26:08,236 - lava_callback - INFO - Received callback for node 68d46ffaa5c2c05277bcc68d INFO: 172.21.0.1:54282 - "POST /node/68d46ffaa5c2c05277bcc68d HTTP/1.0" 202 Accepted 2025-09-26 22:26:08,243 - lava_callback - INFO - Starting async job submit for node 68d46ffaa5c2c05277bcc68d 2025-09-26 22:26:08,243 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 22:26:08,276 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d46ffaa5c2c05277bcc68d HTTP/1.1" 200 None 2025-09-26 22:26:08,276 - lava_callback - DEBUG - Processing node 68d46ffaa5c2c05277bcc68d: ltp-controllers 2025-09-26 22:26:08,277 - lava_callback - ERROR - Error processing callback for node 68d46ffaa5c2c05277bcc68d: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 22:26:09,008 - lava_callback - INFO - Received callback for node 68d46ffca5c2c05277bcc690 INFO: 172.21.0.1:54288 - "POST /node/68d46ffca5c2c05277bcc690 HTTP/1.0" 202 Accepted 2025-09-26 22:26:09,014 - lava_callback - INFO - Starting async job submit for node 68d46ffca5c2c05277bcc690 2025-09-26 22:26:09,015 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 22:26:09,027 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d46ffca5c2c05277bcc690 HTTP/1.1" 200 None 2025-09-26 22:26:09,028 - lava_callback - DEBUG - Processing node 68d46ffca5c2c05277bcc690: ltp-cve 2025-09-26 22:26:09,028 - lava_callback - ERROR - Error processing callback for node 68d46ffca5c2c05277bcc690: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:58338 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58352 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54934 - "GET / HTTP/1.0" 200 OK 2025-09-26 22:26:26,122 - lava_callback - INFO - Received callback for node 68d47002a5c2c05277bcc696 INFO: 172.21.0.1:54944 - "POST /node/68d47002a5c2c05277bcc696 HTTP/1.0" 202 Accepted 2025-09-26 22:26:26,128 - lava_callback - INFO - Starting async job submit for node 68d47002a5c2c05277bcc696 2025-09-26 22:26:26,129 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 22:26:26,141 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d47002a5c2c05277bcc696 HTTP/1.1" 200 None 2025-09-26 22:26:26,142 - lava_callback - DEBUG - Processing node 68d47002a5c2c05277bcc696: ltp-fs-bind 2025-09-26 22:26:26,142 - lava_callback - ERROR - Error processing callback for node 68d47002a5c2c05277bcc696: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:54946 - "GET / HTTP/1.0" 200 OK 2025-09-26 22:26:26,849 - lava_callback - INFO - Received callback for node 68d47004a5c2c05277bcc699 INFO: 172.21.0.1:54956 - "POST /node/68d47004a5c2c05277bcc699 HTTP/1.0" 202 Accepted 2025-09-26 22:26:26,855 - lava_callback - INFO - Starting async job submit for node 68d47004a5c2c05277bcc699 2025-09-26 22:26:26,856 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 22:26:26,889 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d47004a5c2c05277bcc699 HTTP/1.1" 200 None 2025-09-26 22:26:26,889 - lava_callback - DEBUG - Processing node 68d47004a5c2c05277bcc699: ltp-fsx 2025-09-26 22:26:26,889 - lava_callback - ERROR - Error processing callback for node 68d47004a5c2c05277bcc699: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 22:26:27,602 - lava_callback - INFO - Received callback for node 68d47007a5c2c05277bcc69c INFO: 172.21.0.1:54964 - "POST /node/68d47007a5c2c05277bcc69c HTTP/1.0" 202 Accepted 2025-09-26 22:26:27,608 - lava_callback - INFO - Starting async job submit for node 68d47007a5c2c05277bcc69c 2025-09-26 22:26:27,609 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 22:26:27,644 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d47007a5c2c05277bcc69c HTTP/1.1" 200 None 2025-09-26 22:26:27,644 - lava_callback - DEBUG - Processing node 68d47007a5c2c05277bcc69c: ltp-input 2025-09-26 22:26:27,644 - lava_callback - ERROR - Error processing callback for node 68d47007a5c2c05277bcc69c: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 22:26:28,397 - lava_callback - INFO - Received callback for node 68d4700aa5c2c05277bcc6a2 INFO: 172.21.0.1:54972 - "POST /node/68d4700aa5c2c05277bcc6a2 HTTP/1.0" 202 Accepted 2025-09-26 22:26:28,407 - lava_callback - INFO - Starting async job submit for node 68d4700aa5c2c05277bcc6a2 2025-09-26 22:26:28,408 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 22:26:28,423 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4700aa5c2c05277bcc6a2 HTTP/1.1" 200 None 2025-09-26 22:26:28,423 - lava_callback - DEBUG - Processing node 68d4700aa5c2c05277bcc6a2: ltp-ipc 2025-09-26 22:26:28,423 - lava_callback - ERROR - Error processing callback for node 68d4700aa5c2c05277bcc6a2: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 22:26:29,156 - lava_callback - INFO - Received callback for node 68d4700ca5c2c05277bcc6a5 INFO: 172.21.0.1:54978 - "POST /node/68d4700ca5c2c05277bcc6a5 HTTP/1.0" 202 Accepted 2025-09-26 22:26:29,162 - lava_callback - INFO - Starting async job submit for node 68d4700ca5c2c05277bcc6a5 2025-09-26 22:26:29,163 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 22:26:29,175 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4700ca5c2c05277bcc6a5 HTTP/1.1" 200 None 2025-09-26 22:26:29,175 - lava_callback - DEBUG - Processing node 68d4700ca5c2c05277bcc6a5: ltp-pty 2025-09-26 22:26:29,175 - lava_callback - ERROR - Error processing callback for node 68d4700ca5c2c05277bcc6a5: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 22:26:29,878 - lava_callback - INFO - Received callback for node 68d4700fa5c2c05277bcc6a8 INFO: 172.21.0.1:38626 - "POST /node/68d4700fa5c2c05277bcc6a8 HTTP/1.0" 202 Accepted 2025-09-26 22:26:29,884 - lava_callback - INFO - Starting async job submit for node 68d4700fa5c2c05277bcc6a8 2025-09-26 22:26:29,885 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 22:26:29,901 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4700fa5c2c05277bcc6a8 HTTP/1.1" 200 None 2025-09-26 22:26:29,902 - lava_callback - DEBUG - Processing node 68d4700fa5c2c05277bcc6a8: ltp-sched 2025-09-26 22:26:29,902 - lava_callback - ERROR - Error processing callback for node 68d4700fa5c2c05277bcc6a8: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 22:26:30,689 - lava_callback - INFO - Received callback for node 68d47012a5c2c05277bcc6ab INFO: 172.21.0.1:38632 - "POST /node/68d47012a5c2c05277bcc6ab HTTP/1.0" 202 Accepted 2025-09-26 22:26:30,695 - lava_callback - INFO - Starting async job submit for node 68d47012a5c2c05277bcc6ab 2025-09-26 22:26:30,696 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 22:26:30,710 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d47012a5c2c05277bcc6ab HTTP/1.1" 200 None 2025-09-26 22:26:30,711 - lava_callback - DEBUG - Processing node 68d47012a5c2c05277bcc6ab: ltp-smoketest 2025-09-26 22:26:30,711 - lava_callback - ERROR - Error processing callback for node 68d47012a5c2c05277bcc6ab: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 22:26:31,428 - lava_callback - INFO - Received callback for node 68d47015a5c2c05277bcc6ae INFO: 172.21.0.1:38638 - "POST /node/68d47015a5c2c05277bcc6ae HTTP/1.0" 202 Accepted 2025-09-26 22:26:31,434 - lava_callback - INFO - Starting async job submit for node 68d47015a5c2c05277bcc6ae 2025-09-26 22:26:31,435 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 22:26:31,447 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d47015a5c2c05277bcc6ae HTTP/1.1" 200 None 2025-09-26 22:26:31,447 - lava_callback - DEBUG - Processing node 68d47015a5c2c05277bcc6ae: ltp-syscalls 2025-09-26 22:26:31,448 - lava_callback - ERROR - Error processing callback for node 68d47015a5c2c05277bcc6ae: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:38646 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38648 - "GET / HTTP/1.0" 200 OK 2025-09-26 22:26:46,166 - lava_callback - INFO - Received callback for node 68d47018a5c2c05277bcc6b1 INFO: 172.21.0.1:40766 - "POST /node/68d47018a5c2c05277bcc6b1 HTTP/1.0" 202 Accepted 2025-09-26 22:26:46,172 - lava_callback - INFO - Starting async job submit for node 68d47018a5c2c05277bcc6b1 2025-09-26 22:26:46,173 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 22:26:46,185 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d47018a5c2c05277bcc6b1 HTTP/1.1" 200 None 2025-09-26 22:26:46,185 - lava_callback - DEBUG - Processing node 68d47018a5c2c05277bcc6b1: ltp-syscalls-ipc 2025-09-26 22:26:46,185 - lava_callback - ERROR - Error processing callback for node 68d47018a5c2c05277bcc6b1: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:40772 - "GET / HTTP/1.0" 200 OK 2025-09-26 22:26:46,919 - lava_callback - INFO - Received callback for node 68d4701ba5c2c05277bcc6b7 INFO: 172.21.0.1:40776 - "POST /node/68d4701ba5c2c05277bcc6b7 HTTP/1.0" 202 Accepted 2025-09-26 22:26:46,925 - lava_callback - INFO - Starting async job submit for node 68d4701ba5c2c05277bcc6b7 2025-09-26 22:26:46,926 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 22:26:46,939 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4701ba5c2c05277bcc6b7 HTTP/1.1" 200 None 2025-09-26 22:26:46,939 - lava_callback - DEBUG - Processing node 68d4701ba5c2c05277bcc6b7: ltp-timers 2025-09-26 22:26:46,939 - lava_callback - ERROR - Error processing callback for node 68d4701ba5c2c05277bcc6b7: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 22:26:47,702 - lava_callback - INFO - Received callback for node 68d4701ca5c2c05277bcc6ba INFO: 172.21.0.1:40788 - "POST /node/68d4701ca5c2c05277bcc6ba HTTP/1.0" 202 Accepted 2025-09-26 22:26:47,708 - lava_callback - INFO - Starting async job submit for node 68d4701ca5c2c05277bcc6ba 2025-09-26 22:26:47,709 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 22:26:47,722 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4701ca5c2c05277bcc6ba HTTP/1.1" 200 None 2025-09-26 22:26:47,722 - lava_callback - DEBUG - Processing node 68d4701ca5c2c05277bcc6ba: ltp-watchqueue 2025-09-26 22:26:47,722 - lava_callback - ERROR - Error processing callback for node 68d4701ca5c2c05277bcc6ba: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:55014 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45630 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36392 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36408 - "GET / HTTP/1.0" 200 OK 2025-09-26 22:27:22,814 - lava_callback - INFO - Received callback for node 68d51db6b70fb2a1e20f1844 INFO: 172.21.0.1:52476 - "POST /node/68d51db6b70fb2a1e20f1844 HTTP/1.0" 202 Accepted 2025-09-26 22:27:22,819 - lava_callback - INFO - Starting async job submit for node 68d51db6b70fb2a1e20f1844 2025-09-26 22:27:22,837 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 22:27:22,850 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d51db6b70fb2a1e20f1844 HTTP/1.1" 200 None 2025-09-26 22:27:22,850 - lava_callback - DEBUG - Processing node 68d51db6b70fb2a1e20f1844: baseline-arm64-broonie 2025-09-26 22:27:23,155 - lava_callback - ERROR - Error processing callback for node 68d51db6b70fb2a1e20f1844: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:52486 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52494 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34370 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34384 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54968 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54634 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46856 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34428 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34434 - "GET / HTTP/1.0" 200 OK 2025-09-26 22:28:17,698 - lava_callback - INFO - Received callback for node 68d51901b70fb2a1e20f116d INFO: 172.21.0.1:34440 - "POST /node/68d51901b70fb2a1e20f116d HTTP/1.0" 202 Accepted 2025-09-26 22:28:17,705 - lava_callback - INFO - Starting async job submit for node 68d51901b70fb2a1e20f116d 2025-09-26 22:28:17,727 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 22:28:17,757 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d51901b70fb2a1e20f116d HTTP/1.1" 200 None 2025-09-26 22:28:17,757 - lava_callback - DEBUG - Processing node 68d51901b70fb2a1e20f116d: baseline-arm 2025-09-26 22:28:18,017 - lava_callback - ERROR - Error processing callback for node 68d51901b70fb2a1e20f116d: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:60746 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60752 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50890 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50900 - "GET / HTTP/1.0" 200 OK 2025-09-26 22:28:42,804 - lava_callback - INFO - Received callback for node 68d46f8ba5c2c05277bcc5e7 INFO: 172.21.0.1:42522 - "POST /node/68d46f8ba5c2c05277bcc5e7 HTTP/1.0" 202 Accepted 2025-09-26 22:28:42,813 - lava_callback - INFO - Starting async job submit for node 68d46f8ba5c2c05277bcc5e7 2025-09-26 22:28:42,854 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 22:28:42,868 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d46f8ba5c2c05277bcc5e7 HTTP/1.1" 200 None 2025-09-26 22:28:42,869 - lava_callback - DEBUG - Processing node 68d46f8ba5c2c05277bcc5e7: ltp-input 2025-09-26 22:28:43,302 - lava_callback - ERROR - Error processing callback for node 68d46f8ba5c2c05277bcc5e7: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None 2025-09-26 22:28:44,426 - lava_callback - INFO - Received callback for node 68d46f8ea5c2c05277bcc5ea INFO: 172.21.0.1:42532 - "POST /node/68d46f8ea5c2c05277bcc5ea HTTP/1.0" 202 Accepted 2025-09-26 22:28:44,434 - lava_callback - INFO - Starting async job submit for node 68d46f8ea5c2c05277bcc5ea 2025-09-26 22:28:44,466 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 22:28:44,481 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d46f8ea5c2c05277bcc5ea HTTP/1.1" 200 None 2025-09-26 22:28:44,481 - lava_callback - DEBUG - Processing node 68d46f8ea5c2c05277bcc5ea: ltp-ipc 2025-09-26 22:28:44,903 - lava_callback - ERROR - Error processing callback for node 68d46f8ea5c2c05277bcc5ea: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:42542 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48156 - "GET / HTTP/1.0" 200 OK 2025-09-26 22:29:04,128 - lava_callback - INFO - Received callback for node 68d4871fa5c2c05277bcdd9f INFO: 172.21.0.1:38430 - "POST /node/68d4871fa5c2c05277bcdd9f HTTP/1.0" 202 Accepted 2025-09-26 22:29:04,135 - lava_callback - INFO - Starting async job submit for node 68d4871fa5c2c05277bcdd9f 2025-09-26 22:29:04,176 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 22:29:04,188 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4871fa5c2c05277bcdd9f HTTP/1.1" 200 None 2025-09-26 22:29:04,188 - lava_callback - DEBUG - Processing node 68d4871fa5c2c05277bcdd9f: baseline-arm64 2025-09-26 22:29:05,099 - lava_callback - ERROR - Error processing callback for node 68d4871fa5c2c05277bcdd9f: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:38436 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47240 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47256 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38462 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38468 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44128 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44142 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38776 - "GET / HTTP/1.0" 200 OK 2025-09-26 22:29:55,260 - lava_callback - INFO - Received callback for node 68d4877ba5c2c05277bcde1d INFO: 172.21.0.1:54854 - "POST /node/68d4877ba5c2c05277bcde1d HTTP/1.0" 202 Accepted 2025-09-26 22:29:55,267 - lava_callback - INFO - Starting async job submit for node 68d4877ba5c2c05277bcde1d 2025-09-26 22:29:55,307 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 22:29:55,322 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4877ba5c2c05277bcde1d HTTP/1.1" 200 None 2025-09-26 22:29:55,322 - lava_callback - DEBUG - Processing node 68d4877ba5c2c05277bcde1d: baseline-arm64 2025-09-26 22:29:56,186 - lava_callback - ERROR - Error processing callback for node 68d4877ba5c2c05277bcde1d: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:54868 - "GET / HTTP/1.0" 200 OK 2025-09-26 22:30:00,687 - lava_callback - INFO - Received callback for node 68d51902b70fb2a1e20f1173 INFO: 172.21.0.1:49802 - "POST /node/68d51902b70fb2a1e20f1173 HTTP/1.0" 202 Accepted 2025-09-26 22:30:00,694 - lava_callback - INFO - Starting async job submit for node 68d51902b70fb2a1e20f1173 2025-09-26 22:30:00,718 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 22:30:00,752 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d51902b70fb2a1e20f1173 HTTP/1.1" 200 None 2025-09-26 22:30:00,752 - lava_callback - DEBUG - Processing node 68d51902b70fb2a1e20f1173: baseline-arm 2025-09-26 22:30:00,982 - lava_callback - ERROR - Error processing callback for node 68d51902b70fb2a1e20f1173: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:49814 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55142 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55148 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54534 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54550 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46392 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46394 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45300 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49146 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54106 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42630 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42632 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50474 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50486 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40264 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40276 - "GET / HTTP/1.0" 200 OK 2025-09-26 22:31:44,003 - lava_callback - INFO - Received callback for node 68d4877ea5c2c05277bcde20 INFO: 172.21.0.1:41020 - "POST /node/68d4877ea5c2c05277bcde20 HTTP/1.0" 202 Accepted 2025-09-26 22:31:44,010 - lava_callback - INFO - Starting async job submit for node 68d4877ea5c2c05277bcde20 2025-09-26 22:31:44,051 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 22:31:44,075 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4877ea5c2c05277bcde20 HTTP/1.1" 200 None 2025-09-26 22:31:44,075 - lava_callback - DEBUG - Processing node 68d4877ea5c2c05277bcde20: baseline-arm64 2025-09-26 22:31:44,903 - lava_callback - ERROR - Error processing callback for node 68d4877ea5c2c05277bcde20: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:41022 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36830 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42410 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49092 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49106 - "GET / HTTP/1.0" 200 OK 2025-09-26 22:32:18,064 - lava_callback - INFO - Received callback for node 68d51903b70fb2a1e20f1176 INFO: 172.21.0.1:49116 - "POST /node/68d51903b70fb2a1e20f1176 HTTP/1.0" 202 Accepted 2025-09-26 22:32:18,070 - lava_callback - INFO - Starting async job submit for node 68d51903b70fb2a1e20f1176 2025-09-26 22:32:18,090 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 22:32:18,126 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d51903b70fb2a1e20f1176 HTTP/1.1" 200 None 2025-09-26 22:32:18,126 - lava_callback - DEBUG - Processing node 68d51903b70fb2a1e20f1176: baseline-arm 2025-09-26 22:32:18,381 - lava_callback - ERROR - Error processing callback for node 68d51903b70fb2a1e20f1176: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:54486 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54492 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46162 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46170 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51580 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43404 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46970 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50518 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50520 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49966 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49978 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55682 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55692 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35660 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49910 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46856 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39860 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39876 - "GET / HTTP/1.0" 200 OK 2025-09-26 22:34:20,654 - lava_callback - INFO - Received callback for node 68d51904b70fb2a1e20f1179 INFO: 172.21.0.1:49478 - "POST /node/68d51904b70fb2a1e20f1179 HTTP/1.0" 202 Accepted 2025-09-26 22:34:20,660 - lava_callback - INFO - Starting async job submit for node 68d51904b70fb2a1e20f1179 2025-09-26 22:34:20,681 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 22:34:20,717 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d51904b70fb2a1e20f1179 HTTP/1.1" 200 None 2025-09-26 22:34:20,718 - lava_callback - DEBUG - Processing node 68d51904b70fb2a1e20f1179: baseline-arm 2025-09-26 22:34:20,947 - lava_callback - ERROR - Error processing callback for node 68d51904b70fb2a1e20f1179: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:49490 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49502 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60296 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60306 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52874 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40248 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43822 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36884 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36886 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44732 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44744 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34076 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34086 - "GET / HTTP/1.0" 200 OK 2025-09-26 22:35:43,234 - lava_callback - INFO - Received callback for node 68d4708ea5c2c05277bcc6c1 INFO: 172.21.0.1:39706 - "POST /node/68d4708ea5c2c05277bcc6c1 HTTP/1.0" 202 Accepted 2025-09-26 22:35:43,241 - lava_callback - INFO - Starting async job submit for node 68d4708ea5c2c05277bcc6c1 2025-09-26 22:35:43,274 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 22:35:43,287 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4708ea5c2c05277bcc6c1 HTTP/1.1" 200 None 2025-09-26 22:35:43,288 - lava_callback - DEBUG - Processing node 68d4708ea5c2c05277bcc6c1: ltp-sched 2025-09-26 22:35:43,760 - lava_callback - ERROR - Error processing callback for node 68d4708ea5c2c05277bcc6c1: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:39720 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56820 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52920 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55858 - "GET /api/metrics HTTP/1.0" 200 OK 2025-09-26 22:36:15,792 - lava_callback - INFO - Received callback for node 68d51905b70fb2a1e20f117c INFO: 172.21.0.1:55870 - "POST /node/68d51905b70fb2a1e20f117c HTTP/1.0" 202 Accepted 2025-09-26 22:36:15,798 - lava_callback - INFO - Starting async job submit for node 68d51905b70fb2a1e20f117c 2025-09-26 22:36:15,837 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 22:36:15,854 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d51905b70fb2a1e20f117c HTTP/1.1" 200 None 2025-09-26 22:36:15,854 - lava_callback - DEBUG - Processing node 68d51905b70fb2a1e20f117c: baseline-arm 2025-09-26 22:36:16,113 - lava_callback - ERROR - Error processing callback for node 68d51905b70fb2a1e20f117c: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:55874 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36198 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36210 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58348 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58352 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52616 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43822 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55306 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43196 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43212 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54828 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54842 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50616 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50630 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40600 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42504 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33928 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38338 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38350 - "GET / HTTP/1.0" 200 OK 2025-09-26 22:38:20,553 - lava_callback - INFO - Received callback for node 68d51906b70fb2a1e20f1182 INFO: 172.21.0.1:51896 - "POST /node/68d51906b70fb2a1e20f1182 HTTP/1.0" 202 Accepted 2025-09-26 22:38:20,559 - lava_callback - INFO - Starting async job submit for node 68d51906b70fb2a1e20f1182 2025-09-26 22:38:20,580 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 22:38:20,603 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d51906b70fb2a1e20f1182 HTTP/1.1" 200 None 2025-09-26 22:38:20,604 - lava_callback - DEBUG - Processing node 68d51906b70fb2a1e20f1182: baseline-arm 2025-09-26 22:38:20,834 - lava_callback - ERROR - Error processing callback for node 68d51906b70fb2a1e20f1182: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:51910 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51912 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32786 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32790 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45704 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39834 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42230 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45456 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45466 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53282 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53294 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54768 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54784 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33572 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41790 - "GET / HTTP/1.0" 200 OK 2025-09-26 22:39:59,404 - lava_callback - INFO - Received callback for node 68d47335a5c2c05277bcc815 INFO: 172.21.0.1:57336 - "POST /node/68d47335a5c2c05277bcc815 HTTP/1.0" 202 Accepted 2025-09-26 22:39:59,409 - lava_callback - INFO - Starting async job submit for node 68d47335a5c2c05277bcc815 2025-09-26 22:39:59,410 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 22:39:59,440 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d47335a5c2c05277bcc815 HTTP/1.1" 200 None 2025-09-26 22:39:59,440 - lava_callback - DEBUG - Processing node 68d47335a5c2c05277bcc815: baseline-arm64-baylibre 2025-09-26 22:39:59,440 - lava_callback - ERROR - Error processing callback for node 68d47335a5c2c05277bcc815: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 22:40:01,335 - lava_callback - INFO - Received callback for node 68d46f04a5c2c05277bcc521 INFO: 172.21.0.1:57352 - "POST /node/68d46f04a5c2c05277bcc521 HTTP/1.0" 202 Accepted 2025-09-26 22:40:01,341 - lava_callback - INFO - Starting async job submit for node 68d46f04a5c2c05277bcc521 2025-09-26 22:40:01,371 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 22:40:01,383 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d46f04a5c2c05277bcc521 HTTP/1.1" 200 None 2025-09-26 22:40:01,384 - lava_callback - DEBUG - Processing node 68d46f04a5c2c05277bcc521: baseline-arm 2025-09-26 22:40:01,649 - lava_callback - ERROR - Error processing callback for node 68d46f04a5c2c05277bcc521: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:57354 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39698 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39712 - "GET / HTTP/1.0" 200 OK 2025-09-26 22:40:17,944 - lava_callback - INFO - Received callback for node 68d51907b70fb2a1e20f1185 INFO: 172.21.0.1:39726 - "POST /node/68d51907b70fb2a1e20f1185 HTTP/1.0" 202 Accepted 2025-09-26 22:40:17,950 - lava_callback - INFO - Starting async job submit for node 68d51907b70fb2a1e20f1185 2025-09-26 22:40:17,970 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 22:40:18,001 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d51907b70fb2a1e20f1185 HTTP/1.1" 200 None 2025-09-26 22:40:18,002 - lava_callback - DEBUG - Processing node 68d51907b70fb2a1e20f1185: baseline-arm 2025-09-26 22:40:18,249 - lava_callback - ERROR - Error processing callback for node 68d51907b70fb2a1e20f1185: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:59008 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59016 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43484 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43500 - "GET / HTTP/1.0" 200 OK 2025-09-26 22:40:39,458 - lava_callback - INFO - Received callback for node 68d4735ea5c2c05277bcc852 INFO: 172.21.0.1:37458 - "POST /node/68d4735ea5c2c05277bcc852 HTTP/1.0" 202 Accepted 2025-09-26 22:40:39,463 - lava_callback - INFO - Starting async job submit for node 68d4735ea5c2c05277bcc852 2025-09-26 22:40:39,464 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 22:40:39,479 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4735ea5c2c05277bcc852 HTTP/1.1" 200 None 2025-09-26 22:40:39,479 - lava_callback - DEBUG - Processing node 68d4735ea5c2c05277bcc852: baseline-arm64-baylibre 2025-09-26 22:40:39,479 - lava_callback - ERROR - Error processing callback for node 68d4735ea5c2c05277bcc852: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:37460 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57968 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33444 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37782 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37786 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60948 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60960 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49868 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49884 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44882 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55542 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49260 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60986 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60992 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56778 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56784 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46704 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46708 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37650 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54766 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52244 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51968 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51972 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57306 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57312 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40620 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40636 - "GET / HTTP/1.0" 200 OK 2025-09-26 22:43:43,792 - lava_callback - INFO - Received callback for node 68d46fafa5c2c05277bcc60e INFO: 172.21.0.1:56118 - "POST /node/68d46fafa5c2c05277bcc60e HTTP/1.0" 202 Accepted 2025-09-26 22:43:43,800 - lava_callback - INFO - Starting async job submit for node 68d46fafa5c2c05277bcc60e 2025-09-26 22:43:43,827 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 22:43:43,867 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d46fafa5c2c05277bcc60e HTTP/1.1" 200 None 2025-09-26 22:43:43,868 - lava_callback - DEBUG - Processing node 68d46fafa5c2c05277bcc60e: ltp-syscalls-ipc 2025-09-26 22:43:44,596 - lava_callback - ERROR - Error processing callback for node 68d46fafa5c2c05277bcc60e: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:56132 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42656 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53358 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45874 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45882 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56684 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56696 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56680 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56694 - "GET / HTTP/1.0" 200 OK 2025-09-26 22:44:45,270 - lava_callback - INFO - Received callback for node 68d47457a5c2c05277bcc926 INFO: 172.21.0.1:60712 - "POST /node/68d47457a5c2c05277bcc926 HTTP/1.0" 202 Accepted 2025-09-26 22:44:45,276 - lava_callback - INFO - Starting async job submit for node 68d47457a5c2c05277bcc926 2025-09-26 22:44:45,276 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 22:44:45,291 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d47457a5c2c05277bcc926 HTTP/1.1" 200 None 2025-09-26 22:44:45,292 - lava_callback - DEBUG - Processing node 68d47457a5c2c05277bcc926: baseline-arm64-clabbe 2025-09-26 22:44:45,292 - lava_callback - ERROR - Error processing callback for node 68d47457a5c2c05277bcc926: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:60720 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56176 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57312 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38760 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38764 - "GET / HTTP/1.0" 200 OK 2025-09-26 22:45:25,255 - lava_callback - INFO - Received callback for node 68d47479a5c2c05277bcc93f INFO: 172.21.0.1:45110 - "POST /node/68d47479a5c2c05277bcc93f HTTP/1.0" 202 Accepted 2025-09-26 22:45:25,260 - lava_callback - INFO - Starting async job submit for node 68d47479a5c2c05277bcc93f 2025-09-26 22:45:25,261 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 22:45:25,274 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d47479a5c2c05277bcc93f HTTP/1.1" 200 None 2025-09-26 22:45:25,274 - lava_callback - DEBUG - Processing node 68d47479a5c2c05277bcc93f: baseline-arm64-clabbe 2025-09-26 22:45:25,274 - lava_callback - ERROR - Error processing callback for node 68d47479a5c2c05277bcc93f: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 22:45:25,911 - lava_callback - INFO - Received callback for node 68d47480a5c2c05277bcc942 INFO: 172.21.0.1:45120 - "POST /node/68d47480a5c2c05277bcc942 HTTP/1.0" 202 Accepted 2025-09-26 22:45:25,916 - lava_callback - INFO - Starting async job submit for node 68d47480a5c2c05277bcc942 2025-09-26 22:45:25,917 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 22:45:25,930 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d47480a5c2c05277bcc942 HTTP/1.1" 200 None 2025-09-26 22:45:25,930 - lava_callback - DEBUG - Processing node 68d47480a5c2c05277bcc942: baseline-arm64-clabbe 2025-09-26 22:45:25,930 - lava_callback - ERROR - Error processing callback for node 68d47480a5c2c05277bcc942: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:45128 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45138 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42116 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42128 - "GET / HTTP/1.0" 200 OK 2025-09-26 22:45:45,307 - lava_callback - INFO - Received callback for node 68d47484a5c2c05277bcc945 INFO: 172.21.0.1:40462 - "POST /node/68d47484a5c2c05277bcc945 HTTP/1.0" 202 Accepted 2025-09-26 22:45:45,313 - lava_callback - INFO - Starting async job submit for node 68d47484a5c2c05277bcc945 2025-09-26 22:45:45,313 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 22:45:45,326 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d47484a5c2c05277bcc945 HTTP/1.1" 200 None 2025-09-26 22:45:45,326 - lava_callback - DEBUG - Processing node 68d47484a5c2c05277bcc945: baseline-arm64-clabbe 2025-09-26 22:45:45,327 - lava_callback - ERROR - Error processing callback for node 68d47484a5c2c05277bcc945: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 22:45:45,969 - lava_callback - INFO - Received callback for node 68d47489a5c2c05277bcc948 INFO: 172.21.0.1:40470 - "POST /node/68d47489a5c2c05277bcc948 HTTP/1.0" 202 Accepted 2025-09-26 22:45:45,974 - lava_callback - INFO - Starting async job submit for node 68d47489a5c2c05277bcc948 2025-09-26 22:45:45,975 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 22:45:45,988 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d47489a5c2c05277bcc948 HTTP/1.1" 200 None 2025-09-26 22:45:45,988 - lava_callback - DEBUG - Processing node 68d47489a5c2c05277bcc948: baseline-arm64-clabbe 2025-09-26 22:45:45,988 - lava_callback - ERROR - Error processing callback for node 68d47489a5c2c05277bcc948: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:40476 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58584 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38954 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54662 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54672 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35246 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35262 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50882 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50884 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35208 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33240 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45990 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57696 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57706 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41980 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41988 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44672 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44676 - "GET / HTTP/1.0" 200 OK 2025-09-26 22:47:43,693 - lava_callback - INFO - Received callback for node 68d470b8a5c2c05277bcc6d6 INFO: 172.21.0.1:57168 - "POST /node/68d470b8a5c2c05277bcc6d6 HTTP/1.0" 202 Accepted 2025-09-26 22:47:43,700 - lava_callback - INFO - Starting async job submit for node 68d470b8a5c2c05277bcc6d6 2025-09-26 22:47:43,728 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 22:47:43,741 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d470b8a5c2c05277bcc6d6 HTTP/1.1" 200 None 2025-09-26 22:47:43,742 - lava_callback - DEBUG - Processing node 68d470b8a5c2c05277bcc6d6: ltp-syscalls-ipc 2025-09-26 22:47:44,619 - lava_callback - ERROR - Error processing callback for node 68d470b8a5c2c05277bcc6d6: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:57178 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55458 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34156 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39362 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39376 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48056 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48070 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39158 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39170 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52762 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41876 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46448 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40928 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40944 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59696 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59708 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41854 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41868 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49954 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33850 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52196 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45816 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45832 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38050 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38066 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58296 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58302 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38250 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55098 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34832 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50322 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50324 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60562 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47016 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47024 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47028 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44212 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36972 - "GET / HTTP/1.0" 200 OK 2025-09-26 22:52:00,070 - lava_callback - INFO - Received callback for node 68d4760aa5c2c05277bccad3 INFO: 172.21.0.1:48006 - "POST /node/68d4760aa5c2c05277bccad3 HTTP/1.0" 202 Accepted 2025-09-26 22:52:00,075 - lava_callback - INFO - Starting async job submit for node 68d4760aa5c2c05277bccad3 2025-09-26 22:52:00,076 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 22:52:00,092 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4760aa5c2c05277bccad3 HTTP/1.1" 200 None 2025-09-26 22:52:00,093 - lava_callback - DEBUG - Processing node 68d4760aa5c2c05277bccad3: baseline-arm64-baylibre 2025-09-26 22:52:00,093 - lava_callback - ERROR - Error processing callback for node 68d4760aa5c2c05277bccad3: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:48018 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49786 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49794 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44376 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41790 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41798 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41808 - "GET / HTTP/1.0" 200 OK 2025-09-26 22:52:40,011 - lava_callback - INFO - Received callback for node 68d4762fa5c2c05277bccaf5 INFO: 172.21.0.1:59998 - "POST /node/68d4762fa5c2c05277bccaf5 HTTP/1.0" 202 Accepted 2025-09-26 22:52:40,016 - lava_callback - INFO - Starting async job submit for node 68d4762fa5c2c05277bccaf5 2025-09-26 22:52:40,017 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 22:52:40,029 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4762fa5c2c05277bccaf5 HTTP/1.1" 200 None 2025-09-26 22:52:40,030 - lava_callback - DEBUG - Processing node 68d4762fa5c2c05277bccaf5: baseline-arm64-baylibre 2025-09-26 22:52:40,030 - lava_callback - ERROR - Error processing callback for node 68d4762fa5c2c05277bccaf5: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:60006 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47294 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57196 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44844 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44860 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53500 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59920 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59934 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59950 - "GET / HTTP/1.0" 200 OK 2025-09-26 22:53:45,651 - lava_callback - INFO - Received callback for node 68d47673a5c2c05277bccb22 INFO: 172.21.0.1:38496 - "POST /node/68d47673a5c2c05277bccb22 HTTP/1.0" 202 Accepted 2025-09-26 22:53:45,656 - lava_callback - INFO - Starting async job submit for node 68d47673a5c2c05277bccb22 2025-09-26 22:53:45,657 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 22:53:45,673 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d47673a5c2c05277bccb22 HTTP/1.1" 200 None 2025-09-26 22:53:45,674 - lava_callback - DEBUG - Processing node 68d47673a5c2c05277bccb22: baseline-arm64-clabbe 2025-09-26 22:53:45,674 - lava_callback - ERROR - Error processing callback for node 68d47673a5c2c05277bccb22: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:38504 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40678 - "GET / HTTP/1.0" 200 OK 2025-09-26 22:54:05,622 - lava_callback - INFO - Received callback for node 68d47683a5c2c05277bccb2b INFO: 172.21.0.1:53236 - "POST /node/68d47683a5c2c05277bccb2b HTTP/1.0" 202 Accepted 2025-09-26 22:54:05,627 - lava_callback - INFO - Starting async job submit for node 68d47683a5c2c05277bccb2b 2025-09-26 22:54:05,628 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 22:54:05,641 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d47683a5c2c05277bccb2b HTTP/1.1" 200 None 2025-09-26 22:54:05,641 - lava_callback - DEBUG - Processing node 68d47683a5c2c05277bccb2b: baseline-arm64-clabbe 2025-09-26 22:54:05,641 - lava_callback - ERROR - Error processing callback for node 68d47683a5c2c05277bccb2b: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 22:54:06,310 - lava_callback - INFO - Received callback for node 68d47688a5c2c05277bccb2e INFO: 172.21.0.1:53248 - "POST /node/68d47688a5c2c05277bccb2e HTTP/1.0" 202 Accepted 2025-09-26 22:54:06,316 - lava_callback - INFO - Starting async job submit for node 68d47688a5c2c05277bccb2e 2025-09-26 22:54:06,316 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 22:54:06,329 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d47688a5c2c05277bccb2e HTTP/1.1" 200 None 2025-09-26 22:54:06,329 - lava_callback - DEBUG - Processing node 68d47688a5c2c05277bccb2e: baseline-arm64-clabbe 2025-09-26 22:54:06,329 - lava_callback - ERROR - Error processing callback for node 68d47688a5c2c05277bccb2e: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:53264 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52078 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52086 - "GET / HTTP/1.0" 200 OK 2025-09-26 22:54:21,361 - lava_callback - INFO - Received callback for node 68d51f4db70fb2a1e20f195c INFO: 172.21.0.1:33028 - "POST /node/68d51f4db70fb2a1e20f195c HTTP/1.0" 202 Accepted 2025-09-26 22:54:21,367 - lava_callback - INFO - Starting async job submit for node 68d51f4db70fb2a1e20f195c 2025-09-26 22:54:21,388 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 22:54:21,400 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d51f4db70fb2a1e20f195c HTTP/1.1" 200 None 2025-09-26 22:54:21,400 - lava_callback - DEBUG - Processing node 68d51f4db70fb2a1e20f195c: baseline-arm 2025-09-26 22:54:21,624 - lava_callback - ERROR - Error processing callback for node 68d51f4db70fb2a1e20f195c: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None 2025-09-26 22:54:25,663 - lava_callback - INFO - Received callback for node 68d4768da5c2c05277bccb31 INFO: 172.21.0.1:33030 - "POST /node/68d4768da5c2c05277bccb31 HTTP/1.0" 202 Accepted 2025-09-26 22:54:25,668 - lava_callback - INFO - Starting async job submit for node 68d4768da5c2c05277bccb31 2025-09-26 22:54:25,669 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 22:54:25,682 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4768da5c2c05277bccb31 HTTP/1.1" 200 None 2025-09-26 22:54:25,683 - lava_callback - DEBUG - Processing node 68d4768da5c2c05277bccb31: baseline-arm64-clabbe 2025-09-26 22:54:25,683 - lava_callback - ERROR - Error processing callback for node 68d4768da5c2c05277bccb31: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 22:54:26,352 - lava_callback - INFO - Received callback for node 68d47692a5c2c05277bccb38 INFO: 172.21.0.1:33034 - "POST /node/68d47692a5c2c05277bccb38 HTTP/1.0" 202 Accepted 2025-09-26 22:54:26,360 - lava_callback - INFO - Starting async job submit for node 68d47692a5c2c05277bccb38 2025-09-26 22:54:26,361 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 22:54:26,376 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d47692a5c2c05277bccb38 HTTP/1.1" 200 None 2025-09-26 22:54:26,377 - lava_callback - DEBUG - Processing node 68d47692a5c2c05277bccb38: baseline-arm64-clabbe 2025-09-26 22:54:26,377 - lava_callback - ERROR - Error processing callback for node 68d47692a5c2c05277bccb38: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:33036 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52170 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52184 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52186 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39406 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46768 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57622 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56466 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56468 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33134 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33890 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33902 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33906 - "GET / HTTP/1.0" 200 OK 2025-09-26 22:55:43,631 - lava_callback - INFO - Received callback for node 68d47836a5c2c05277bccc35 INFO: 172.21.0.1:38326 - "POST /node/68d47836a5c2c05277bccc35 HTTP/1.0" 202 Accepted 2025-09-26 22:55:43,637 - lava_callback - INFO - Starting async job submit for node 68d47836a5c2c05277bccc35 2025-09-26 22:55:43,658 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 22:55:43,671 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d47836a5c2c05277bccc35 HTTP/1.1" 200 None 2025-09-26 22:55:43,672 - lava_callback - DEBUG - Processing node 68d47836a5c2c05277bccc35: baseline-arm-broonie 2025-09-26 22:55:43,912 - lava_callback - ERROR - Error processing callback for node 68d47836a5c2c05277bccc35: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:38342 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53126 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56620 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52886 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52900 - "GET / HTTP/1.0" 200 OK 2025-09-26 22:56:19,624 - lava_callback - INFO - Received callback for node 68d51f50b70fb2a1e20f1968 INFO: 172.21.0.1:58258 - "POST /node/68d51f50b70fb2a1e20f1968 HTTP/1.0" 202 Accepted 2025-09-26 22:56:19,630 - lava_callback - INFO - Starting async job submit for node 68d51f50b70fb2a1e20f1968 2025-09-26 22:56:19,651 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 22:56:19,663 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d51f50b70fb2a1e20f1968 HTTP/1.1" 200 None 2025-09-26 22:56:19,664 - lava_callback - DEBUG - Processing node 68d51f50b70fb2a1e20f1968: baseline-arm 2025-09-26 22:56:19,897 - lava_callback - ERROR - Error processing callback for node 68d51f50b70fb2a1e20f1968: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:58274 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39226 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39230 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39232 - "GET / HTTP/1.0" 200 OK 2025-09-26 22:56:43,563 - lava_callback - INFO - Received callback for node 68d4782ca5c2c05277bccc22 INFO: 172.21.0.1:38822 - "POST /node/68d4782ca5c2c05277bccc22 HTTP/1.0" 202 Accepted 2025-09-26 22:56:43,568 - lava_callback - INFO - Starting async job submit for node 68d4782ca5c2c05277bccc22 2025-09-26 22:56:43,590 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 22:56:43,602 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4782ca5c2c05277bccc22 HTTP/1.1" 200 None 2025-09-26 22:56:43,603 - lava_callback - DEBUG - Processing node 68d4782ca5c2c05277bccc22: baseline-arm-broonie 2025-09-26 22:56:43,797 - lava_callback - ERROR - Error processing callback for node 68d4782ca5c2c05277bccc22: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:38824 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49594 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44670 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47026 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47038 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53762 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55610 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55614 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55624 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35892 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49242 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41162 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34624 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34636 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37820 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38868 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38880 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38892 - "GET / HTTP/1.0" 200 OK 2025-09-26 22:58:44,245 - lava_callback - INFO - Received callback for node 68d6d48c3c94e462c6881e39 INFO: 172.21.0.1:44882 - "POST /node/68d6d48c3c94e462c6881e39 HTTP/1.0" 202 Accepted 2025-09-26 22:58:44,252 - lava_callback - INFO - Starting async job submit for node 68d6d48c3c94e462c6881e39 2025-09-26 22:58:44,285 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 22:58:44,298 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6d48c3c94e462c6881e39 HTTP/1.1" 200 None 2025-09-26 22:58:44,298 - lava_callback - DEBUG - Processing node 68d6d48c3c94e462c6881e39: baseline-arm64-broonie 2025-09-26 22:58:44,769 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 22:58:44,909 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 22:58:44,909 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-68d6d48c3c94e462c6881e39/log.txt.gz 2025-09-26 22:58:44,919 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 22:58:45,074 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 22:58:45,075 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-68d6d48c3c94e462c6881e39/lava_callback.json.gz 2025-09-26 22:58:45,109 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 22:58:45,121 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6d48c3c94e462c6881e39 HTTP/1.1" 200 None 2025-09-26 22:58:45,122 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 22:58:45,134 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6bd8e3c94e462c68814f1 HTTP/1.1" 200 None 2025-09-26 22:58:45,135 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 22:58:45,156 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6d48c3c94e462c6881e39 HTTP/1.1" 200 None 2025-09-26 22:58:45,157 - lava_callback - INFO - Completed processing callback for node 68d6d48c3c94e462c6881e39 INFO: 172.21.0.1:44892 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54426 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45350 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52220 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52226 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43214 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37448 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37458 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37468 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53892 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57958 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42074 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38058 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38074 - "GET / HTTP/1.0" 200 OK 2025-09-26 23:00:24,181 - lava_callback - INFO - Received callback for node 68d477ffa5c2c05277bccbec INFO: 172.21.0.1:43462 - "POST /node/68d477ffa5c2c05277bccbec HTTP/1.0" 202 Accepted 2025-09-26 23:00:24,187 - lava_callback - INFO - Starting async job submit for node 68d477ffa5c2c05277bccbec 2025-09-26 23:00:24,188 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 23:00:24,217 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d477ffa5c2c05277bccbec HTTP/1.1" 200 None 2025-09-26 23:00:24,218 - lava_callback - DEBUG - Processing node 68d477ffa5c2c05277bccbec: baseline-arm 2025-09-26 23:00:24,218 - lava_callback - ERROR - Error processing callback for node 68d477ffa5c2c05277bccbec: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 23:00:24,864 - lava_callback - INFO - Received callback for node 68d47800a5c2c05277bccbef INFO: 172.21.0.1:43468 - "POST /node/68d47800a5c2c05277bccbef HTTP/1.0" 202 Accepted 2025-09-26 23:00:24,871 - lava_callback - INFO - Starting async job submit for node 68d47800a5c2c05277bccbef 2025-09-26 23:00:24,872 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 23:00:24,885 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d47800a5c2c05277bccbef HTTP/1.1" 200 None 2025-09-26 23:00:24,885 - lava_callback - DEBUG - Processing node 68d47800a5c2c05277bccbef: baseline-arm 2025-09-26 23:00:24,885 - lava_callback - ERROR - Error processing callback for node 68d47800a5c2c05277bccbef: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:43472 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48272 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48286 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48294 - "GET / HTTP/1.0" 200 OK 2025-09-26 23:00:43,775 - lava_callback - INFO - Received callback for node 68d47813a5c2c05277bccc04 INFO: 172.21.0.1:43216 - "POST /node/68d47813a5c2c05277bccc04 HTTP/1.0" 202 Accepted 2025-09-26 23:00:43,780 - lava_callback - INFO - Starting async job submit for node 68d47813a5c2c05277bccc04 2025-09-26 23:00:43,781 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 23:00:43,794 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d47813a5c2c05277bccc04 HTTP/1.1" 200 None 2025-09-26 23:00:43,794 - lava_callback - DEBUG - Processing node 68d47813a5c2c05277bccc04: baseline-arm 2025-09-26 23:00:43,794 - lava_callback - ERROR - Error processing callback for node 68d47813a5c2c05277bccc04: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 23:00:44,423 - lava_callback - INFO - Received callback for node 68d47815a5c2c05277bccc07 INFO: 172.21.0.1:43224 - "POST /node/68d47815a5c2c05277bccc07 HTTP/1.0" 202 Accepted 2025-09-26 23:00:44,428 - lava_callback - INFO - Starting async job submit for node 68d47815a5c2c05277bccc07 2025-09-26 23:00:44,429 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 23:00:44,459 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d47815a5c2c05277bccc07 HTTP/1.1" 200 None 2025-09-26 23:00:44,459 - lava_callback - DEBUG - Processing node 68d47815a5c2c05277bccc07: baseline-arm 2025-09-26 23:00:44,459 - lava_callback - ERROR - Error processing callback for node 68d47815a5c2c05277bccc07: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:43232 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44108 - "GET / HTTP/1.0" 200 OK 2025-09-26 23:01:04,005 - lava_callback - INFO - Received callback for node 68d47863a5c2c05277bccc62 INFO: 172.21.0.1:42990 - "POST /node/68d47863a5c2c05277bccc62 HTTP/1.0" 202 Accepted 2025-09-26 23:01:04,011 - lava_callback - INFO - Starting async job submit for node 68d47863a5c2c05277bccc62 2025-09-26 23:01:04,037 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 23:01:04,050 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d47863a5c2c05277bccc62 HTTP/1.1" 200 None 2025-09-26 23:01:04,051 - lava_callback - DEBUG - Processing node 68d47863a5c2c05277bccc62: ltp-cap-bounds 2025-09-26 23:01:04,382 - lava_callback - ERROR - Error processing callback for node 68d47863a5c2c05277bccc62: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:42996 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49644 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49648 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51130 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59354 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59362 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59370 - "GET / HTTP/1.0" 200 OK 2025-09-26 23:01:40,352 - lava_callback - INFO - Received callback for node 68d52137b70fb2a1e20f1a84 INFO: 172.21.0.1:40602 - "POST /node/68d52137b70fb2a1e20f1a84 HTTP/1.0" 202 Accepted 2025-09-26 23:01:40,359 - lava_callback - INFO - Starting async job submit for node 68d52137b70fb2a1e20f1a84 2025-09-26 23:01:40,379 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 23:01:40,393 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d52137b70fb2a1e20f1a84 HTTP/1.1" 200 None 2025-09-26 23:01:40,394 - lava_callback - DEBUG - Processing node 68d52137b70fb2a1e20f1a84: baseline-arm 2025-09-26 23:01:40,657 - lava_callback - ERROR - Error processing callback for node 68d52137b70fb2a1e20f1a84: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None 2025-09-26 23:01:46,026 - lava_callback - INFO - Received callback for node 68d47846a5c2c05277bccc41 INFO: 172.21.0.1:40604 - "POST /node/68d47846a5c2c05277bccc41 HTTP/1.0" 202 Accepted 2025-09-26 23:01:46,037 - lava_callback - INFO - Starting async job submit for node 68d47846a5c2c05277bccc41 2025-09-26 23:01:46,038 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 23:01:46,083 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d47846a5c2c05277bccc41 HTTP/1.1" 200 None 2025-09-26 23:01:46,084 - lava_callback - DEBUG - Processing node 68d47846a5c2c05277bccc41: baseline-arm-clabbe 2025-09-26 23:01:46,084 - lava_callback - ERROR - Error processing callback for node 68d47846a5c2c05277bccc41: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:40614 - "GET / HTTP/1.0" 200 OK 2025-09-26 23:01:46,691 - lava_callback - INFO - Received callback for node 68d4784aa5c2c05277bccc47 INFO: 172.21.0.1:40628 - "POST /node/68d4784aa5c2c05277bccc47 HTTP/1.0" 202 Accepted 2025-09-26 23:01:46,696 - lava_callback - INFO - Starting async job submit for node 68d4784aa5c2c05277bccc47 2025-09-26 23:01:46,697 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 23:01:46,731 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4784aa5c2c05277bccc47 HTTP/1.1" 200 None 2025-09-26 23:01:46,732 - lava_callback - DEBUG - Processing node 68d4784aa5c2c05277bccc47: baseline-arm-clabbe 2025-09-26 23:01:46,732 - lava_callback - ERROR - Error processing callback for node 68d4784aa5c2c05277bccc47: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 23:01:47,349 - lava_callback - INFO - Received callback for node 68d4784da5c2c05277bccc4a INFO: 172.21.0.1:40634 - "POST /node/68d4784da5c2c05277bccc4a HTTP/1.0" 202 Accepted 2025-09-26 23:01:47,354 - lava_callback - INFO - Starting async job submit for node 68d4784da5c2c05277bccc4a 2025-09-26 23:01:47,355 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 23:01:47,368 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4784da5c2c05277bccc4a HTTP/1.1" 200 None 2025-09-26 23:01:47,369 - lava_callback - DEBUG - Processing node 68d4784da5c2c05277bccc4a: baseline-arm-clabbe 2025-09-26 23:01:47,369 - lava_callback - ERROR - Error processing callback for node 68d4784da5c2c05277bccc4a: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:40128 - "GET / HTTP/1.0" 200 OK 2025-09-26 23:02:06,006 - lava_callback - INFO - Received callback for node 68d47858a5c2c05277bccc56 INFO: 172.21.0.1:44288 - "POST /node/68d47858a5c2c05277bccc56 HTTP/1.0" 202 Accepted 2025-09-26 23:02:06,012 - lava_callback - INFO - Starting async job submit for node 68d47858a5c2c05277bccc56 2025-09-26 23:02:06,013 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 23:02:06,064 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d47858a5c2c05277bccc56 HTTP/1.1" 200 None 2025-09-26 23:02:06,064 - lava_callback - DEBUG - Processing node 68d47858a5c2c05277bccc56: baseline-arm-clabbe 2025-09-26 23:02:06,064 - lava_callback - ERROR - Error processing callback for node 68d47858a5c2c05277bccc56: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:44294 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49630 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49642 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60882 - "GET / HTTP/1.0" 200 OK 2025-09-26 23:02:27,972 - lava_callback - INFO - Received callback for node 68d47870a5c2c05277bccc71 INFO: 172.21.0.1:60892 - "POST /node/68d47870a5c2c05277bccc71 HTTP/1.0" 202 Accepted 2025-09-26 23:02:27,978 - lava_callback - INFO - Starting async job submit for node 68d47870a5c2c05277bccc71 2025-09-26 23:02:27,979 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 23:02:28,001 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d47870a5c2c05277bccc71 HTTP/1.1" 200 None 2025-09-26 23:02:28,001 - lava_callback - DEBUG - Processing node 68d47870a5c2c05277bccc71: ltp-fs-bind 2025-09-26 23:02:28,001 - lava_callback - ERROR - Error processing callback for node 68d47870a5c2c05277bccc71: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 23:02:28,771 - lava_callback - INFO - Received callback for node 68d47872a5c2c05277bccc74 INFO: 172.21.0.1:60904 - "POST /node/68d47872a5c2c05277bccc74 HTTP/1.0" 202 Accepted 2025-09-26 23:02:28,777 - lava_callback - INFO - Starting async job submit for node 68d47872a5c2c05277bccc74 2025-09-26 23:02:28,778 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 23:02:28,791 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d47872a5c2c05277bccc74 HTTP/1.1" 200 None 2025-09-26 23:02:28,791 - lava_callback - DEBUG - Processing node 68d47872a5c2c05277bccc74: ltp-fsx 2025-09-26 23:02:28,792 - lava_callback - ERROR - Error processing callback for node 68d47872a5c2c05277bccc74: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 23:02:29,508 - lava_callback - INFO - Received callback for node 68d47875a5c2c05277bccc77 INFO: 172.21.0.1:44088 - "POST /node/68d47875a5c2c05277bccc77 HTTP/1.0" 202 Accepted 2025-09-26 23:02:29,514 - lava_callback - INFO - Starting async job submit for node 68d47875a5c2c05277bccc77 2025-09-26 23:02:29,515 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 23:02:29,527 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d47875a5c2c05277bccc77 HTTP/1.1" 200 None 2025-09-26 23:02:29,527 - lava_callback - DEBUG - Processing node 68d47875a5c2c05277bccc77: ltp-input 2025-09-26 23:02:29,527 - lava_callback - ERROR - Error processing callback for node 68d47875a5c2c05277bccc77: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 23:02:30,368 - lava_callback - INFO - Received callback for node 68d47878a5c2c05277bccc7a INFO: 172.21.0.1:44090 - "POST /node/68d47878a5c2c05277bccc7a HTTP/1.0" 202 Accepted 2025-09-26 23:02:30,374 - lava_callback - INFO - Starting async job submit for node 68d47878a5c2c05277bccc7a 2025-09-26 23:02:30,375 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 23:02:30,388 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d47878a5c2c05277bccc7a HTTP/1.1" 200 None 2025-09-26 23:02:30,388 - lava_callback - DEBUG - Processing node 68d47878a5c2c05277bccc7a: ltp-ipc 2025-09-26 23:02:30,389 - lava_callback - ERROR - Error processing callback for node 68d47878a5c2c05277bccc7a: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 23:02:31,168 - lava_callback - INFO - Received callback for node 68d4787aa5c2c05277bccc7d INFO: 172.21.0.1:44106 - "POST /node/68d4787aa5c2c05277bccc7d HTTP/1.0" 202 Accepted 2025-09-26 23:02:31,174 - lava_callback - INFO - Starting async job submit for node 68d4787aa5c2c05277bccc7d 2025-09-26 23:02:31,175 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 23:02:31,216 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4787aa5c2c05277bccc7d HTTP/1.1" 200 None 2025-09-26 23:02:31,217 - lava_callback - DEBUG - Processing node 68d4787aa5c2c05277bccc7d: ltp-pty 2025-09-26 23:02:31,217 - lava_callback - ERROR - Error processing callback for node 68d4787aa5c2c05277bccc7d: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 23:02:31,962 - lava_callback - INFO - Received callback for node 68d4787ca5c2c05277bccc80 INFO: 172.21.0.1:44110 - "POST /node/68d4787ca5c2c05277bccc80 HTTP/1.0" 202 Accepted 2025-09-26 23:02:31,968 - lava_callback - INFO - Starting async job submit for node 68d4787ca5c2c05277bccc80 2025-09-26 23:02:31,969 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.21.0.1:44122 - "GET / HTTP/1.0" 200 OK 2025-09-26 23:02:32,104 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4787ca5c2c05277bccc80 HTTP/1.1" 200 None 2025-09-26 23:02:32,105 - lava_callback - DEBUG - Processing node 68d4787ca5c2c05277bccc80: ltp-sched 2025-09-26 23:02:32,105 - lava_callback - ERROR - Error processing callback for node 68d4787ca5c2c05277bccc80: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 23:02:32,735 - lava_callback - INFO - Received callback for node 68d47880a5c2c05277bccc83 INFO: 172.21.0.1:44128 - "POST /node/68d47880a5c2c05277bccc83 HTTP/1.0" 202 Accepted 2025-09-26 23:02:32,741 - lava_callback - INFO - Starting async job submit for node 68d47880a5c2c05277bccc83 2025-09-26 23:02:32,742 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 23:02:32,754 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d47880a5c2c05277bccc83 HTTP/1.1" 200 None 2025-09-26 23:02:32,754 - lava_callback - DEBUG - Processing node 68d47880a5c2c05277bccc83: ltp-smoketest 2025-09-26 23:02:32,754 - lava_callback - ERROR - Error processing callback for node 68d47880a5c2c05277bccc83: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 23:02:33,532 - lava_callback - INFO - Received callback for node 68d47885a5c2c05277bccc89 INFO: 172.21.0.1:44144 - "POST /node/68d47885a5c2c05277bccc89 HTTP/1.0" 202 Accepted 2025-09-26 23:02:33,538 - lava_callback - INFO - Starting async job submit for node 68d47885a5c2c05277bccc89 2025-09-26 23:02:33,539 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 23:02:33,559 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d47885a5c2c05277bccc89 HTTP/1.1" 200 None 2025-09-26 23:02:33,559 - lava_callback - DEBUG - Processing node 68d47885a5c2c05277bccc89: ltp-syscalls-ipc 2025-09-26 23:02:33,559 - lava_callback - ERROR - Error processing callback for node 68d47885a5c2c05277bccc89: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 23:02:34,279 - lava_callback - INFO - Received callback for node 68d47888a5c2c05277bccc8c INFO: 172.21.0.1:44148 - "POST /node/68d47888a5c2c05277bccc8c HTTP/1.0" 202 Accepted 2025-09-26 23:02:34,285 - lava_callback - INFO - Starting async job submit for node 68d47888a5c2c05277bccc8c 2025-09-26 23:02:34,285 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 23:02:34,298 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d47888a5c2c05277bccc8c HTTP/1.1" 200 None 2025-09-26 23:02:34,298 - lava_callback - DEBUG - Processing node 68d47888a5c2c05277bccc8c: ltp-timers 2025-09-26 23:02:34,298 - lava_callback - ERROR - Error processing callback for node 68d47888a5c2c05277bccc8c: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:44154 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44156 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35976 - "GET / HTTP/1.0" 200 OK 2025-09-26 23:02:48,305 - lava_callback - INFO - Received callback for node 68d4788aa5c2c05277bccc8f INFO: 172.21.0.1:35988 - "POST /node/68d4788aa5c2c05277bccc8f HTTP/1.0" 202 Accepted 2025-09-26 23:02:48,311 - lava_callback - INFO - Starting async job submit for node 68d4788aa5c2c05277bccc8f 2025-09-26 23:02:48,312 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 23:02:48,329 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4788aa5c2c05277bccc8f HTTP/1.1" 200 None 2025-09-26 23:02:48,329 - lava_callback - DEBUG - Processing node 68d4788aa5c2c05277bccc8f: ltp-watchqueue 2025-09-26 23:02:48,329 - lava_callback - ERROR - Error processing callback for node 68d4788aa5c2c05277bccc8f: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:44326 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46052 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52752 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52760 - "GET / HTTP/1.0" 200 OK 2025-09-26 23:03:24,207 - lava_callback - INFO - Received callback for node 68d47865a5c2c05277bccc65 INFO: 172.21.0.1:37496 - "POST /node/68d47865a5c2c05277bccc65 HTTP/1.0" 202 Accepted 2025-09-26 23:03:24,214 - lava_callback - INFO - Starting async job submit for node 68d47865a5c2c05277bccc65 2025-09-26 23:03:24,313 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 23:03:24,349 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d47865a5c2c05277bccc65 HTTP/1.1" 200 None 2025-09-26 23:03:24,349 - lava_callback - DEBUG - Processing node 68d47865a5c2c05277bccc65: ltp-containers 2025-09-26 23:03:25,164 - lava_callback - ERROR - Error processing callback for node 68d47865a5c2c05277bccc65: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:37508 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39280 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39296 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39298 - "GET / HTTP/1.0" 200 OK 2025-09-26 23:03:42,743 - lava_callback - INFO - Received callback for node 68d52138b70fb2a1e20f1a87 INFO: 172.21.0.1:50806 - "POST /node/68d52138b70fb2a1e20f1a87 HTTP/1.0" 202 Accepted 2025-09-26 23:03:42,749 - lava_callback - INFO - Starting async job submit for node 68d52138b70fb2a1e20f1a87 2025-09-26 23:03:42,769 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 23:03:42,807 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d52138b70fb2a1e20f1a87 HTTP/1.1" 200 None 2025-09-26 23:03:42,807 - lava_callback - DEBUG - Processing node 68d52138b70fb2a1e20f1a87: baseline-arm 2025-09-26 23:03:43,038 - lava_callback - ERROR - Error processing callback for node 68d52138b70fb2a1e20f1a87: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None 2025-09-26 23:03:46,063 - lava_callback - INFO - Received callback for node 68d6deae3c94e462c6882190 INFO: 172.21.0.1:50808 - "POST /node/68d6deae3c94e462c6882190 HTTP/1.0" 202 Accepted 2025-09-26 23:03:46,069 - lava_callback - INFO - Starting async job submit for node 68d6deae3c94e462c6882190 2025-09-26 23:03:46,105 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 23:03:46,137 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6deae3c94e462c6882190 HTTP/1.1" 200 None 2025-09-26 23:03:46,138 - lava_callback - DEBUG - Processing node 68d6deae3c94e462c6882190: baseline-arm64-broonie INFO: 172.21.0.1:50816 - "GET / HTTP/1.0" 200 OK 2025-09-26 23:03:46,625 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 23:03:46,774 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 23:03:46,774 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-68d6deae3c94e462c6882190/log.txt.gz 2025-09-26 23:03:46,784 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 23:03:46,918 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 23:03:46,919 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-68d6deae3c94e462c6882190/lava_callback.json.gz 2025-09-26 23:03:46,953 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 23:03:46,965 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6deae3c94e462c6882190 HTTP/1.1" 200 None 2025-09-26 23:03:46,966 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 23:03:47,001 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d62c44c1bc82a30b37a916 HTTP/1.1" 200 None 2025-09-26 23:03:47,002 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 23:03:47,027 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6deae3c94e462c6882190 HTTP/1.1" 200 None 2025-09-26 23:03:47,028 - lava_callback - INFO - Completed processing callback for node 68d6deae3c94e462c6882190 INFO: 172.21.0.1:49896 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40142 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46168 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46174 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37128 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56394 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56396 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56410 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34858 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38740 - "GET / HTTP/1.0" 200 OK 2025-09-26 23:05:04,975 - lava_callback - INFO - Received callback for node 68d6e4cf3c94e462c6882218 INFO: 172.21.0.1:44046 - "POST /node/68d6e4cf3c94e462c6882218 HTTP/1.0" 202 Accepted 2025-09-26 23:05:04,981 - lava_callback - INFO - Starting async job submit for node 68d6e4cf3c94e462c6882218 2025-09-26 23:05:05,002 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 23:05:05,036 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6e4cf3c94e462c6882218 HTTP/1.1" 200 None 2025-09-26 23:05:05,037 - lava_callback - DEBUG - Processing node 68d6e4cf3c94e462c6882218: baseline-arm64-broonie 2025-09-26 23:05:05,330 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 23:05:05,465 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 23:05:05,466 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-68d6e4cf3c94e462c6882218/log.txt.gz 2025-09-26 23:05:05,473 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 23:05:05,606 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 23:05:05,607 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-68d6e4cf3c94e462c6882218/lava_callback.json.gz 2025-09-26 23:05:05,629 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 23:05:05,643 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6e4cf3c94e462c6882218 HTTP/1.1" 200 None 2025-09-26 23:05:05,644 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 23:05:05,676 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d5dc8530c58dac7d9410cb HTTP/1.1" 200 None 2025-09-26 23:05:05,677 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 23:05:05,706 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6e4cf3c94e462c6882218 HTTP/1.1" 200 None 2025-09-26 23:05:05,707 - lava_callback - INFO - Completed processing callback for node 68d6e4cf3c94e462c6882218 INFO: 172.21.0.1:44048 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55106 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55114 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50984 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34076 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34092 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34108 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48452 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39242 - "GET / HTTP/1.0" 200 OK 2025-09-26 23:05:59,506 - lava_callback - INFO - Received callback for node 68d52139b70fb2a1e20f1a8a INFO: 172.21.0.1:48284 - "POST /node/68d52139b70fb2a1e20f1a8a HTTP/1.0" 202 Accepted 2025-09-26 23:05:59,512 - lava_callback - INFO - Starting async job submit for node 68d52139b70fb2a1e20f1a8a 2025-09-26 23:05:59,532 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 23:05:59,564 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d52139b70fb2a1e20f1a8a HTTP/1.1" 200 None 2025-09-26 23:05:59,564 - lava_callback - DEBUG - Processing node 68d52139b70fb2a1e20f1a8a: baseline-arm 2025-09-26 23:05:59,793 - lava_callback - ERROR - Error processing callback for node 68d52139b70fb2a1e20f1a8a: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:48290 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57034 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57050 - "GET / HTTP/1.0" 200 OK 2025-09-26 23:06:20,837 - lava_callback - INFO - Received callback for node 68d4795da5c2c05277bccefd INFO: 172.21.0.1:44332 - "POST /node/68d4795da5c2c05277bccefd HTTP/1.0" 202 Accepted 2025-09-26 23:06:20,842 - lava_callback - INFO - Starting async job submit for node 68d4795da5c2c05277bccefd 2025-09-26 23:06:20,843 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 23:06:20,855 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4795da5c2c05277bccefd HTTP/1.1" 200 None 2025-09-26 23:06:20,856 - lava_callback - DEBUG - Processing node 68d4795da5c2c05277bccefd: baseline-arm64-baylibre 2025-09-26 23:06:20,856 - lava_callback - ERROR - Error processing callback for node 68d4795da5c2c05277bccefd: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:44334 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35778 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35786 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35796 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55992 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41814 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52426 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35182 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35192 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34924 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50926 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50938 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50948 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48502 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60538 - "GET / HTTP/1.0" 200 OK 2025-09-26 23:07:59,070 - lava_callback - INFO - Received callback for node 68d5213ab70fb2a1e20f1a8d INFO: 172.21.0.1:60542 - "POST /node/68d5213ab70fb2a1e20f1a8d HTTP/1.0" 202 Accepted 2025-09-26 23:07:59,076 - lava_callback - INFO - Starting async job submit for node 68d5213ab70fb2a1e20f1a8d 2025-09-26 23:07:59,115 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 23:07:59,149 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d5213ab70fb2a1e20f1a8d HTTP/1.1" 200 None 2025-09-26 23:07:59,150 - lava_callback - DEBUG - Processing node 68d5213ab70fb2a1e20f1a8d: baseline-arm 2025-09-26 23:07:59,391 - lava_callback - ERROR - Error processing callback for node 68d5213ab70fb2a1e20f1a8d: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:46770 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48114 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48126 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41066 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35132 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35138 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35148 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52732 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60036 - "GET / HTTP/1.0" 200 OK 2025-09-26 23:09:06,412 - lava_callback - INFO - Received callback for node 68d479f8a5c2c05277bccf85 INFO: 172.21.0.1:52888 - "POST /node/68d479f8a5c2c05277bccf85 HTTP/1.0" 202 Accepted 2025-09-26 23:09:06,417 - lava_callback - INFO - Starting async job submit for node 68d479f8a5c2c05277bccf85 2025-09-26 23:09:06,418 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 23:09:06,440 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d479f8a5c2c05277bccf85 HTTP/1.1" 200 None 2025-09-26 23:09:06,440 - lava_callback - DEBUG - Processing node 68d479f8a5c2c05277bccf85: baseline-arm64-clabbe 2025-09-26 23:09:06,440 - lava_callback - ERROR - Error processing callback for node 68d479f8a5c2c05277bccf85: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:52894 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40804 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40806 - "GET / HTTP/1.0" 200 OK 2025-09-26 23:09:26,376 - lava_callback - INFO - Received callback for node 68d47a11a5c2c05277bccf8e INFO: 172.21.0.1:48874 - "POST /node/68d47a11a5c2c05277bccf8e HTTP/1.0" 202 Accepted 2025-09-26 23:09:26,382 - lava_callback - INFO - Starting async job submit for node 68d47a11a5c2c05277bccf8e 2025-09-26 23:09:26,382 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 23:09:26,404 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d47a11a5c2c05277bccf8e HTTP/1.1" 200 None 2025-09-26 23:09:26,405 - lava_callback - DEBUG - Processing node 68d47a11a5c2c05277bccf8e: baseline-arm64-clabbe 2025-09-26 23:09:26,405 - lava_callback - ERROR - Error processing callback for node 68d47a11a5c2c05277bccf8e: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:48880 - "GET / HTTP/1.0" 200 OK 2025-09-26 23:09:27,024 - lava_callback - INFO - Received callback for node 68d47a20a5c2c05277bccf97 INFO: 172.21.0.1:48896 - "POST /node/68d47a20a5c2c05277bccf97 HTTP/1.0" 202 Accepted 2025-09-26 23:09:27,029 - lava_callback - INFO - Starting async job submit for node 68d47a20a5c2c05277bccf97 2025-09-26 23:09:27,030 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 23:09:27,042 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d47a20a5c2c05277bccf97 HTTP/1.1" 200 None 2025-09-26 23:09:27,043 - lava_callback - DEBUG - Processing node 68d47a20a5c2c05277bccf97: baseline-arm64-clabbe 2025-09-26 23:09:27,043 - lava_callback - ERROR - Error processing callback for node 68d47a20a5c2c05277bccf97: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:53460 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53466 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53468 - "GET / HTTP/1.0" 200 OK 2025-09-26 23:09:46,368 - lava_callback - INFO - Received callback for node 68d47a31a5c2c05277bccfa3 INFO: 172.21.0.1:54892 - "POST /node/68d47a31a5c2c05277bccfa3 HTTP/1.0" 202 Accepted 2025-09-26 23:09:46,373 - lava_callback - INFO - Starting async job submit for node 68d47a31a5c2c05277bccfa3 2025-09-26 23:09:46,374 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 23:09:46,389 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d47a31a5c2c05277bccfa3 HTTP/1.1" 200 None 2025-09-26 23:09:46,389 - lava_callback - DEBUG - Processing node 68d47a31a5c2c05277bccfa3: baseline-arm64-clabbe 2025-09-26 23:09:46,389 - lava_callback - ERROR - Error processing callback for node 68d47a31a5c2c05277bccfa3: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:54902 - "GET / HTTP/1.0" 200 OK 2025-09-26 23:09:47,026 - lava_callback - INFO - Received callback for node 68d47a2ea5c2c05277bccfa0 INFO: 172.21.0.1:54908 - "POST /node/68d47a2ea5c2c05277bccfa0 HTTP/1.0" 202 Accepted 2025-09-26 23:09:47,031 - lava_callback - INFO - Starting async job submit for node 68d47a2ea5c2c05277bccfa0 2025-09-26 23:09:47,032 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 23:09:47,044 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d47a2ea5c2c05277bccfa0 HTTP/1.1" 200 None 2025-09-26 23:09:47,044 - lava_callback - DEBUG - Processing node 68d47a2ea5c2c05277bccfa0: baseline-arm64-clabbe 2025-09-26 23:09:47,044 - lava_callback - ERROR - Error processing callback for node 68d47a2ea5c2c05277bccfa0: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:36636 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36460 - "GET / HTTP/1.0" 200 OK 2025-09-26 23:10:09,717 - lava_callback - INFO - Received callback for node 68d6e88b3c94e462c68822b0 INFO: 172.21.0.1:54818 - "POST /node/68d6e88b3c94e462c68822b0 HTTP/1.0" 202 Accepted 2025-09-26 23:10:09,723 - lava_callback - INFO - Starting async job submit for node 68d6e88b3c94e462c68822b0 2025-09-26 23:10:09,757 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 23:10:09,770 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6e88b3c94e462c68822b0 HTTP/1.1" 200 None 2025-09-26 23:10:09,771 - lava_callback - DEBUG - Processing node 68d6e88b3c94e462c68822b0: baseline-arm64-broonie 2025-09-26 23:10:10,226 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 23:10:10,366 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 23:10:10,367 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-68d6e88b3c94e462c68822b0/log.txt.gz 2025-09-26 23:10:10,377 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 23:10:10,532 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 23:10:10,533 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-68d6e88b3c94e462c68822b0/lava_callback.json.gz 2025-09-26 23:10:10,567 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 23:10:10,579 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6e88b3c94e462c68822b0 HTTP/1.1" 200 None 2025-09-26 23:10:10,580 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 23:10:10,592 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d5d90f2be2da466069ac22 HTTP/1.1" 200 None 2025-09-26 23:10:10,593 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 23:10:10,618 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6e88b3c94e462c68822b0 HTTP/1.1" 200 None 2025-09-26 23:10:10,620 - lava_callback - INFO - Completed processing callback for node 68d6e88b3c94e462c68822b0 INFO: 172.21.0.1:54824 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54838 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53690 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60154 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60168 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60170 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58708 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56150 - "GET / HTTP/1.0" 200 OK 2025-09-26 23:11:01,742 - lava_callback - INFO - Received callback for node 68d5213bb70fb2a1e20f1a90 INFO: 172.21.0.1:49312 - "POST /node/68d5213bb70fb2a1e20f1a90 HTTP/1.0" 202 Accepted 2025-09-26 23:11:01,749 - lava_callback - INFO - Starting async job submit for node 68d5213bb70fb2a1e20f1a90 2025-09-26 23:11:01,791 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 23:11:01,804 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d5213bb70fb2a1e20f1a90 HTTP/1.1" 200 None 2025-09-26 23:11:01,804 - lava_callback - DEBUG - Processing node 68d5213bb70fb2a1e20f1a90: baseline-arm 2025-09-26 23:11:02,033 - lava_callback - ERROR - Error processing callback for node 68d5213bb70fb2a1e20f1a90: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:49326 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42244 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42254 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43240 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40420 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40434 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40440 - "GET / HTTP/1.0" 200 OK 2025-09-26 23:11:45,169 - lava_callback - INFO - Received callback for node 68d493a4a5c2c05277bcee7d INFO: 172.21.0.1:34216 - "POST /node/68d493a4a5c2c05277bcee7d HTTP/1.0" 202 Accepted 2025-09-26 23:11:45,176 - lava_callback - INFO - Starting async job submit for node 68d493a4a5c2c05277bcee7d 2025-09-26 23:11:45,196 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 23:11:45,219 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d493a4a5c2c05277bcee7d HTTP/1.1" 200 None 2025-09-26 23:11:45,219 - lava_callback - DEBUG - Processing node 68d493a4a5c2c05277bcee7d: baseline-arm64 2025-09-26 23:11:45,554 - lava_callback - ERROR - Error processing callback for node 68d493a4a5c2c05277bcee7d: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:34232 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52034 - "GET / HTTP/1.0" 200 OK 2025-09-26 23:12:01,225 - lava_callback - INFO - Received callback for node 68d47aaea5c2c05277bcd178 INFO: 172.21.0.1:33852 - "POST /node/68d47aaea5c2c05277bcd178 HTTP/1.0" 202 Accepted 2025-09-26 23:12:01,231 - lava_callback - INFO - Starting async job submit for node 68d47aaea5c2c05277bcd178 2025-09-26 23:12:01,232 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 23:12:01,246 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d47aaea5c2c05277bcd178 HTTP/1.1" 200 None 2025-09-26 23:12:01,246 - lava_callback - DEBUG - Processing node 68d47aaea5c2c05277bcd178: baseline-arm64-baylibre 2025-09-26 23:12:01,246 - lava_callback - ERROR - Error processing callback for node 68d47aaea5c2c05277bcd178: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 23:12:05,260 - lava_callback - INFO - Received callback for node 68d6e97c3c94e462c68822ea INFO: 172.21.0.1:33864 - "POST /node/68d6e97c3c94e462c68822ea HTTP/1.0" 202 Accepted 2025-09-26 23:12:05,266 - lava_callback - INFO - Starting async job submit for node 68d6e97c3c94e462c68822ea 2025-09-26 23:12:05,288 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 23:12:05,304 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6e97c3c94e462c68822ea HTTP/1.1" 200 None 2025-09-26 23:12:05,305 - lava_callback - DEBUG - Processing node 68d6e97c3c94e462c68822ea: baseline-arm64-broonie 2025-09-26 23:12:05,632 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 23:12:05,803 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 23:12:05,803 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-68d6e97c3c94e462c68822ea/log.txt.gz 2025-09-26 23:12:05,811 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 23:12:05,945 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 23:12:05,946 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-68d6e97c3c94e462c68822ea/lava_callback.json.gz 2025-09-26 23:12:05,967 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 23:12:05,980 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6e97c3c94e462c68822ea HTTP/1.1" 200 None 2025-09-26 23:12:05,981 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 23:12:05,993 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d637475d233c363a761c21 HTTP/1.1" 200 None 2025-09-26 23:12:05,994 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 23:12:06,023 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6e97c3c94e462c68822ea HTTP/1.1" 200 None 2025-09-26 23:12:06,023 - lava_callback - INFO - Completed processing callback for node 68d6e97c3c94e462c68822ea INFO: 172.21.0.1:33874 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45956 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45958 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59140 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48254 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48258 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48270 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50128 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41412 - "GET / HTTP/1.0" 200 OK 2025-09-26 23:12:59,935 - lava_callback - INFO - Received callback for node 68d5213cb70fb2a1e20f1a93 INFO: 172.21.0.1:48546 - "POST /node/68d5213cb70fb2a1e20f1a93 HTTP/1.0" 202 Accepted 2025-09-26 23:12:59,941 - lava_callback - INFO - Starting async job submit for node 68d5213cb70fb2a1e20f1a93 2025-09-26 23:12:59,961 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 23:12:59,992 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d5213cb70fb2a1e20f1a93 HTTP/1.1" 200 None 2025-09-26 23:12:59,992 - lava_callback - DEBUG - Processing node 68d5213cb70fb2a1e20f1a93: baseline-arm 2025-09-26 23:13:00,221 - lava_callback - ERROR - Error processing callback for node 68d5213cb70fb2a1e20f1a93: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:48548 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43396 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43400 - "GET / HTTP/1.0" 200 OK 2025-09-26 23:13:25,291 - lava_callback - INFO - Received callback for node 68d493afa5c2c05277bceea2 INFO: 172.21.0.1:56772 - "POST /node/68d493afa5c2c05277bceea2 HTTP/1.0" 202 Accepted 2025-09-26 23:13:25,297 - lava_callback - INFO - Starting async job submit for node 68d493afa5c2c05277bceea2 2025-09-26 23:13:25,322 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 23:13:25,338 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d493afa5c2c05277bceea2 HTTP/1.1" 200 None 2025-09-26 23:13:25,339 - lava_callback - DEBUG - Processing node 68d493afa5c2c05277bceea2: baseline-arm64 2025-09-26 23:13:25,675 - lava_callback - ERROR - Error processing callback for node 68d493afa5c2c05277bceea2: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:56774 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60580 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60592 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60600 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41550 - "GET / HTTP/1.0" 200 OK 2025-09-26 23:13:46,574 - lava_callback - INFO - Received callback for node 68d47b15a5c2c05277bcd1ba INFO: 172.21.0.1:41562 - "POST /node/68d47b15a5c2c05277bcd1ba HTTP/1.0" 202 Accepted 2025-09-26 23:13:46,580 - lava_callback - INFO - Starting async job submit for node 68d47b15a5c2c05277bcd1ba 2025-09-26 23:13:46,581 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 23:13:46,598 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d47b15a5c2c05277bcd1ba HTTP/1.1" 200 None 2025-09-26 23:13:46,599 - lava_callback - DEBUG - Processing node 68d47b15a5c2c05277bcd1ba: baseline-arm64-clabbe 2025-09-26 23:13:46,599 - lava_callback - ERROR - Error processing callback for node 68d47b15a5c2c05277bcd1ba: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:34170 - "GET / HTTP/1.0" 200 OK 2025-09-26 23:14:01,287 - lava_callback - INFO - Received callback for node 68d47b2ea5c2c05277bcd1c6 INFO: 172.21.0.1:33846 - "POST /node/68d47b2ea5c2c05277bcd1c6 HTTP/1.0" 202 Accepted 2025-09-26 23:14:01,292 - lava_callback - INFO - Starting async job submit for node 68d47b2ea5c2c05277bcd1c6 2025-09-26 23:14:01,293 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 23:14:01,306 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d47b2ea5c2c05277bcd1c6 HTTP/1.1" 200 None 2025-09-26 23:14:01,307 - lava_callback - DEBUG - Processing node 68d47b2ea5c2c05277bcd1c6: baseline-arm64-baylibre 2025-09-26 23:14:01,307 - lava_callback - ERROR - Error processing callback for node 68d47b2ea5c2c05277bcd1c6: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:33862 - "GET / HTTP/1.0" 200 OK 2025-09-26 23:14:06,537 - lava_callback - INFO - Received callback for node 68d47b29a5c2c05277bcd1c3 INFO: 172.21.0.1:33868 - "POST /node/68d47b29a5c2c05277bcd1c3 HTTP/1.0" 202 Accepted 2025-09-26 23:14:06,542 - lava_callback - INFO - Starting async job submit for node 68d47b29a5c2c05277bcd1c3 2025-09-26 23:14:06,543 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 23:14:06,584 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d47b29a5c2c05277bcd1c3 HTTP/1.1" 200 None 2025-09-26 23:14:06,585 - lava_callback - DEBUG - Processing node 68d47b29a5c2c05277bcd1c3: baseline-arm64-clabbe 2025-09-26 23:14:06,585 - lava_callback - ERROR - Error processing callback for node 68d47b29a5c2c05277bcd1c3: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 23:14:07,209 - lava_callback - INFO - Received callback for node 68d47b34a5c2c05277bcd1c9 INFO: 172.21.0.1:33876 - "POST /node/68d47b34a5c2c05277bcd1c9 HTTP/1.0" 202 Accepted 2025-09-26 23:14:07,214 - lava_callback - INFO - Starting async job submit for node 68d47b34a5c2c05277bcd1c9 2025-09-26 23:14:07,215 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 23:14:07,227 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d47b34a5c2c05277bcd1c9 HTTP/1.1" 200 None 2025-09-26 23:14:07,228 - lava_callback - DEBUG - Processing node 68d47b34a5c2c05277bcd1c9: baseline-arm64-clabbe 2025-09-26 23:14:07,228 - lava_callback - ERROR - Error processing callback for node 68d47b34a5c2c05277bcd1c9: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-26 23:14:07,865 - lava_callback - INFO - Received callback for node 68d47b39a5c2c05277bcd1cc INFO: 172.21.0.1:33880 - "POST /node/68d47b39a5c2c05277bcd1cc HTTP/1.0" 202 Accepted 2025-09-26 23:14:07,870 - lava_callback - INFO - Starting async job submit for node 68d47b39a5c2c05277bcd1cc 2025-09-26 23:14:07,871 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 23:14:07,900 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d47b39a5c2c05277bcd1cc HTTP/1.1" 200 None 2025-09-26 23:14:07,901 - lava_callback - DEBUG - Processing node 68d47b39a5c2c05277bcd1cc: baseline-arm64-clabbe 2025-09-26 23:14:07,901 - lava_callback - ERROR - Error processing callback for node 68d47b39a5c2c05277bcd1cc: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:35794 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35806 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55130 - "GET / HTTP/1.0" 200 OK 2025-09-26 23:14:26,554 - lava_callback - INFO - Received callback for node 68d47b3ea5c2c05277bcd1cf INFO: 172.21.0.1:55132 - "POST /node/68d47b3ea5c2c05277bcd1cf HTTP/1.0" 202 Accepted 2025-09-26 23:14:26,560 - lava_callback - INFO - Starting async job submit for node 68d47b3ea5c2c05277bcd1cf 2025-09-26 23:14:26,560 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 23:14:26,597 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d47b3ea5c2c05277bcd1cf HTTP/1.1" 200 None 2025-09-26 23:14:26,597 - lava_callback - DEBUG - Processing node 68d47b3ea5c2c05277bcd1cf: baseline-arm64-clabbe 2025-09-26 23:14:26,597 - lava_callback - ERROR - Error processing callback for node 68d47b3ea5c2c05277bcd1cf: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:54470 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54478 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54486 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33648 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54782 - "GET / HTTP/1.0" 200 OK 2025-09-26 23:15:00,059 - lava_callback - INFO - Received callback for node 68d5213db70fb2a1e20f1a96 INFO: 172.21.0.1:42094 - "POST /node/68d5213db70fb2a1e20f1a96 HTTP/1.0" 202 Accepted 2025-09-26 23:15:00,065 - lava_callback - INFO - Starting async job submit for node 68d5213db70fb2a1e20f1a96 2025-09-26 23:15:00,086 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 23:15:00,098 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d5213db70fb2a1e20f1a96 HTTP/1.1" 200 None 2025-09-26 23:15:00,098 - lava_callback - DEBUG - Processing node 68d5213db70fb2a1e20f1a96: baseline-arm 2025-09-26 23:15:00,345 - lava_callback - ERROR - Error processing callback for node 68d5213db70fb2a1e20f1a96: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None 2025-09-26 23:15:01,355 - lava_callback - INFO - Received callback for node 68d47b68a5c2c05277bcd1e7 INFO: 172.21.0.1:42100 - "POST /node/68d47b68a5c2c05277bcd1e7 HTTP/1.0" 202 Accepted 2025-09-26 23:15:01,361 - lava_callback - INFO - Starting async job submit for node 68d47b68a5c2c05277bcd1e7 2025-09-26 23:15:01,362 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 23:15:01,375 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d47b68a5c2c05277bcd1e7 HTTP/1.1" 200 None 2025-09-26 23:15:01,375 - lava_callback - DEBUG - Processing node 68d47b68a5c2c05277bcd1e7: baseline-arm64-baylibre 2025-09-26 23:15:01,375 - lava_callback - ERROR - Error processing callback for node 68d47b68a5c2c05277bcd1e7: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:42116 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56224 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56236 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40098 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51210 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51214 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51224 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43954 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39890 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48382 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52842 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52854 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51812 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33992 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33996 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34010 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47808 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41612 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44440 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42794 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42804 - "GET / HTTP/1.0" 200 OK 2025-09-26 23:17:19,594 - lava_callback - INFO - Received callback for node 68d5213eb70fb2a1e20f1a99 INFO: 172.21.0.1:42294 - "POST /node/68d5213eb70fb2a1e20f1a99 HTTP/1.0" 202 Accepted 2025-09-26 23:17:19,599 - lava_callback - INFO - Starting async job submit for node 68d5213eb70fb2a1e20f1a99 2025-09-26 23:17:19,618 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 23:17:19,630 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d5213eb70fb2a1e20f1a99 HTTP/1.1" 200 None 2025-09-26 23:17:19,631 - lava_callback - DEBUG - Processing node 68d5213eb70fb2a1e20f1a99: baseline-arm 2025-09-26 23:17:19,859 - lava_callback - ERROR - Error processing callback for node 68d5213eb70fb2a1e20f1a99: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:42308 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53598 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53610 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53626 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48954 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39292 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57654 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33970 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33982 - "GET / HTTP/1.0" 200 OK 2025-09-26 23:18:25,358 - lava_callback - INFO - Received callback for node 68d47868a5c2c05277bccc68 INFO: 172.21.0.1:49516 - "POST /node/68d47868a5c2c05277bccc68 HTTP/1.0" 202 Accepted 2025-09-26 23:18:25,371 - lava_callback - INFO - Starting async job submit for node 68d47868a5c2c05277bccc68 2025-09-26 23:18:25,718 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 23:18:25,731 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d47868a5c2c05277bccc68 HTTP/1.1" 200 None 2025-09-26 23:18:25,731 - lava_callback - DEBUG - Processing node 68d47868a5c2c05277bccc68: ltp-controllers INFO: 172.21.0.1:49532 - "GET / HTTP/1.0" 200 OK 2025-09-26 23:18:27,338 - lava_callback - INFO - Received callback for node 68d6ebb33c94e462c68823c1 INFO: 172.21.0.1:49548 - "POST /node/68d6ebb33c94e462c68823c1 HTTP/1.0" 202 Accepted 2025-09-26 23:18:27,360 - lava_callback - INFO - Starting async job submit for node 68d6ebb33c94e462c68823c1 2025-09-26 23:18:27,505 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 23:18:27,591 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6ebb33c94e462c68823c1 HTTP/1.1" 200 None 2025-09-26 23:18:27,601 - lava_callback - DEBUG - Processing node 68d6ebb33c94e462c68823c1: baseline-arm64-broonie 2025-09-26 23:18:28,368 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 23:18:28,609 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 23:18:28,666 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-68d6ebb33c94e462c68823c1/log.txt.gz 2025-09-26 23:18:28,744 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 23:18:28,968 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 23:18:28,978 - lava_callback - ERROR - Error processing callback for node 68d47868a5c2c05277bccc68: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None 2025-09-26 23:18:28,980 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-68d6ebb33c94e462c68823c1/lava_callback.json.gz 2025-09-26 23:18:29,002 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 23:18:29,022 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6ebb33c94e462c68823c1 HTTP/1.1" 200 None 2025-09-26 23:18:29,023 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 23:18:29,035 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d5dc8530c58dac7d9410cb HTTP/1.1" 200 None 2025-09-26 23:18:29,037 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 23:18:29,063 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6ebb33c94e462c68823c1 HTTP/1.1" 200 None 2025-09-26 23:18:29,064 - lava_callback - INFO - Completed processing callback for node 68d6ebb33c94e462c68823c1 INFO: 172.21.0.1:56890 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56900 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56916 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46466 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39550 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60298 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39592 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39594 - "GET / HTTP/1.0" 200 OK 2025-09-26 23:19:18,950 - lava_callback - INFO - Received callback for node 68d5215ab70fb2a1e20f1ae6 INFO: 172.21.0.1:39602 - "POST /node/68d5215ab70fb2a1e20f1ae6 HTTP/1.0" 202 Accepted 2025-09-26 23:19:18,956 - lava_callback - INFO - Starting async job submit for node 68d5215ab70fb2a1e20f1ae6 2025-09-26 23:19:18,977 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 23:19:18,994 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d5215ab70fb2a1e20f1ae6 HTTP/1.1" 200 None 2025-09-26 23:19:18,995 - lava_callback - DEBUG - Processing node 68d5215ab70fb2a1e20f1ae6: baseline-arm 2025-09-26 23:19:19,247 - lava_callback - ERROR - Error processing callback for node 68d5215ab70fb2a1e20f1ae6: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:49164 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43264 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43280 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43288 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39944 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54824 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57674 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41664 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41676 - "GET / HTTP/1.0" 200 OK 2025-09-26 23:20:25,610 - lava_callback - INFO - Received callback for node 68d6ed823c94e462c68823db INFO: 172.21.0.1:51792 - "POST /node/68d6ed823c94e462c68823db HTTP/1.0" 202 Accepted 2025-09-26 23:20:25,616 - lava_callback - INFO - Starting async job submit for node 68d6ed823c94e462c68823db 2025-09-26 23:20:25,637 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 23:20:25,675 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6ed823c94e462c68823db HTTP/1.1" 200 None 2025-09-26 23:20:25,675 - lava_callback - DEBUG - Processing node 68d6ed823c94e462c68823db: baseline-arm64-broonie 2025-09-26 23:20:25,967 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 23:20:26,147 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 23:20:26,147 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-68d6ed823c94e462c68823db/log.txt.gz 2025-09-26 23:20:26,155 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 23:20:26,314 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 23:20:26,315 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-68d6ed823c94e462c68823db/lava_callback.json.gz 2025-09-26 23:20:26,336 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 23:20:26,349 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6ed823c94e462c68823db HTTP/1.1" 200 None 2025-09-26 23:20:26,350 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 23:20:26,362 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d637475d233c363a761c21 HTTP/1.1" 200 None 2025-09-26 23:20:26,363 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 23:20:26,394 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6ed823c94e462c68823db HTTP/1.1" 200 None 2025-09-26 23:20:26,395 - lava_callback - INFO - Completed processing callback for node 68d6ed823c94e462c68823db INFO: 172.21.0.1:51800 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50498 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50500 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50514 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38808 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58536 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54902 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45386 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45390 - "GET / HTTP/1.0" 200 OK 2025-09-26 23:21:20,484 - lava_callback - INFO - Received callback for node 68d5215bb70fb2a1e20f1ae9 INFO: 172.21.0.1:40630 - "POST /node/68d5215bb70fb2a1e20f1ae9 HTTP/1.0" 202 Accepted 2025-09-26 23:21:20,490 - lava_callback - INFO - Starting async job submit for node 68d5215bb70fb2a1e20f1ae9 2025-09-26 23:21:20,516 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 23:21:20,529 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d5215bb70fb2a1e20f1ae9 HTTP/1.1" 200 None 2025-09-26 23:21:20,529 - lava_callback - DEBUG - Processing node 68d5215bb70fb2a1e20f1ae9: baseline-arm 2025-09-26 23:21:20,754 - lava_callback - ERROR - Error processing callback for node 68d5215bb70fb2a1e20f1ae9: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:40638 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48522 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48532 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48546 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39418 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35386 - "GET / HTTP/1.0" 200 OK 2025-09-26 23:22:04,942 - lava_callback - INFO - Received callback for node 68d521ecb70fb2a1e20f1c34 INFO: 172.21.0.1:60524 - "POST /node/68d521ecb70fb2a1e20f1c34 HTTP/1.0" 202 Accepted 2025-09-26 23:22:04,948 - lava_callback - INFO - Starting async job submit for node 68d521ecb70fb2a1e20f1c34 2025-09-26 23:22:04,963 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 23:22:04,975 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d521ecb70fb2a1e20f1c34 HTTP/1.1" 200 None 2025-09-26 23:22:04,976 - lava_callback - DEBUG - Processing node 68d521ecb70fb2a1e20f1c34: baseline-arm64-broonie 2025-09-26 23:22:05,252 - lava_callback - ERROR - Error processing callback for node 68d521ecb70fb2a1e20f1c34: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:60540 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45306 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45308 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59920 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41016 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41022 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41026 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39974 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46844 - "GET / HTTP/1.0" 200 OK 2025-09-26 23:23:04,998 - lava_callback - INFO - Received callback for node 68d4786aa5c2c05277bccc6b INFO: 172.21.0.1:33868 - "POST /node/68d4786aa5c2c05277bccc6b HTTP/1.0" 202 Accepted 2025-09-26 23:23:05,006 - lava_callback - INFO - Starting async job submit for node 68d4786aa5c2c05277bccc6b 2025-09-26 23:23:05,112 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 23:23:05,125 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4786aa5c2c05277bccc6b HTTP/1.1" 200 None 2025-09-26 23:23:05,125 - lava_callback - DEBUG - Processing node 68d4786aa5c2c05277bccc6b: ltp-cve 2025-09-26 23:23:05,873 - lava_callback - ERROR - Error processing callback for node 68d4786aa5c2c05277bccc6b: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:33876 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37046 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37060 - "GET / HTTP/1.0" 200 OK 2025-09-26 23:23:22,662 - lava_callback - INFO - Received callback for node 68d5215db70fb2a1e20f1aec INFO: 172.21.0.1:39316 - "POST /node/68d5215db70fb2a1e20f1aec HTTP/1.0" 202 Accepted 2025-09-26 23:23:22,668 - lava_callback - INFO - Starting async job submit for node 68d5215db70fb2a1e20f1aec 2025-09-26 23:23:22,689 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 23:23:22,701 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d5215db70fb2a1e20f1aec HTTP/1.1" 200 None 2025-09-26 23:23:22,702 - lava_callback - DEBUG - Processing node 68d5215db70fb2a1e20f1aec: baseline-arm 2025-09-26 23:23:22,930 - lava_callback - ERROR - Error processing callback for node 68d5215db70fb2a1e20f1aec: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:39324 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40496 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40498 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40502 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43530 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47092 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52324 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41944 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41948 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34628 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33418 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33434 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33440 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40214 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60482 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35662 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36012 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36020 - "GET / HTTP/1.0" 200 OK 2025-09-26 23:25:19,833 - lava_callback - INFO - Received callback for node 68d5215db70fb2a1e20f1aef INFO: 172.21.0.1:54234 - "POST /node/68d5215db70fb2a1e20f1aef HTTP/1.0" 202 Accepted 2025-09-26 23:25:19,840 - lava_callback - INFO - Starting async job submit for node 68d5215db70fb2a1e20f1aef 2025-09-26 23:25:19,858 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 23:25:19,870 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d5215db70fb2a1e20f1aef HTTP/1.1" 200 None 2025-09-26 23:25:19,871 - lava_callback - DEBUG - Processing node 68d5215db70fb2a1e20f1aef: baseline-arm 2025-09-26 23:25:20,094 - lava_callback - ERROR - Error processing callback for node 68d5215db70fb2a1e20f1aef: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:54248 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36320 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36324 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36326 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40530 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54682 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37918 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42304 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42320 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52468 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52926 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52928 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52934 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45616 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40650 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60372 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33116 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33122 - "GET / HTTP/1.0" 200 OK 2025-09-26 23:27:20,260 - lava_callback - INFO - Received callback for node 68d5215fb70fb2a1e20f1af2 INFO: 172.21.0.1:35108 - "POST /node/68d5215fb70fb2a1e20f1af2 HTTP/1.0" 202 Accepted 2025-09-26 23:27:20,265 - lava_callback - INFO - Starting async job submit for node 68d5215fb70fb2a1e20f1af2 2025-09-26 23:27:20,284 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 23:27:20,301 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d5215fb70fb2a1e20f1af2 HTTP/1.1" 200 None 2025-09-26 23:27:20,301 - lava_callback - DEBUG - Processing node 68d5215fb70fb2a1e20f1af2: baseline-arm 2025-09-26 23:27:20,545 - lava_callback - ERROR - Error processing callback for node 68d5215fb70fb2a1e20f1af2: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:35116 - "GET / HTTP/1.0" 200 OK 2025-09-26 23:27:26,647 - lava_callback - INFO - Received callback for node 68d6fcfc3c94e462c6882529 INFO: 172.21.0.1:35128 - "POST /node/68d6fcfc3c94e462c6882529 HTTP/1.0" 202 Accepted 2025-09-26 23:27:26,653 - lava_callback - INFO - Starting async job submit for node 68d6fcfc3c94e462c6882529 2025-09-26 23:27:26,674 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 23:27:26,687 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6fcfc3c94e462c6882529 HTTP/1.1" 200 None 2025-09-26 23:27:26,687 - lava_callback - DEBUG - Processing node 68d6fcfc3c94e462c6882529: baseline-arm64-broonie 2025-09-26 23:27:27,011 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 23:27:27,152 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 23:27:27,153 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-68d6fcfc3c94e462c6882529/log.txt.gz 2025-09-26 23:27:27,160 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 23:27:27,315 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 23:27:27,316 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-68d6fcfc3c94e462c6882529/lava_callback.json.gz 2025-09-26 23:27:27,338 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 23:27:27,351 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6fcfc3c94e462c6882529 HTTP/1.1" 200 None 2025-09-26 23:27:27,352 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 23:27:27,363 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d64d12a5c3ab168ecc7a3d HTTP/1.1" 200 None 2025-09-26 23:27:27,364 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 23:27:27,393 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6fcfc3c94e462c6882529 HTTP/1.1" 200 None 2025-09-26 23:27:27,393 - lava_callback - INFO - Completed processing callback for node 68d6fcfc3c94e462c6882529 INFO: 172.21.0.1:59356 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59362 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59370 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57610 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40088 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36394 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54734 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54740 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44582 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56974 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56978 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56990 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52704 - "GET / HTTP/1.0" 200 OK 2025-09-26 23:28:47,472 - lava_callback - INFO - Received callback for node 68d48948a5c2c05277bcdf74 INFO: 172.21.0.1:52714 - "POST /node/68d48948a5c2c05277bcdf74 HTTP/1.0" 202 Accepted 2025-09-26 23:28:47,480 - lava_callback - INFO - Starting async job submit for node 68d48948a5c2c05277bcdf74 2025-09-26 23:28:47,520 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 23:28:47,557 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d48948a5c2c05277bcdf74 HTTP/1.1" 200 None 2025-09-26 23:28:47,557 - lava_callback - DEBUG - Processing node 68d48948a5c2c05277bcdf74: baseline-arm64 2025-09-26 23:28:48,384 - lava_callback - ERROR - Error processing callback for node 68d48948a5c2c05277bcdf74: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:56834 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43666 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54470 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54484 - "GET / HTTP/1.0" 200 OK 2025-09-26 23:29:19,002 - lava_callback - INFO - Received callback for node 68d5215fb70fb2a1e20f1af5 INFO: 172.21.0.1:54492 - "POST /node/68d5215fb70fb2a1e20f1af5 HTTP/1.0" 202 Accepted 2025-09-26 23:29:19,008 - lava_callback - INFO - Starting async job submit for node 68d5215fb70fb2a1e20f1af5 2025-09-26 23:29:19,028 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 23:29:19,062 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d5215fb70fb2a1e20f1af5 HTTP/1.1" 200 None 2025-09-26 23:29:19,062 - lava_callback - DEBUG - Processing node 68d5215fb70fb2a1e20f1af5: baseline-arm 2025-09-26 23:29:19,288 - lava_callback - ERROR - Error processing callback for node 68d5215fb70fb2a1e20f1af5: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:34554 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47748 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47754 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47768 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59074 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52420 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54038 - "GET / HTTP/1.0" 200 OK 2025-09-26 23:30:08,120 - lava_callback - INFO - Received callback for node 68d4894ba5c2c05277bcdf7b INFO: 172.21.0.1:54042 - "POST /node/68d4894ba5c2c05277bcdf7b HTTP/1.0" 202 Accepted 2025-09-26 23:30:08,128 - lava_callback - INFO - Starting async job submit for node 68d4894ba5c2c05277bcdf7b 2025-09-26 23:30:08,168 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 23:30:08,209 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4894ba5c2c05277bcdf7b HTTP/1.1" 200 None 2025-09-26 23:30:08,209 - lava_callback - DEBUG - Processing node 68d4894ba5c2c05277bcdf7b: baseline-arm64 2025-09-26 23:30:09,052 - lava_callback - ERROR - Error processing callback for node 68d4894ba5c2c05277bcdf7b: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:56958 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56970 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44916 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45320 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45322 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45338 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47324 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52148 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59056 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38462 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38472 - "GET / HTTP/1.0" 200 OK 2025-09-26 23:31:20,199 - lava_callback - INFO - Received callback for node 68d52162b70fb2a1e20f1af8 INFO: 172.21.0.1:34390 - "POST /node/68d52162b70fb2a1e20f1af8 HTTP/1.0" 202 Accepted 2025-09-26 23:31:20,205 - lava_callback - INFO - Starting async job submit for node 68d52162b70fb2a1e20f1af8 2025-09-26 23:31:20,226 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 23:31:20,244 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d52162b70fb2a1e20f1af8 HTTP/1.1" 200 None 2025-09-26 23:31:20,244 - lava_callback - DEBUG - Processing node 68d52162b70fb2a1e20f1af8: baseline-arm 2025-09-26 23:31:20,488 - lava_callback - ERROR - Error processing callback for node 68d52162b70fb2a1e20f1af8: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:34400 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58888 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58904 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46836 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46842 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51782 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55652 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55232 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55248 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34408 - "GET / HTTP/1.0" 200 OK 2025-09-26 23:32:26,534 - lava_callback - INFO - Received callback for node 68d6fdcc3c94e462c688253f INFO: 172.21.0.1:34410 - "POST /node/68d6fdcc3c94e462c688253f HTTP/1.0" 202 Accepted 2025-09-26 23:32:26,541 - lava_callback - INFO - Starting async job submit for node 68d6fdcc3c94e462c688253f 2025-09-26 23:32:26,575 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 23:32:26,587 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6fdcc3c94e462c688253f HTTP/1.1" 200 None 2025-09-26 23:32:26,587 - lava_callback - DEBUG - Processing node 68d6fdcc3c94e462c688253f: baseline-arm64-broonie 2025-09-26 23:32:27,035 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 23:32:27,206 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 23:32:27,207 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-68d6fdcc3c94e462c688253f/log.txt.gz 2025-09-26 23:32:27,217 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 23:32:27,352 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 23:32:27,353 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-68d6fdcc3c94e462c688253f/lava_callback.json.gz 2025-09-26 23:32:27,387 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 23:32:27,403 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6fdcc3c94e462c688253f HTTP/1.1" 200 None 2025-09-26 23:32:27,403 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 23:32:27,415 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d64d12a5c3ab168ecc7a3d HTTP/1.1" 200 None 2025-09-26 23:32:27,415 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 23:32:27,441 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d6fdcc3c94e462c688253f HTTP/1.1" 200 None 2025-09-26 23:32:27,441 - lava_callback - INFO - Completed processing callback for node 68d6fdcc3c94e462c688253f INFO: 172.21.0.1:35140 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35146 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47848 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47858 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44436 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48870 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35218 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35222 - "GET / HTTP/1.0" 200 OK 2025-09-26 23:33:20,280 - lava_callback - INFO - Received callback for node 68d52163b70fb2a1e20f1afb INFO: 172.21.0.1:47094 - "POST /node/68d52163b70fb2a1e20f1afb HTTP/1.0" 202 Accepted 2025-09-26 23:33:20,286 - lava_callback - INFO - Starting async job submit for node 68d52163b70fb2a1e20f1afb 2025-09-26 23:33:20,306 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 23:33:20,318 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d52163b70fb2a1e20f1afb HTTP/1.1" 200 None 2025-09-26 23:33:20,319 - lava_callback - DEBUG - Processing node 68d52163b70fb2a1e20f1afb: baseline-arm 2025-09-26 23:33:20,567 - lava_callback - ERROR - Error processing callback for node 68d52163b70fb2a1e20f1afb: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:47106 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40250 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40252 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55398 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55402 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46948 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38008 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40444 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40460 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37986 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47112 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47118 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60012 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60026 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60466 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46156 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46276 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46284 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56808 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51178 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51192 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52690 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52704 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42298 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55118 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60170 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60178 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48316 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34164 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34168 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52126 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52136 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34570 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41684 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58510 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58520 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48116 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50406 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50412 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40382 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40390 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60282 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39416 - "GET / HTTP/1.0" 200 OK 2025-09-26 23:38:10,234 - lava_callback - INFO - Received callback for node 68d486e7a5c2c05277bcdd59 INFO: 172.21.0.1:36534 - "POST /node/68d486e7a5c2c05277bcdd59 HTTP/1.0" 202 Accepted 2025-09-26 23:38:10,240 - lava_callback - INFO - Starting async job submit for node 68d486e7a5c2c05277bcdd59 2025-09-26 23:38:10,266 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 23:38:10,305 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d486e7a5c2c05277bcdd59 HTTP/1.1" 200 None 2025-09-26 23:38:10,306 - lava_callback - DEBUG - Processing node 68d486e7a5c2c05277bcdd59: baseline-arm-broonie 2025-09-26 23:38:10,492 - lava_callback - ERROR - Error processing callback for node 68d486e7a5c2c05277bcdd59: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:36542 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36554 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34298 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60292 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60302 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52320 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52328 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40034 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54122 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44862 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44868 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58536 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57852 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57864 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52742 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52754 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51546 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43952 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56500 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56504 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37200 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43656 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43658 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49372 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49382 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33852 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50068 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39170 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39176 - "GET / HTTP/1.0" 200 OK 2025-09-26 23:41:23,085 - lava_callback - INFO - Received callback for node 68d48675a5c2c05277bcd758 INFO: 172.21.0.1:50384 - "POST /node/68d48675a5c2c05277bcd758 HTTP/1.0" 202 Accepted 2025-09-26 23:41:23,091 - lava_callback - INFO - Starting async job submit for node 68d48675a5c2c05277bcd758 2025-09-26 23:41:23,122 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 23:41:23,140 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d48675a5c2c05277bcd758 HTTP/1.1" 200 None 2025-09-26 23:41:23,141 - lava_callback - DEBUG - Processing node 68d48675a5c2c05277bcd758: baseline-arm 2025-09-26 23:41:23,398 - lava_callback - ERROR - Error processing callback for node 68d48675a5c2c05277bcd758: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:50396 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35672 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35688 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37138 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37148 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44872 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41552 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41730 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41738 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41480 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47338 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47342 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33220 - "GET / HTTP/1.0" 200 OK 2025-09-26 23:42:45,599 - lava_callback - INFO - Received callback for node 68d4873da5c2c05277bcddd2 INFO: 172.21.0.1:33222 - "POST /node/68d4873da5c2c05277bcddd2 HTTP/1.0" 202 Accepted 2025-09-26 23:42:45,606 - lava_callback - INFO - Starting async job submit for node 68d4873da5c2c05277bcddd2 2025-09-26 23:42:45,632 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 23:42:45,650 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4873da5c2c05277bcddd2 HTTP/1.1" 200 None 2025-09-26 23:42:45,651 - lava_callback - DEBUG - Processing node 68d4873da5c2c05277bcddd2: ltp-cap-bounds 2025-09-26 23:42:46,001 - lava_callback - ERROR - Error processing callback for node 68d4873da5c2c05277bcddd2: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:33230 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43332 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39456 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44374 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44382 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35374 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40192 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40196 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34274 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34290 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45484 - "GET / HTTP/1.0" 200 OK 2025-09-26 23:44:03,199 - lava_callback - INFO - Received callback for node 68d534c2b70fb2a1e20f20ac INFO: 172.21.0.1:39342 - "POST /node/68d534c2b70fb2a1e20f20ac HTTP/1.0" 202 Accepted 2025-09-26 23:44:03,204 - lava_callback - INFO - Starting async job submit for node 68d534c2b70fb2a1e20f20ac 2025-09-26 23:44:03,225 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 23:44:03,237 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d534c2b70fb2a1e20f20ac HTTP/1.1" 200 None 2025-09-26 23:44:03,237 - lava_callback - DEBUG - Processing node 68d534c2b70fb2a1e20f20ac: baseline-arm 2025-09-26 23:44:03,460 - lava_callback - ERROR - Error processing callback for node 68d534c2b70fb2a1e20f20ac: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:39356 - "GET / HTTP/1.0" 200 OK 2025-09-26 23:44:11,403 - lava_callback - INFO - Received callback for node 68d7084a3c94e462c688263f INFO: 172.21.0.1:54922 - "POST /node/68d7084a3c94e462c688263f HTTP/1.0" 202 Accepted 2025-09-26 23:44:11,409 - lava_callback - INFO - Starting async job submit for node 68d7084a3c94e462c688263f 2025-09-26 23:44:11,443 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 23:44:11,478 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d7084a3c94e462c688263f HTTP/1.1" 200 None 2025-09-26 23:44:11,478 - lava_callback - DEBUG - Processing node 68d7084a3c94e462c688263f: baseline-arm64-broonie 2025-09-26 23:44:11,953 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 23:44:12,090 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 23:44:12,091 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-68d7084a3c94e462c688263f/log.txt.gz 2025-09-26 23:44:12,101 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 23:44:12,245 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 23:44:12,246 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-68d7084a3c94e462c688263f/lava_callback.json.gz 2025-09-26 23:44:12,280 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 23:44:12,311 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d7084a3c94e462c688263f HTTP/1.1" 200 None 2025-09-26 23:44:12,312 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 23:44:12,328 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6b9dd2f83206171a92b48 HTTP/1.1" 200 None 2025-09-26 23:44:12,329 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 23:44:12,353 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d7084a3c94e462c688263f HTTP/1.1" 200 None 2025-09-26 23:44:12,353 - lava_callback - INFO - Completed processing callback for node 68d7084a3c94e462c688263f INFO: 172.21.0.1:54934 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54940 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41316 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38786 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38790 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60908 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60916 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46262 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47790 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48930 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48940 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35678 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49412 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49422 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34216 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34220 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53026 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54182 - "GET / HTTP/1.0" 200 OK 2025-09-26 23:46:08,121 - lava_callback - INFO - Received callback for node 68d48741a5c2c05277bcddd5 INFO: 172.21.0.1:54190 - "POST /node/68d48741a5c2c05277bcddd5 HTTP/1.0" 202 Accepted 2025-09-26 23:46:08,129 - lava_callback - INFO - Starting async job submit for node 68d48741a5c2c05277bcddd5 2025-09-26 23:46:08,229 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 23:46:08,242 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d48741a5c2c05277bcddd5 HTTP/1.1" 200 None 2025-09-26 23:46:08,243 - lava_callback - DEBUG - Processing node 68d48741a5c2c05277bcddd5: ltp-containers 2025-09-26 23:46:09,024 - lava_callback - ERROR - Error processing callback for node 68d48741a5c2c05277bcddd5: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:40116 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40130 - "GET / HTTP/1.0" 200 OK 2025-09-26 23:46:21,110 - lava_callback - INFO - Received callback for node 68d534c3b70fb2a1e20f20af INFO: 172.21.0.1:50366 - "POST /node/68d534c3b70fb2a1e20f20af HTTP/1.0" 202 Accepted 2025-09-26 23:46:21,116 - lava_callback - INFO - Starting async job submit for node 68d534c3b70fb2a1e20f20af 2025-09-26 23:46:21,136 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 23:46:21,149 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d534c3b70fb2a1e20f20af HTTP/1.1" 200 None 2025-09-26 23:46:21,149 - lava_callback - DEBUG - Processing node 68d534c3b70fb2a1e20f20af: baseline-arm 2025-09-26 23:46:21,379 - lava_callback - ERROR - Error processing callback for node 68d534c3b70fb2a1e20f20af: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:50382 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39430 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39432 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51132 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51140 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51736 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59504 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57144 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57152 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46636 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49582 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49596 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34818 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34826 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44908 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57668 - "GET / HTTP/1.0" 200 OK 2025-09-26 23:48:07,044 - lava_callback - INFO - Received callback for node 68d47882a5c2c05277bccc86 INFO: 172.21.0.1:57676 - "POST /node/68d47882a5c2c05277bccc86 HTTP/1.0" 202 Accepted 2025-09-26 23:48:07,061 - lava_callback - INFO - Starting async job submit for node 68d47882a5c2c05277bccc86 2025-09-26 23:48:08,400 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 23:48:08,428 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d47882a5c2c05277bccc86 HTTP/1.1" 200 None 2025-09-26 23:48:08,429 - lava_callback - DEBUG - Processing node 68d47882a5c2c05277bccc86: ltp-syscalls INFO: 172.21.0.1:41162 - "GET /api/metrics HTTP/1.0" 200 OK 2025-09-26 23:48:15,282 - lava_callback - ERROR - Error processing callback for node 68d47882a5c2c05277bccc86: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:41168 - "GET / HTTP/1.0" 200 OK 2025-09-26 23:48:21,021 - lava_callback - INFO - Received callback for node 68d534c4b70fb2a1e20f20b2 INFO: 172.21.0.1:40536 - "POST /node/68d534c4b70fb2a1e20f20b2 HTTP/1.0" 202 Accepted 2025-09-26 23:48:21,027 - lava_callback - INFO - Starting async job submit for node 68d534c4b70fb2a1e20f20b2 2025-09-26 23:48:21,047 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 23:48:21,067 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d534c4b70fb2a1e20f20b2 HTTP/1.1" 200 None 2025-09-26 23:48:21,068 - lava_callback - DEBUG - Processing node 68d534c4b70fb2a1e20f20b2: baseline-arm 2025-09-26 23:48:21,295 - lava_callback - ERROR - Error processing callback for node 68d534c4b70fb2a1e20f20b2: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:40538 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48120 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48130 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57468 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57482 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34228 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49392 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51416 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51424 - "GET / HTTP/1.0" 200 OK 2025-09-26 23:49:26,376 - lava_callback - INFO - Received callback for node 68d71aa53c94e462c6882810 INFO: 172.21.0.1:38800 - "POST /node/68d71aa53c94e462c6882810 HTTP/1.0" 202 Accepted 2025-09-26 23:49:26,382 - lava_callback - INFO - Starting async job submit for node 68d71aa53c94e462c6882810 2025-09-26 23:49:26,416 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 23:49:26,435 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d71aa53c94e462c6882810 HTTP/1.1" 200 None 2025-09-26 23:49:26,435 - lava_callback - DEBUG - Processing node 68d71aa53c94e462c6882810: baseline-arm64-broonie INFO: 172.21.0.1:38814 - "GET / HTTP/1.0" 200 OK 2025-09-26 23:49:26,883 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 23:49:27,033 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 23:49:27,034 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-68d71aa53c94e462c6882810/log.txt.gz 2025-09-26 23:49:27,044 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 23:49:27,179 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 23:49:27,180 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-68d71aa53c94e462c6882810/lava_callback.json.gz 2025-09-26 23:49:27,215 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 23:49:27,229 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d71aa53c94e462c6882810 HTTP/1.1" 200 None 2025-09-26 23:49:27,231 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 23:49:27,245 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6bd8e3c94e462c68814f1 HTTP/1.1" 200 None 2025-09-26 23:49:27,247 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 23:49:27,268 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d71aa53c94e462c6882810 HTTP/1.1" 200 None 2025-09-26 23:49:27,269 - lava_callback - INFO - Completed processing callback for node 68d71aa53c94e462c6882810 INFO: 172.21.0.1:55206 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55216 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34610 - "GET / HTTP/1.0" 200 OK 2025-09-26 23:49:45,752 - lava_callback - INFO - Received callback for node 68d486f4a5c2c05277bcdd77 INFO: 172.21.0.1:34614 - "POST /node/68d486f4a5c2c05277bcdd77 HTTP/1.0" 202 Accepted 2025-09-26 23:49:45,759 - lava_callback - INFO - Starting async job submit for node 68d486f4a5c2c05277bcdd77 2025-09-26 23:49:45,781 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 23:49:45,794 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d486f4a5c2c05277bcdd77 HTTP/1.1" 200 None 2025-09-26 23:49:45,794 - lava_callback - DEBUG - Processing node 68d486f4a5c2c05277bcdd77: baseline-arm-broonie 2025-09-26 23:49:46,016 - lava_callback - ERROR - Error processing callback for node 68d486f4a5c2c05277bcdd77: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:34618 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41438 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36844 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42746 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42750 - "GET / HTTP/1.0" 200 OK 2025-09-26 23:50:20,788 - lava_callback - INFO - Received callback for node 68d534c5b70fb2a1e20f20b5 INFO: 172.21.0.1:39776 - "POST /node/68d534c5b70fb2a1e20f20b5 HTTP/1.0" 202 Accepted 2025-09-26 23:50:20,794 - lava_callback - INFO - Starting async job submit for node 68d534c5b70fb2a1e20f20b5 2025-09-26 23:50:20,815 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 23:50:20,846 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d534c5b70fb2a1e20f20b5 HTTP/1.1" 200 None 2025-09-26 23:50:20,846 - lava_callback - DEBUG - Processing node 68d534c5b70fb2a1e20f20b5: baseline-arm 2025-09-26 23:50:21,073 - lava_callback - ERROR - Error processing callback for node 68d534c5b70fb2a1e20f20b5: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:39784 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51032 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51036 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38768 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38770 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60882 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47988 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39292 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39302 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57086 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44756 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44758 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33180 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33186 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58868 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35882 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42728 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42730 - "GET / HTTP/1.0" 200 OK 2025-09-26 23:52:17,821 - lava_callback - INFO - Received callback for node 68d53504b70fb2a1e20f20e5 INFO: 172.21.0.1:42744 - "POST /node/68d53504b70fb2a1e20f20e5 HTTP/1.0" 202 Accepted 2025-09-26 23:52:17,827 - lava_callback - INFO - Starting async job submit for node 68d53504b70fb2a1e20f20e5 2025-09-26 23:52:17,847 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 23:52:17,860 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d53504b70fb2a1e20f20e5 HTTP/1.1" 200 None 2025-09-26 23:52:17,860 - lava_callback - DEBUG - Processing node 68d53504b70fb2a1e20f20e5: baseline-arm 2025-09-26 23:52:18,089 - lava_callback - ERROR - Error processing callback for node 68d53504b70fb2a1e20f20e5: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:36668 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40074 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40078 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40270 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40276 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48310 - "GET / HTTP/1.0" 200 OK 2025-09-26 23:53:06,320 - lava_callback - INFO - Received callback for node 68d485faa5c2c05277bcd71e INFO: 172.21.0.1:43322 - "POST /node/68d485faa5c2c05277bcd71e HTTP/1.0" 202 Accepted 2025-09-26 23:53:06,328 - lava_callback - INFO - Starting async job submit for node 68d485faa5c2c05277bcd71e 2025-09-26 23:53:06,356 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 23:53:06,382 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d485faa5c2c05277bcd71e HTTP/1.1" 200 None 2025-09-26 23:53:06,382 - lava_callback - DEBUG - Processing node 68d485faa5c2c05277bcd71e: ltp-syscalls-ipc INFO: 172.21.0.1:43324 - "GET / HTTP/1.0" 200 OK 2025-09-26 23:53:07,262 - lava_callback - ERROR - Error processing callback for node 68d485faa5c2c05277bcd71e: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:32806 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:32818 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42884 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46894 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46910 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39950 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39958 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52992 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33628 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40578 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40582 - "GET / HTTP/1.0" 200 OK 2025-09-26 23:54:20,118 - lava_callback - INFO - Received callback for node 68d53505b70fb2a1e20f20e8 INFO: 172.21.0.1:33546 - "POST /node/68d53505b70fb2a1e20f20e8 HTTP/1.0" 202 Accepted 2025-09-26 23:54:20,125 - lava_callback - INFO - Starting async job submit for node 68d53505b70fb2a1e20f20e8 2025-09-26 23:54:20,146 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 23:54:20,158 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d53505b70fb2a1e20f20e8 HTTP/1.1" 200 None 2025-09-26 23:54:20,158 - lava_callback - DEBUG - Processing node 68d53505b70fb2a1e20f20e8: baseline-arm 2025-09-26 23:54:20,387 - lava_callback - ERROR - Error processing callback for node 68d53505b70fb2a1e20f20e8: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None 2025-09-26 23:54:26,197 - lava_callback - INFO - Received callback for node 68d71bd33c94e462c6882814 INFO: 172.21.0.1:33552 - "POST /node/68d71bd33c94e462c6882814 HTTP/1.0" 202 Accepted 2025-09-26 23:54:26,203 - lava_callback - INFO - Starting async job submit for node 68d71bd33c94e462c6882814 2025-09-26 23:54:26,258 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 23:54:26,272 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d71bd33c94e462c6882814 HTTP/1.1" 200 None 2025-09-26 23:54:26,273 - lava_callback - DEBUG - Processing node 68d71bd33c94e462c6882814: baseline-arm64-broonie INFO: 172.21.0.1:33566 - "GET / HTTP/1.0" 200 OK 2025-09-26 23:54:26,715 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 23:54:26,861 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 23:54:26,862 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-68d71bd33c94e462c6882814/log.txt.gz 2025-09-26 23:54:26,872 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 23:54:27,006 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 23:54:27,006 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-68d71bd33c94e462c6882814/lava_callback.json.gz 2025-09-26 23:54:27,041 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 23:54:27,053 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d71bd33c94e462c6882814 HTTP/1.1" 200 None 2025-09-26 23:54:27,054 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 23:54:27,066 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d62c44c1bc82a30b37a916 HTTP/1.1" 200 None 2025-09-26 23:54:27,067 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 23:54:27,091 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d71bd33c94e462c6882814 HTTP/1.1" 200 None 2025-09-26 23:54:27,091 - lava_callback - INFO - Completed processing callback for node 68d71bd33c94e462c6882814 INFO: 172.21.0.1:47050 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47064 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48646 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48656 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34504 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60240 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57292 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57306 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60424 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49390 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49392 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35366 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35378 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52058 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44848 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52860 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52876 - "GET / HTTP/1.0" 200 OK 2025-09-26 23:56:20,509 - lava_callback - INFO - Received callback for node 68d53506b70fb2a1e20f20eb INFO: 172.21.0.1:57610 - "POST /node/68d53506b70fb2a1e20f20eb HTTP/1.0" 202 Accepted 2025-09-26 23:56:20,515 - lava_callback - INFO - Starting async job submit for node 68d53506b70fb2a1e20f20eb 2025-09-26 23:56:20,534 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 23:56:20,547 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d53506b70fb2a1e20f20eb HTTP/1.1" 200 None 2025-09-26 23:56:20,547 - lava_callback - DEBUG - Processing node 68d53506b70fb2a1e20f20eb: baseline-arm 2025-09-26 23:56:20,767 - lava_callback - ERROR - Error processing callback for node 68d53506b70fb2a1e20f20eb: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:57612 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49450 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49460 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48096 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48098 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56024 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48546 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49356 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49368 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45464 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44780 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44786 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32928 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32930 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54106 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52412 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41498 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41508 - "GET / HTTP/1.0" 200 OK 2025-09-26 23:58:22,240 - lava_callback - INFO - Received callback for node 68d53507b70fb2a1e20f20ee INFO: 172.21.0.1:57964 - "POST /node/68d53507b70fb2a1e20f20ee HTTP/1.0" 202 Accepted 2025-09-26 23:58:22,245 - lava_callback - INFO - Starting async job submit for node 68d53507b70fb2a1e20f20ee 2025-09-26 23:58:22,291 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 23:58:22,325 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d53507b70fb2a1e20f20ee HTTP/1.1" 200 None 2025-09-26 23:58:22,325 - lava_callback - DEBUG - Processing node 68d53507b70fb2a1e20f20ee: baseline-arm 2025-09-26 23:58:22,552 - lava_callback - ERROR - Error processing callback for node 68d53507b70fb2a1e20f20ee: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:57968 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50500 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50514 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56104 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56112 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45312 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45344 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34920 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34922 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54686 - "GET / HTTP/1.0" 200 OK 2025-09-26 23:59:26,698 - lava_callback - INFO - Received callback for node 68d48745a5c2c05277bcddd8 INFO: 172.21.0.1:54694 - "POST /node/68d48745a5c2c05277bcddd8 HTTP/1.0" 202 Accepted 2025-09-26 23:59:26,709 - lava_callback - INFO - Starting async job submit for node 68d48745a5c2c05277bcddd8 2025-09-26 23:59:27,045 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 23:59:27,076 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d48745a5c2c05277bcddd8 HTTP/1.1" 200 None 2025-09-26 23:59:27,077 - lava_callback - DEBUG - Processing node 68d48745a5c2c05277bcddd8: ltp-controllers 2025-09-26 23:59:29,929 - lava_callback - ERROR - Error processing callback for node 68d48745a5c2c05277bcddd8: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:55644 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55660 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47408 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47412 - "GET / HTTP/1.0" 200 OK 2025-09-26 23:59:49,318 - lava_callback - INFO - Received callback for node 68d7228c3c94e462c6882844 INFO: 172.21.0.1:41740 - "POST /node/68d7228c3c94e462c6882844 HTTP/1.0" 202 Accepted 2025-09-26 23:59:49,324 - lava_callback - INFO - Starting async job submit for node 68d7228c3c94e462c6882844 2025-09-26 23:59:49,359 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 23:59:49,376 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d7228c3c94e462c6882844 HTTP/1.1" 200 None 2025-09-26 23:59:49,377 - lava_callback - DEBUG - Processing node 68d7228c3c94e462c6882844: baseline-arm64-broonie 2025-09-26 23:59:49,814 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 23:59:49,936 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 23:59:49,936 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-68d7228c3c94e462c6882844/log.txt.gz 2025-09-26 23:59:49,947 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-26 23:59:50,088 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-26 23:59:50,089 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-68d7228c3c94e462c6882844/lava_callback.json.gz 2025-09-26 23:59:50,123 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 23:59:50,136 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d7228c3c94e462c6882844 HTTP/1.1" 200 None 2025-09-26 23:59:50,137 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 23:59:50,148 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d64d12a5c3ab168ecc7a3d HTTP/1.1" 200 None 2025-09-26 23:59:50,149 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-26 23:59:50,177 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d7228c3c94e462c6882844 HTTP/1.1" 200 None 2025-09-26 23:59:50,178 - lava_callback - INFO - Completed processing callback for node 68d7228c3c94e462c6882844 INFO: 172.21.0.1:41750 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56096 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53408 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53422 - "GET / HTTP/1.0" 200 OK 2025-09-27 00:00:20,341 - lava_callback - INFO - Received callback for node 68d5357db70fb2a1e20f2190 INFO: 172.21.0.1:46918 - "POST /node/68d5357db70fb2a1e20f2190 HTTP/1.0" 202 Accepted 2025-09-27 00:00:20,352 - lava_callback - INFO - Starting async job submit for node 68d5357db70fb2a1e20f2190 2025-09-27 00:00:20,384 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-27 00:00:20,400 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d5357db70fb2a1e20f2190 HTTP/1.1" 200 None 2025-09-27 00:00:20,401 - lava_callback - DEBUG - Processing node 68d5357db70fb2a1e20f2190: baseline-arm 2025-09-27 00:00:20,722 - lava_callback - ERROR - Error processing callback for node 68d5357db70fb2a1e20f2190: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:46934 - "GET / HTTP/1.0" 200 OK 2025-09-27 00:00:26,701 - lava_callback - INFO - Received callback for node 68d48751a5c2c05277bcdde1 INFO: 172.21.0.1:46946 - "POST /node/68d48751a5c2c05277bcdde1 HTTP/1.0" 202 Accepted 2025-09-27 00:00:26,710 - lava_callback - INFO - Starting async job submit for node 68d48751a5c2c05277bcdde1 2025-09-27 00:00:26,924 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-27 00:00:26,941 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d48751a5c2c05277bcdde1 HTTP/1.1" 200 None 2025-09-27 00:00:26,941 - lava_callback - DEBUG - Processing node 68d48751a5c2c05277bcdde1: ltp-fs-bind 2025-09-27 00:00:27,613 - lava_callback - ERROR - Error processing callback for node 68d48751a5c2c05277bcdde1: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:39398 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39408 - "GET / HTTP/1.0" 200 OK 2025-09-27 00:00:44,368 - lava_callback - INFO - Received callback for node 68d48678a5c2c05277bcd76d INFO: 172.21.0.1:56272 - "POST /node/68d48678a5c2c05277bcd76d HTTP/1.0" 202 Accepted 2025-09-27 00:00:44,378 - lava_callback - INFO - Starting async job submit for node 68d48678a5c2c05277bcd76d 2025-09-27 00:00:44,428 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-27 00:00:44,451 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d48678a5c2c05277bcd76d HTTP/1.1" 200 None 2025-09-27 00:00:44,452 - lava_callback - DEBUG - Processing node 68d48678a5c2c05277bcd76d: baseline-arm 2025-09-27 00:00:44,949 - lava_callback - ERROR - Error processing callback for node 68d48678a5c2c05277bcd76d: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:56274 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56286 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41208 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38816 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35368 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35384 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45344 - "GET / HTTP/1.0" 200 OK 2025-09-27 00:01:27,182 - lava_callback - INFO - Received callback for node 68d7254c3c94e462c6882848 INFO: 172.21.0.1:45346 - "POST /node/68d7254c3c94e462c6882848 HTTP/1.0" 202 Accepted 2025-09-27 00:01:27,190 - lava_callback - INFO - Starting async job submit for node 68d7254c3c94e462c6882848 2025-09-27 00:01:27,221 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-27 00:01:27,239 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d7254c3c94e462c6882848 HTTP/1.1" 200 None 2025-09-27 00:01:27,239 - lava_callback - DEBUG - Processing node 68d7254c3c94e462c6882848: baseline-arm64-broonie 2025-09-27 00:01:27,799 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-27 00:01:27,960 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-27 00:01:27,961 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-68d7254c3c94e462c6882848/log.txt.gz 2025-09-27 00:01:27,970 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-27 00:01:28,116 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-27 00:01:28,117 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-68d7254c3c94e462c6882848/lava_callback.json.gz 2025-09-27 00:01:28,144 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-27 00:01:28,159 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d7254c3c94e462c6882848 HTTP/1.1" 200 None 2025-09-27 00:01:28,160 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-27 00:01:28,173 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6b9dd2f83206171a92b48 HTTP/1.1" 200 None 2025-09-27 00:01:28,174 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-27 00:01:28,213 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d7254c3c94e462c6882848 HTTP/1.1" 200 None 2025-09-27 00:01:28,214 - lava_callback - INFO - Completed processing callback for node 68d7254c3c94e462c6882848 INFO: 172.21.0.1:54846 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54862 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60168 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60184 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35532 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33882 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38242 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38258 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35106 - "GET / HTTP/1.0" 200 OK 2025-09-27 00:02:26,670 - lava_callback - INFO - Received callback for node 68d4867fa5c2c05277bcd773 INFO: 172.21.0.1:35114 - "POST /node/68d4867fa5c2c05277bcd773 HTTP/1.0" 202 Accepted 2025-09-27 00:02:26,680 - lava_callback - INFO - Starting async job submit for node 68d4867fa5c2c05277bcd773 2025-09-27 00:02:26,681 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-27 00:02:26,697 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4867fa5c2c05277bcd773 HTTP/1.1" 200 None 2025-09-27 00:02:26,698 - lava_callback - DEBUG - Processing node 68d4867fa5c2c05277bcd773: baseline-arm 2025-09-27 00:02:26,698 - lava_callback - ERROR - Error processing callback for node 68d4867fa5c2c05277bcd773: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:38788 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38792 - "GET / HTTP/1.0" 200 OK 2025-09-27 00:02:39,608 - lava_callback - INFO - Received callback for node 68d5357db70fb2a1e20f2193 INFO: 172.21.0.1:43586 - "POST /node/68d5357db70fb2a1e20f2193 HTTP/1.0" 202 Accepted 2025-09-27 00:02:39,617 - lava_callback - INFO - Starting async job submit for node 68d5357db70fb2a1e20f2193 2025-09-27 00:02:39,656 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-27 00:02:39,689 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d5357db70fb2a1e20f2193 HTTP/1.1" 200 None 2025-09-27 00:02:39,690 - lava_callback - DEBUG - Processing node 68d5357db70fb2a1e20f2193: baseline-arm 2025-09-27 00:02:39,996 - lava_callback - ERROR - Error processing callback for node 68d5357db70fb2a1e20f2193: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None 2025-09-27 00:02:46,341 - lava_callback - INFO - Received callback for node 68d486a4a5c2c05277bcdcfe INFO: 172.21.0.1:43594 - "POST /node/68d486a4a5c2c05277bcdcfe HTTP/1.0" 202 Accepted 2025-09-27 00:02:46,349 - lava_callback - INFO - Starting async job submit for node 68d486a4a5c2c05277bcdcfe 2025-09-27 00:02:46,349 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-27 00:02:46,364 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d486a4a5c2c05277bcdcfe HTTP/1.1" 200 None 2025-09-27 00:02:46,364 - lava_callback - DEBUG - Processing node 68d486a4a5c2c05277bcdcfe: baseline-arm 2025-09-27 00:02:46,365 - lava_callback - ERROR - Error processing callback for node 68d486a4a5c2c05277bcdcfe: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:43604 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43610 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33788 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49816 - "GET / HTTP/1.0" 200 OK 2025-09-27 00:03:07,354 - lava_callback - INFO - Received callback for node 68d486a5a5c2c05277bcdd01 INFO: 172.21.0.1:49820 - "POST /node/68d486a5a5c2c05277bcdd01 HTTP/1.0" 202 Accepted 2025-09-27 00:03:07,361 - lava_callback - INFO - Starting async job submit for node 68d486a5a5c2c05277bcdd01 2025-09-27 00:03:07,362 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-27 00:03:07,403 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d486a5a5c2c05277bcdd01 HTTP/1.1" 200 None 2025-09-27 00:03:07,403 - lava_callback - DEBUG - Processing node 68d486a5a5c2c05277bcdd01: baseline-arm 2025-09-27 00:03:07,403 - lava_callback - ERROR - Error processing callback for node 68d486a5a5c2c05277bcdd01: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-27 00:03:08,027 - lava_callback - INFO - Received callback for node 68d486aca5c2c05277bcdd04 INFO: 172.21.0.1:49822 - "POST /node/68d486aca5c2c05277bcdd04 HTTP/1.0" 202 Accepted 2025-09-27 00:03:08,034 - lava_callback - INFO - Starting async job submit for node 68d486aca5c2c05277bcdd04 2025-09-27 00:03:08,035 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-27 00:03:08,058 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d486aca5c2c05277bcdd04 HTTP/1.1" 200 None 2025-09-27 00:03:08,058 - lava_callback - DEBUG - Processing node 68d486aca5c2c05277bcdd04: baseline-arm 2025-09-27 00:03:08,058 - lava_callback - ERROR - Error processing callback for node 68d486aca5c2c05277bcdd04: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-27 00:03:08,737 - lava_callback - INFO - Received callback for node 68d486aca5c2c05277bcdd07 INFO: 172.21.0.1:49828 - "POST /node/68d486aca5c2c05277bcdd07 HTTP/1.0" 202 Accepted 2025-09-27 00:03:08,744 - lava_callback - INFO - Starting async job submit for node 68d486aca5c2c05277bcdd07 2025-09-27 00:03:08,746 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-27 00:03:08,764 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d486aca5c2c05277bcdd07 HTTP/1.1" 200 None 2025-09-27 00:03:08,764 - lava_callback - DEBUG - Processing node 68d486aca5c2c05277bcdd07: baseline-arm 2025-09-27 00:03:08,764 - lava_callback - ERROR - Error processing callback for node 68d486aca5c2c05277bcdd07: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-27 00:03:09,420 - lava_callback - INFO - Received callback for node 68d486b0a5c2c05277bcdd0a INFO: 172.21.0.1:50548 - "POST /node/68d486b0a5c2c05277bcdd0a HTTP/1.0" 202 Accepted 2025-09-27 00:03:09,426 - lava_callback - INFO - Starting async job submit for node 68d486b0a5c2c05277bcdd0a 2025-09-27 00:03:09,427 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-27 00:03:09,443 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d486b0a5c2c05277bcdd0a HTTP/1.1" 200 None 2025-09-27 00:03:09,443 - lava_callback - DEBUG - Processing node 68d486b0a5c2c05277bcdd0a: baseline-arm 2025-09-27 00:03:09,444 - lava_callback - ERROR - Error processing callback for node 68d486b0a5c2c05277bcdd0a: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-27 00:03:10,095 - lava_callback - INFO - Received callback for node 68d486b2a5c2c05277bcdd0d INFO: 172.21.0.1:50564 - "POST /node/68d486b2a5c2c05277bcdd0d HTTP/1.0" 202 Accepted 2025-09-27 00:03:10,102 - lava_callback - INFO - Starting async job submit for node 68d486b2a5c2c05277bcdd0d 2025-09-27 00:03:10,103 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-27 00:03:10,117 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d486b2a5c2c05277bcdd0d HTTP/1.1" 200 None 2025-09-27 00:03:10,118 - lava_callback - DEBUG - Processing node 68d486b2a5c2c05277bcdd0d: baseline-arm 2025-09-27 00:03:10,118 - lava_callback - ERROR - Error processing callback for node 68d486b2a5c2c05277bcdd0d: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-27 00:03:10,777 - lava_callback - INFO - Received callback for node 68d486b7a5c2c05277bcdd28 INFO: 172.21.0.1:50578 - "POST /node/68d486b7a5c2c05277bcdd28 HTTP/1.0" 202 Accepted 2025-09-27 00:03:10,784 - lava_callback - INFO - Starting async job submit for node 68d486b7a5c2c05277bcdd28 2025-09-27 00:03:10,785 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-27 00:03:10,799 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d486b7a5c2c05277bcdd28 HTTP/1.1" 200 None 2025-09-27 00:03:10,799 - lava_callback - DEBUG - Processing node 68d486b7a5c2c05277bcdd28: baseline-arm 2025-09-27 00:03:10,799 - lava_callback - ERROR - Error processing callback for node 68d486b7a5c2c05277bcdd28: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-27 00:03:11,379 - lava_callback - INFO - Received callback for node 68d726873c94e462c688284c INFO: 172.21.0.1:50592 - "POST /node/68d726873c94e462c688284c HTTP/1.0" 202 Accepted 2025-09-27 00:03:11,391 - lava_callback - INFO - Starting async job submit for node 68d726873c94e462c688284c 2025-09-27 00:03:11,421 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-27 00:03:11,458 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d726873c94e462c688284c HTTP/1.1" 200 None 2025-09-27 00:03:11,458 - lava_callback - DEBUG - Processing node 68d726873c94e462c688284c: baseline-arm64-broonie 2025-09-27 00:03:11,881 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-27 00:03:12,079 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-27 00:03:12,079 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-68d726873c94e462c688284c/log.txt.gz 2025-09-27 00:03:12,088 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-27 00:03:12,273 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-27 00:03:12,274 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-68d726873c94e462c688284c/lava_callback.json.gz 2025-09-27 00:03:12,306 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-27 00:03:12,322 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d726873c94e462c688284c HTTP/1.1" 200 None 2025-09-27 00:03:12,323 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-27 00:03:12,337 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6bd8e3c94e462c68814f1 HTTP/1.1" 200 None 2025-09-27 00:03:12,338 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-27 00:03:12,371 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d726873c94e462c688284c HTTP/1.1" 200 None 2025-09-27 00:03:12,372 - lava_callback - INFO - Completed processing callback for node 68d726873c94e462c688284c INFO: 172.21.0.1:50600 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50612 - "GET / HTTP/1.0" 200 OK 2025-09-27 00:03:26,317 - lava_callback - INFO - Received callback for node 68d486baa5c2c05277bcdd2b INFO: 172.21.0.1:51814 - "POST /node/68d486baa5c2c05277bcdd2b HTTP/1.0" 202 Accepted 2025-09-27 00:03:26,326 - lava_callback - INFO - Starting async job submit for node 68d486baa5c2c05277bcdd2b 2025-09-27 00:03:26,327 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-27 00:03:26,345 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d486baa5c2c05277bcdd2b HTTP/1.1" 200 None 2025-09-27 00:03:26,345 - lava_callback - DEBUG - Processing node 68d486baa5c2c05277bcdd2b: baseline-arm 2025-09-27 00:03:26,345 - lava_callback - ERROR - Error processing callback for node 68d486baa5c2c05277bcdd2b: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:51818 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37844 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37858 - "GET / HTTP/1.0" 200 OK 2025-09-27 00:03:46,425 - lava_callback - INFO - Received callback for node 68d48756a5c2c05277bcdde7 INFO: 172.21.0.1:51202 - "POST /node/68d48756a5c2c05277bcdde7 HTTP/1.0" 202 Accepted 2025-09-27 00:03:46,438 - lava_callback - INFO - Starting async job submit for node 68d48756a5c2c05277bcdde7 2025-09-27 00:03:46,478 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-27 00:03:46,495 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d48756a5c2c05277bcdde7 HTTP/1.1" 200 None 2025-09-27 00:03:46,496 - lava_callback - DEBUG - Processing node 68d48756a5c2c05277bcdde7: ltp-fsx INFO: 172.21.0.1:51214 - "GET / HTTP/1.0" 200 OK 2025-09-27 00:03:47,024 - lava_callback - ERROR - Error processing callback for node 68d48756a5c2c05277bcdde7: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:51228 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33030 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33540 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49236 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49246 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39820 - "GET / HTTP/1.0" 200 OK 2025-09-27 00:04:28,752 - lava_callback - INFO - Received callback for node 68d48703a5c2c05277bcdd87 INFO: 172.21.0.1:39824 - "POST /node/68d48703a5c2c05277bcdd87 HTTP/1.0" 202 Accepted 2025-09-27 00:04:28,763 - lava_callback - INFO - Starting async job submit for node 68d48703a5c2c05277bcdd87 2025-09-27 00:04:28,764 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-27 00:04:28,785 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d48703a5c2c05277bcdd87 HTTP/1.1" 200 None 2025-09-27 00:04:28,786 - lava_callback - DEBUG - Processing node 68d48703a5c2c05277bcdd87: baseline-arm-clabbe 2025-09-27 00:04:28,786 - lava_callback - ERROR - Error processing callback for node 68d48703a5c2c05277bcdd87: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:56878 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56880 - "GET / HTTP/1.0" 200 OK 2025-09-27 00:04:42,462 - lava_callback - INFO - Received callback for node 68d5357fb70fb2a1e20f2196 INFO: 172.21.0.1:45292 - "POST /node/68d5357fb70fb2a1e20f2196 HTTP/1.0" 202 Accepted 2025-09-27 00:04:42,469 - lava_callback - INFO - Starting async job submit for node 68d5357fb70fb2a1e20f2196 2025-09-27 00:04:42,489 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-27 00:04:42,502 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d5357fb70fb2a1e20f2196 HTTP/1.1" 200 None 2025-09-27 00:04:42,502 - lava_callback - DEBUG - Processing node 68d5357fb70fb2a1e20f2196: baseline-arm 2025-09-27 00:04:42,727 - lava_callback - ERROR - Error processing callback for node 68d5357fb70fb2a1e20f2196: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None 2025-09-27 00:04:46,447 - lava_callback - INFO - Received callback for node 68d48759a5c2c05277bcddea INFO: 172.21.0.1:45298 - "POST /node/68d48759a5c2c05277bcddea HTTP/1.0" 202 Accepted 2025-09-27 00:04:46,454 - lava_callback - INFO - Starting async job submit for node 68d48759a5c2c05277bcddea 2025-09-27 00:04:46,486 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-27 00:04:46,499 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d48759a5c2c05277bcddea HTTP/1.1" 200 None 2025-09-27 00:04:46,500 - lava_callback - DEBUG - Processing node 68d48759a5c2c05277bcddea: ltp-input INFO: 172.21.0.1:45300 - "GET / HTTP/1.0" 200 OK 2025-09-27 00:04:46,961 - lava_callback - ERROR - Error processing callback for node 68d48759a5c2c05277bcddea: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:45304 - "GET / HTTP/1.0" 200 OK 2025-09-27 00:04:48,677 - lava_callback - INFO - Received callback for node 68d4870ca5c2c05277bcdd8d INFO: 172.21.0.1:45308 - "POST /node/68d4870ca5c2c05277bcdd8d HTTP/1.0" 202 Accepted 2025-09-27 00:04:48,683 - lava_callback - INFO - Starting async job submit for node 68d4870ca5c2c05277bcdd8d 2025-09-27 00:04:48,683 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-27 00:04:48,718 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4870ca5c2c05277bcdd8d HTTP/1.1" 200 None 2025-09-27 00:04:48,719 - lava_callback - DEBUG - Processing node 68d4870ca5c2c05277bcdd8d: baseline-arm-clabbe 2025-09-27 00:04:48,719 - lava_callback - ERROR - Error processing callback for node 68d4870ca5c2c05277bcdd8d: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-27 00:04:49,340 - lava_callback - INFO - Received callback for node 68d4870fa5c2c05277bcdd90 INFO: 172.21.0.1:45916 - "POST /node/68d4870fa5c2c05277bcdd90 HTTP/1.0" 202 Accepted 2025-09-27 00:04:49,346 - lava_callback - INFO - Starting async job submit for node 68d4870fa5c2c05277bcdd90 2025-09-27 00:04:49,347 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-27 00:04:49,360 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4870fa5c2c05277bcdd90 HTTP/1.1" 200 None 2025-09-27 00:04:49,360 - lava_callback - DEBUG - Processing node 68d4870fa5c2c05277bcdd90: baseline-arm-clabbe 2025-09-27 00:04:49,360 - lava_callback - ERROR - Error processing callback for node 68d4870fa5c2c05277bcdd90: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:45926 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42804 - "GET / HTTP/1.0" 200 OK 2025-09-27 00:05:08,728 - lava_callback - INFO - Received callback for node 68d48727a5c2c05277bcddac INFO: 172.21.0.1:42820 - "POST /node/68d48727a5c2c05277bcddac HTTP/1.0" 202 Accepted 2025-09-27 00:05:08,734 - lava_callback - INFO - Starting async job submit for node 68d48727a5c2c05277bcddac 2025-09-27 00:05:08,734 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-27 00:05:08,747 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d48727a5c2c05277bcddac HTTP/1.1" 200 None 2025-09-27 00:05:08,747 - lava_callback - DEBUG - Processing node 68d48727a5c2c05277bcddac: baseline-arm-clabbe 2025-09-27 00:05:08,747 - lava_callback - ERROR - Error processing callback for node 68d48727a5c2c05277bcddac: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:38576 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38590 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39226 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56992 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57004 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37636 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37648 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32966 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47324 - "GET / HTTP/1.0" 200 OK 2025-09-27 00:06:11,117 - lava_callback - INFO - Received callback for node 68d48765a5c2c05277bcddf3 INFO: 172.21.0.1:43472 - "POST /node/68d48765a5c2c05277bcddf3 HTTP/1.0" 202 Accepted 2025-09-27 00:06:11,124 - lava_callback - INFO - Starting async job submit for node 68d48765a5c2c05277bcddf3 2025-09-27 00:06:11,124 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-27 00:06:11,145 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d48765a5c2c05277bcddf3 HTTP/1.1" 200 None 2025-09-27 00:06:11,145 - lava_callback - DEBUG - Processing node 68d48765a5c2c05277bcddf3: ltp-sched 2025-09-27 00:06:11,146 - lava_callback - ERROR - Error processing callback for node 68d48765a5c2c05277bcddf3: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-27 00:06:11,922 - lava_callback - INFO - Received callback for node 68d4876aa5c2c05277bcddf6 INFO: 172.21.0.1:43482 - "POST /node/68d4876aa5c2c05277bcddf6 HTTP/1.0" 202 Accepted 2025-09-27 00:06:11,932 - lava_callback - INFO - Starting async job submit for node 68d4876aa5c2c05277bcddf6 2025-09-27 00:06:11,934 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-27 00:06:11,973 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4876aa5c2c05277bcddf6 HTTP/1.1" 200 None 2025-09-27 00:06:11,974 - lava_callback - DEBUG - Processing node 68d4876aa5c2c05277bcddf6: ltp-smoketest 2025-09-27 00:06:11,974 - lava_callback - ERROR - Error processing callback for node 68d4876aa5c2c05277bcddf6: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:43494 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43506 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43680 - "GET / HTTP/1.0" 200 OK 2025-09-27 00:06:31,328 - lava_callback - INFO - Received callback for node 68d48772a5c2c05277bcde14 INFO: 172.21.0.1:50186 - "POST /node/68d48772a5c2c05277bcde14 HTTP/1.0" 202 Accepted 2025-09-27 00:06:31,334 - lava_callback - INFO - Starting async job submit for node 68d48772a5c2c05277bcde14 2025-09-27 00:06:31,335 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-27 00:06:31,348 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d48772a5c2c05277bcde14 HTTP/1.1" 200 None 2025-09-27 00:06:31,349 - lava_callback - DEBUG - Processing node 68d48772a5c2c05277bcde14: ltp-syscalls-ipc 2025-09-27 00:06:31,349 - lava_callback - ERROR - Error processing callback for node 68d48772a5c2c05277bcde14: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-27 00:06:32,128 - lava_callback - INFO - Received callback for node 68d48777a5c2c05277bcde17 INFO: 172.21.0.1:50198 - "POST /node/68d48777a5c2c05277bcde17 HTTP/1.0" 202 Accepted 2025-09-27 00:06:32,134 - lava_callback - INFO - Starting async job submit for node 68d48777a5c2c05277bcde17 2025-09-27 00:06:32,135 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-27 00:06:32,147 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d48777a5c2c05277bcde17 HTTP/1.1" 200 None 2025-09-27 00:06:32,148 - lava_callback - DEBUG - Processing node 68d48777a5c2c05277bcde17: ltp-timers 2025-09-27 00:06:32,148 - lava_callback - ERROR - Error processing callback for node 68d48777a5c2c05277bcde17: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-27 00:06:32,852 - lava_callback - INFO - Received callback for node 68d48779a5c2c05277bcde1a INFO: 172.21.0.1:50214 - "POST /node/68d48779a5c2c05277bcde1a HTTP/1.0" 202 Accepted 2025-09-27 00:06:32,864 - lava_callback - INFO - Starting async job submit for node 68d48779a5c2c05277bcde1a 2025-09-27 00:06:32,865 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-27 00:06:32,886 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d48779a5c2c05277bcde1a HTTP/1.1" 200 None 2025-09-27 00:06:32,887 - lava_callback - DEBUG - Processing node 68d48779a5c2c05277bcde1a: ltp-watchqueue 2025-09-27 00:06:32,887 - lava_callback - ERROR - Error processing callback for node 68d48779a5c2c05277bcde1a: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:50226 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50228 - "GET / HTTP/1.0" 200 OK 2025-09-27 00:06:40,922 - lava_callback - INFO - Received callback for node 68d5357fb70fb2a1e20f2199 INFO: 172.21.0.1:58656 - "POST /node/68d5357fb70fb2a1e20f2199 HTTP/1.0" 202 Accepted 2025-09-27 00:06:40,928 - lava_callback - INFO - Starting async job submit for node 68d5357fb70fb2a1e20f2199 2025-09-27 00:06:40,948 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-27 00:06:40,983 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d5357fb70fb2a1e20f2199 HTTP/1.1" 200 None 2025-09-27 00:06:40,984 - lava_callback - DEBUG - Processing node 68d5357fb70fb2a1e20f2199: baseline-arm 2025-09-27 00:06:41,209 - lava_callback - ERROR - Error processing callback for node 68d5357fb70fb2a1e20f2199: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:58664 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58680 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57430 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43206 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45358 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45366 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47332 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52484 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52488 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37194 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37204 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52404 - "GET / HTTP/1.0" 200 OK 2025-09-27 00:08:06,479 - lava_callback - INFO - Received callback for node 68d4875da5c2c05277bcdded INFO: 172.21.0.1:50422 - "POST /node/68d4875da5c2c05277bcdded HTTP/1.0" 202 Accepted 2025-09-27 00:08:06,485 - lava_callback - INFO - Starting async job submit for node 68d4875da5c2c05277bcdded 2025-09-27 00:08:06,511 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.21.0.1:50424 - "GET / HTTP/1.0" 200 OK 2025-09-27 00:08:06,524 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4875da5c2c05277bcdded HTTP/1.1" 200 None 2025-09-27 00:08:06,524 - lava_callback - DEBUG - Processing node 68d4875da5c2c05277bcdded: ltp-ipc 2025-09-27 00:08:06,874 - lava_callback - ERROR - Error processing callback for node 68d4875da5c2c05277bcdded: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:59900 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59916 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55712 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35316 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35328 - "GET / HTTP/1.0" 200 OK 2025-09-27 00:08:38,223 - lava_callback - INFO - Received callback for node 68d53581b70fb2a1e20f219c INFO: 172.21.0.1:35340 - "POST /node/68d53581b70fb2a1e20f219c HTTP/1.0" 202 Accepted 2025-09-27 00:08:38,229 - lava_callback - INFO - Starting async job submit for node 68d53581b70fb2a1e20f219c 2025-09-27 00:08:38,249 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-27 00:08:38,286 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d53581b70fb2a1e20f219c HTTP/1.1" 200 None 2025-09-27 00:08:38,286 - lava_callback - DEBUG - Processing node 68d53581b70fb2a1e20f219c: baseline-arm 2025-09-27 00:08:38,511 - lava_callback - ERROR - Error processing callback for node 68d53581b70fb2a1e20f219c: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:43692 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43702 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36682 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33146 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57704 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57714 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50824 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55726 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55740 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39300 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39304 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47202 - "GET / HTTP/1.0" 200 OK 2025-09-27 00:10:06,400 - lava_callback - INFO - Received callback for node 68d48c00a5c2c05277bce54d INFO: 172.21.0.1:55326 - "POST /node/68d48c00a5c2c05277bce54d HTTP/1.0" 202 Accepted 2025-09-27 00:10:06,406 - lava_callback - INFO - Starting async job submit for node 68d48c00a5c2c05277bce54d 2025-09-27 00:10:06,427 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-27 00:10:06,439 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d48c00a5c2c05277bce54d HTTP/1.1" 200 None 2025-09-27 00:10:06,440 - lava_callback - DEBUG - Processing node 68d48c00a5c2c05277bce54d: baseline-arm-broonie INFO: 172.21.0.1:55334 - "GET / HTTP/1.0" 200 OK 2025-09-27 00:10:06,632 - lava_callback - ERROR - Error processing callback for node 68d48c00a5c2c05277bce54d: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:43638 - "GET /api/metrics HTTP/1.0" 200 OK 2025-09-27 00:10:16,313 - lava_callback - INFO - Received callback for node 68d4c701a5c2c05277bd004b INFO: 172.21.0.1:43644 - "POST /node/68d4c701a5c2c05277bd004b HTTP/1.0" 202 Accepted 2025-09-27 00:10:16,319 - lava_callback - INFO - Starting async job submit for node 68d4c701a5c2c05277bd004b 2025-09-27 00:10:16,340 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-27 00:10:16,353 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4c701a5c2c05277bd004b HTTP/1.1" 200 None 2025-09-27 00:10:16,353 - lava_callback - DEBUG - Processing node 68d4c701a5c2c05277bd004b: baseline-arm64 INFO: 172.21.0.1:43648 - "GET / HTTP/1.0" 200 OK 2025-09-27 00:10:16,680 - lava_callback - ERROR - Error processing callback for node 68d4c701a5c2c05277bd004b: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None 2025-09-27 00:10:23,523 - lava_callback - INFO - Received callback for node 68d4c701a5c2c05277bd004e INFO: 172.21.0.1:50702 - "POST /node/68d4c701a5c2c05277bd004e HTTP/1.0" 202 Accepted 2025-09-27 00:10:23,530 - lava_callback - INFO - Starting async job submit for node 68d4c701a5c2c05277bd004e 2025-09-27 00:10:23,550 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-27 00:10:23,563 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4c701a5c2c05277bd004e HTTP/1.1" 200 None 2025-09-27 00:10:23,563 - lava_callback - DEBUG - Processing node 68d4c701a5c2c05277bd004e: baseline-arm64 2025-09-27 00:10:23,860 - lava_callback - ERROR - Error processing callback for node 68d4c701a5c2c05277bd004e: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:50706 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59480 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59490 - "GET / HTTP/1.0" 200 OK 2025-09-27 00:10:42,354 - lava_callback - INFO - Received callback for node 68d53581b70fb2a1e20f219f INFO: 172.21.0.1:49378 - "POST /node/68d53581b70fb2a1e20f219f HTTP/1.0" 202 Accepted 2025-09-27 00:10:42,360 - lava_callback - INFO - Starting async job submit for node 68d53581b70fb2a1e20f219f 2025-09-27 00:10:42,381 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-27 00:10:42,393 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d53581b70fb2a1e20f219f HTTP/1.1" 200 None 2025-09-27 00:10:42,394 - lava_callback - DEBUG - Processing node 68d53581b70fb2a1e20f219f: baseline-arm 2025-09-27 00:10:42,620 - lava_callback - ERROR - Error processing callback for node 68d53581b70fb2a1e20f219f: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:49380 - "GET / HTTP/1.0" 200 OK 2025-09-27 00:10:47,727 - lava_callback - INFO - Received callback for node 68d4c704a5c2c05277bd0051 INFO: 172.21.0.1:49382 - "POST /node/68d4c704a5c2c05277bd0051 HTTP/1.0" 202 Accepted 2025-09-27 00:10:47,735 - lava_callback - INFO - Starting async job submit for node 68d4c704a5c2c05277bd0051 2025-09-27 00:10:47,770 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-27 00:10:47,784 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4c704a5c2c05277bd0051 HTTP/1.1" 200 None 2025-09-27 00:10:47,784 - lava_callback - DEBUG - Processing node 68d4c704a5c2c05277bd0051: baseline-arm64 INFO: 172.21.0.1:49396 - "GET / HTTP/1.0" 200 OK 2025-09-27 00:10:49,136 - lava_callback - ERROR - Error processing callback for node 68d4c704a5c2c05277bd0051: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:51086 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51340 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50310 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50320 - "GET / HTTP/1.0" 200 OK 2025-09-27 00:11:23,481 - lava_callback - INFO - Received callback for node 68d4c705a5c2c05277bd0054 INFO: 172.21.0.1:33552 - "POST /node/68d4c705a5c2c05277bd0054 HTTP/1.0" 202 Accepted 2025-09-27 00:11:23,487 - lava_callback - INFO - Starting async job submit for node 68d4c705a5c2c05277bd0054 2025-09-27 00:11:23,508 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-27 00:11:23,521 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4c705a5c2c05277bd0054 HTTP/1.1" 200 None 2025-09-27 00:11:23,521 - lava_callback - DEBUG - Processing node 68d4c705a5c2c05277bd0054: baseline-arm64 2025-09-27 00:11:23,812 - lava_callback - ERROR - Error processing callback for node 68d4c705a5c2c05277bd0054: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:33558 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56186 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56198 - "GET / HTTP/1.0" 200 OK 2025-09-27 00:11:40,400 - lava_callback - INFO - Received callback for node 68d4c707a5c2c05277bd0057 INFO: 172.21.0.1:50942 - "POST /node/68d4c707a5c2c05277bd0057 HTTP/1.0" 202 Accepted 2025-09-27 00:11:40,406 - lava_callback - INFO - Starting async job submit for node 68d4c707a5c2c05277bd0057 2025-09-27 00:11:40,427 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-27 00:11:40,458 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4c707a5c2c05277bd0057 HTTP/1.1" 200 None 2025-09-27 00:11:40,458 - lava_callback - DEBUG - Processing node 68d4c707a5c2c05277bd0057: baseline-arm64 2025-09-27 00:11:40,778 - lava_callback - ERROR - Error processing callback for node 68d4c707a5c2c05277bd0057: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:50946 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50960 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36680 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51824 - "GET / HTTP/1.0" 200 OK 2025-09-27 00:12:14,555 - lava_callback - INFO - Received callback for node 68d4c708a5c2c05277bd005a INFO: 172.21.0.1:46014 - "POST /node/68d4c708a5c2c05277bd005a HTTP/1.0" 202 Accepted 2025-09-27 00:12:14,561 - lava_callback - INFO - Starting async job submit for node 68d4c708a5c2c05277bd005a 2025-09-27 00:12:14,582 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-27 00:12:14,616 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4c708a5c2c05277bd005a HTTP/1.1" 200 None 2025-09-27 00:12:14,617 - lava_callback - DEBUG - Processing node 68d4c708a5c2c05277bd005a: baseline-arm64 2025-09-27 00:12:14,909 - lava_callback - ERROR - Error processing callback for node 68d4c708a5c2c05277bd005a: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:46024 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46034 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34594 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55830 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55836 - "GET / HTTP/1.0" 200 OK 2025-09-27 00:12:43,450 - lava_callback - INFO - Received callback for node 68d53584b70fb2a1e20f21a2 INFO: 172.21.0.1:39688 - "POST /node/68d53584b70fb2a1e20f21a2 HTTP/1.0" 202 Accepted 2025-09-27 00:12:43,456 - lava_callback - INFO - Starting async job submit for node 68d53584b70fb2a1e20f21a2 2025-09-27 00:12:43,476 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-27 00:12:43,494 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d53584b70fb2a1e20f21a2 HTTP/1.1" 200 None 2025-09-27 00:12:43,494 - lava_callback - DEBUG - Processing node 68d53584b70fb2a1e20f21a2: baseline-arm 2025-09-27 00:12:43,722 - lava_callback - ERROR - Error processing callback for node 68d53584b70fb2a1e20f21a2: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None 2025-09-27 00:12:43,871 - lava_callback - INFO - Received callback for node 68d4c70ba5c2c05277bd005d INFO: 172.21.0.1:39694 - "POST /node/68d4c70ba5c2c05277bd005d HTTP/1.0" 202 Accepted 2025-09-27 00:12:43,877 - lava_callback - INFO - Starting async job submit for node 68d4c70ba5c2c05277bd005d 2025-09-27 00:12:43,898 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-27 00:12:43,911 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4c70ba5c2c05277bd005d HTTP/1.1" 200 None 2025-09-27 00:12:43,911 - lava_callback - DEBUG - Processing node 68d4c70ba5c2c05277bd005d: baseline-arm64 2025-09-27 00:12:44,230 - lava_callback - ERROR - Error processing callback for node 68d4c70ba5c2c05277bd005d: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:39708 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37844 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37856 - "GET / HTTP/1.0" 200 OK 2025-09-27 00:13:00,096 - lava_callback - INFO - Received callback for node 68d4c70ba5c2c05277bd0060 INFO: 172.21.0.1:43118 - "POST /node/68d4c70ba5c2c05277bd0060 HTTP/1.0" 202 Accepted 2025-09-27 00:13:00,102 - lava_callback - INFO - Starting async job submit for node 68d4c70ba5c2c05277bd0060 2025-09-27 00:13:00,122 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-27 00:13:00,136 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4c70ba5c2c05277bd0060 HTTP/1.1" 200 None 2025-09-27 00:13:00,137 - lava_callback - DEBUG - Processing node 68d4c70ba5c2c05277bd0060: baseline-arm64 2025-09-27 00:13:00,453 - lava_callback - ERROR - Error processing callback for node 68d4c70ba5c2c05277bd0060: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:43134 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44722 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44724 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50230 - "GET / HTTP/1.0" 200 OK 2025-09-27 00:13:34,746 - lava_callback - INFO - Received callback for node 68d4c77ea5c2c05277bd014a INFO: 172.21.0.1:34920 - "POST /node/68d4c77ea5c2c05277bd014a HTTP/1.0" 202 Accepted 2025-09-27 00:13:34,752 - lava_callback - INFO - Starting async job submit for node 68d4c77ea5c2c05277bd014a 2025-09-27 00:13:34,773 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-27 00:13:34,814 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4c77ea5c2c05277bd014a HTTP/1.1" 200 None 2025-09-27 00:13:34,815 - lava_callback - DEBUG - Processing node 68d4c77ea5c2c05277bd014a: baseline-arm64 2025-09-27 00:13:35,134 - lava_callback - ERROR - Error processing callback for node 68d4c77ea5c2c05277bd014a: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:34922 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34926 - "GET / HTTP/1.0" 200 OK 2025-09-27 00:13:43,467 - lava_callback - INFO - Received callback for node 68d4c77fa5c2c05277bd014d INFO: 172.21.0.1:39110 - "POST /node/68d4c77fa5c2c05277bd014d HTTP/1.0" 202 Accepted 2025-09-27 00:13:43,473 - lava_callback - INFO - Starting async job submit for node 68d4c77fa5c2c05277bd014d 2025-09-27 00:13:43,493 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-27 00:13:43,506 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4c77fa5c2c05277bd014d HTTP/1.1" 200 None 2025-09-27 00:13:43,506 - lava_callback - DEBUG - Processing node 68d4c77fa5c2c05277bd014d: baseline-arm64 2025-09-27 00:13:43,799 - lava_callback - ERROR - Error processing callback for node 68d4c77fa5c2c05277bd014d: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:39112 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52670 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52686 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34910 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50956 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50972 - "GET / HTTP/1.0" 200 OK 2025-09-27 00:14:17,942 - lava_callback - INFO - Received callback for node 68d4c781a5c2c05277bd0150 INFO: 172.21.0.1:50978 - "POST /node/68d4c781a5c2c05277bd0150 HTTP/1.0" 202 Accepted 2025-09-27 00:14:17,949 - lava_callback - INFO - Starting async job submit for node 68d4c781a5c2c05277bd0150 2025-09-27 00:14:17,969 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-27 00:14:17,984 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4c781a5c2c05277bd0150 HTTP/1.1" 200 None 2025-09-27 00:14:17,984 - lava_callback - DEBUG - Processing node 68d4c781a5c2c05277bd0150: baseline-arm64 2025-09-27 00:14:18,288 - lava_callback - ERROR - Error processing callback for node 68d4c781a5c2c05277bd0150: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:44358 - "GET / HTTP/1.0" 200 OK 2025-09-27 00:14:26,763 - lava_callback - INFO - Received callback for node 68d48c0ca5c2c05277bce573 INFO: 172.21.0.1:44368 - "POST /node/68d48c0ca5c2c05277bce573 HTTP/1.0" 202 Accepted 2025-09-27 00:14:26,770 - lava_callback - INFO - Starting async job submit for node 68d48c0ca5c2c05277bce573 2025-09-27 00:14:26,795 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-27 00:14:26,808 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d48c0ca5c2c05277bce573 HTTP/1.1" 200 None 2025-09-27 00:14:26,808 - lava_callback - DEBUG - Processing node 68d48c0ca5c2c05277bce573: ltp-cap-bounds 2025-09-27 00:14:27,163 - lava_callback - ERROR - Error processing callback for node 68d48c0ca5c2c05277bce573: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None 2025-09-27 00:14:27,232 - lava_callback - INFO - Received callback for node 68d48749a5c2c05277bcdddb INFO: 172.21.0.1:44382 - "POST /node/68d48749a5c2c05277bcdddb HTTP/1.0" 202 Accepted 2025-09-27 00:14:27,240 - lava_callback - INFO - Starting async job submit for node 68d48749a5c2c05277bcdddb 2025-09-27 00:14:27,345 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-27 00:14:27,377 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d48749a5c2c05277bcdddb HTTP/1.1" 200 None 2025-09-27 00:14:27,377 - lava_callback - DEBUG - Processing node 68d48749a5c2c05277bcdddb: ltp-cve 2025-09-27 00:14:28,095 - lava_callback - ERROR - Error processing callback for node 68d48749a5c2c05277bcdddb: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:51602 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51616 - "GET / HTTP/1.0" 200 OK 2025-09-27 00:14:44,716 - lava_callback - INFO - Received callback for node 68d53584b70fb2a1e20f21a5 INFO: 172.21.0.1:52210 - "POST /node/68d53584b70fb2a1e20f21a5 HTTP/1.0" 202 Accepted 2025-09-27 00:14:44,726 - lava_callback - INFO - Starting async job submit for node 68d53584b70fb2a1e20f21a5 2025-09-27 00:14:44,757 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-27 00:14:44,769 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d53584b70fb2a1e20f21a5 HTTP/1.1" 200 None 2025-09-27 00:14:44,770 - lava_callback - DEBUG - Processing node 68d53584b70fb2a1e20f21a5: baseline-arm 2025-09-27 00:14:45,008 - lava_callback - ERROR - Error processing callback for node 68d53584b70fb2a1e20f21a5: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None 2025-09-27 00:14:46,341 - lava_callback - INFO - Received callback for node 68d4c782a5c2c05277bd0153 INFO: 172.21.0.1:52214 - "POST /node/68d4c782a5c2c05277bd0153 HTTP/1.0" 202 Accepted 2025-09-27 00:14:46,347 - lava_callback - INFO - Starting async job submit for node 68d4c782a5c2c05277bd0153 2025-09-27 00:14:46,367 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-27 00:14:46,380 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4c782a5c2c05277bd0153 HTTP/1.1" 200 None 2025-09-27 00:14:46,380 - lava_callback - DEBUG - Processing node 68d4c782a5c2c05277bd0153: baseline-arm64 INFO: 172.21.0.1:52222 - "GET / HTTP/1.0" 200 OK 2025-09-27 00:14:46,674 - lava_callback - ERROR - Error processing callback for node 68d4c782a5c2c05277bd0153: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None 2025-09-27 00:14:46,932 - lava_callback - INFO - Received callback for node 68d48761a5c2c05277bcddf0 INFO: 172.21.0.1:52234 - "POST /node/68d48761a5c2c05277bcddf0 HTTP/1.0" 202 Accepted 2025-09-27 00:14:46,939 - lava_callback - INFO - Starting async job submit for node 68d48761a5c2c05277bcddf0 2025-09-27 00:14:46,978 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-27 00:14:46,990 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d48761a5c2c05277bcddf0 HTTP/1.1" 200 None 2025-09-27 00:14:46,991 - lava_callback - DEBUG - Processing node 68d48761a5c2c05277bcddf0: ltp-pty 2025-09-27 00:14:47,373 - lava_callback - ERROR - Error processing callback for node 68d48761a5c2c05277bcddf0: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:36766 - "GET / HTTP/1.0" 200 OK 2025-09-27 00:14:56,401 - lava_callback - INFO - Received callback for node 68d4c784a5c2c05277bd0156 INFO: 172.21.0.1:36782 - "POST /node/68d4c784a5c2c05277bd0156 HTTP/1.0" 202 Accepted 2025-09-27 00:14:56,407 - lava_callback - INFO - Starting async job submit for node 68d4c784a5c2c05277bd0156 2025-09-27 00:14:56,427 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-27 00:14:56,457 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4c784a5c2c05277bd0156 HTTP/1.1" 200 None 2025-09-27 00:14:56,458 - lava_callback - DEBUG - Processing node 68d4c784a5c2c05277bd0156: baseline-arm64 INFO: 172.21.0.1:36798 - "GET / HTTP/1.0" 200 OK 2025-09-27 00:14:56,753 - lava_callback - ERROR - Error processing callback for node 68d4c784a5c2c05277bd0156: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:35124 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56352 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56366 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43946 - "GET / HTTP/1.0" 200 OK 2025-09-27 00:15:34,235 - lava_callback - INFO - Received callback for node 68d4c785a5c2c05277bd0159 INFO: 172.21.0.1:58758 - "POST /node/68d4c785a5c2c05277bd0159 HTTP/1.0" 202 Accepted 2025-09-27 00:15:34,241 - lava_callback - INFO - Starting async job submit for node 68d4c785a5c2c05277bd0159 2025-09-27 00:15:34,262 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-27 00:15:34,287 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4c785a5c2c05277bd0159 HTTP/1.1" 200 None 2025-09-27 00:15:34,287 - lava_callback - DEBUG - Processing node 68d4c785a5c2c05277bd0159: baseline-arm64 2025-09-27 00:15:34,602 - lava_callback - ERROR - Error processing callback for node 68d4c785a5c2c05277bd0159: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:58764 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58772 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47254 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37978 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37980 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38358 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45214 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45230 - "GET / HTTP/1.0" 200 OK 2025-09-27 00:16:19,157 - lava_callback - INFO - Received callback for node 68d4c788a5c2c05277bd015c INFO: 172.21.0.1:45232 - "POST /node/68d4c788a5c2c05277bd015c HTTP/1.0" 202 Accepted 2025-09-27 00:16:19,163 - lava_callback - INFO - Starting async job submit for node 68d4c788a5c2c05277bd015c 2025-09-27 00:16:19,183 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-27 00:16:19,200 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4c788a5c2c05277bd015c HTTP/1.1" 200 None 2025-09-27 00:16:19,200 - lava_callback - DEBUG - Processing node 68d4c788a5c2c05277bd015c: baseline-arm64 2025-09-27 00:16:19,489 - lava_callback - ERROR - Error processing callback for node 68d4c788a5c2c05277bd015c: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None 2025-09-27 00:16:26,525 - lava_callback - INFO - Received callback for node 68d4c788a5c2c05277bd015f INFO: 172.21.0.1:40182 - "POST /node/68d4c788a5c2c05277bd015f HTTP/1.0" 202 Accepted 2025-09-27 00:16:26,534 - lava_callback - INFO - Starting async job submit for node 68d4c788a5c2c05277bd015f INFO: 172.21.0.1:40186 - "GET / HTTP/1.0" 200 OK 2025-09-27 00:16:26,555 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-27 00:16:26,579 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4c788a5c2c05277bd015f HTTP/1.1" 200 None 2025-09-27 00:16:26,580 - lava_callback - DEBUG - Processing node 68d4c788a5c2c05277bd015f: baseline-arm64 2025-09-27 00:16:26,902 - lava_callback - ERROR - Error processing callback for node 68d4c788a5c2c05277bd015f: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:38524 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38532 - "GET / HTTP/1.0" 200 OK 2025-09-27 00:16:41,564 - lava_callback - INFO - Received callback for node 68d535c8b70fb2a1e20f2213 INFO: 172.21.0.1:58126 - "POST /node/68d535c8b70fb2a1e20f2213 HTTP/1.0" 202 Accepted 2025-09-27 00:16:41,570 - lava_callback - INFO - Starting async job submit for node 68d535c8b70fb2a1e20f2213 2025-09-27 00:16:41,590 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-27 00:16:41,624 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d535c8b70fb2a1e20f2213 HTTP/1.1" 200 None 2025-09-27 00:16:41,624 - lava_callback - DEBUG - Processing node 68d535c8b70fb2a1e20f2213: baseline-arm 2025-09-27 00:16:41,850 - lava_callback - ERROR - Error processing callback for node 68d535c8b70fb2a1e20f2213: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:58130 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34602 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34618 - "GET / HTTP/1.0" 200 OK 2025-09-27 00:17:01,426 - lava_callback - INFO - Received callback for node 68d4c82fa5c2c05277bd025f INFO: 172.21.0.1:52976 - "POST /node/68d4c82fa5c2c05277bd025f HTTP/1.0" 202 Accepted 2025-09-27 00:17:01,432 - lava_callback - INFO - Starting async job submit for node 68d4c82fa5c2c05277bd025f 2025-09-27 00:17:01,455 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-27 00:17:01,469 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4c82fa5c2c05277bd025f HTTP/1.1" 200 None 2025-09-27 00:17:01,469 - lava_callback - DEBUG - Processing node 68d4c82fa5c2c05277bd025f: baseline-arm64 2025-09-27 00:17:01,922 - lava_callback - ERROR - Error processing callback for node 68d4c82fa5c2c05277bd025f: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:52990 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55328 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55344 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53262 - "GET / HTTP/1.0" 200 OK 2025-09-27 00:17:35,556 - lava_callback - INFO - Received callback for node 68d4c831a5c2c05277bd0262 INFO: 172.21.0.1:35430 - "POST /node/68d4c831a5c2c05277bd0262 HTTP/1.0" 202 Accepted 2025-09-27 00:17:35,562 - lava_callback - INFO - Starting async job submit for node 68d4c831a5c2c05277bd0262 2025-09-27 00:17:35,588 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-27 00:17:35,621 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4c831a5c2c05277bd0262 HTTP/1.1" 200 None 2025-09-27 00:17:35,621 - lava_callback - DEBUG - Processing node 68d4c831a5c2c05277bd0262: baseline-arm64 2025-09-27 00:17:36,042 - lava_callback - ERROR - Error processing callback for node 68d4c831a5c2c05277bd0262: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:35444 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35450 - "GET / HTTP/1.0" 200 OK 2025-09-27 00:17:44,401 - lava_callback - INFO - Received callback for node 68d4c834a5c2c05277bd0265 INFO: 172.21.0.1:54982 - "POST /node/68d4c834a5c2c05277bd0265 HTTP/1.0" 202 Accepted 2025-09-27 00:17:44,410 - lava_callback - INFO - Starting async job submit for node 68d4c834a5c2c05277bd0265 2025-09-27 00:17:44,441 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-27 00:17:44,456 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d4c834a5c2c05277bd0265 HTTP/1.1" 200 None 2025-09-27 00:17:44,456 - lava_callback - DEBUG - Processing node 68d4c834a5c2c05277bd0265: baseline-arm64 2025-09-27 00:17:44,971 - lava_callback - ERROR - Error processing callback for node 68d4c834a5c2c05277bd0265: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:54988 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37556 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37566 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54622 - "GET / HTTP/1.0" 200 OK INFO: Shutting down INFO: Waiting for application shutdown. INFO: Application shutdown complete. INFO: Finished server process [1]